DataReader is a plugin that allows for reading feeds of information from several different types of sources of information. The plugin supports reading Excel files, databases using SQL, feeds using XML and JSON. The plugin also supports different types of outputs. The plugin shows different arguments depending on the inputs and the outputs used:

images/download/attachments/41810741/plugins_datareader.png

Functionality

All of the different sources of information behave the same way: They all search for records and translate the records fields to a certain format according to the requested output type. In the case of DataPool output types, the plugin converts each incoming record to a DataPool command and executes it. In the case of Shared Memory, the plugin creates a character separated string of fields and pushes to shared memory.

Input Type - Excel

In this mode the plugin reads a sheet from an Excel file. The parameters in this mode are:

File Name: Determines the name of the file to be read.
Table/Sheet: Determines the name of the sheet to be read from within the file.
Key: Determines the name of the column in the Excel file from which the keys to the DataTable should be taken.
From row: Determines the row number in the Excel sheet from where to start reading the records.
Number of rows: Determines the maximum number of records to read, starting from the row stated in From row.
Fields to Read: Specifies by name the columns to read.
DP Field Name: Specifies the field name in DataPool where to write the output.
Field is a: Specifies the type of the target.
Post Update Action: Specifies commands (either DataPool or Viz commands) to be executed right after the data is read and sent to the target.
Show Data: Dumps the data the plugin reads to a DataPool variable called READER_CONSOLE when set to On. This is extremely useful for development and debugging purposes. This dump includes error messages too.
Return Variables Prefix: Specifies a prefix to add to the name of the DataPool variables that the plugin uses. For example, if the prefix is STV_, then the plugin dumps the debug information to STV_READER_CONSOLE.
Load Automatically: Reads the information automatically every certain amount of time. This amount of time is defined in Automatic Load Period (in seconds).
Automatic Load Period (in seconds): Specifies the period of time between consecutive automatic reads.
Load: Invokes the action of reading the data.

Example

The following is a typical example of a table in an Excel sheet we would be interested in reading from using DataReader.

images/download/attachments/30910247/plugins_datareader-example.png
In this case, if the parameters are configured as:

File name = C:\temp\Indexes.xlsx
Table/Sheet = Indexes$
Key = Stocks
From Row = 5
Number of Rows = 3
Fields To Read = Stocks, Prices
DP Field Name = Indexes
Data Field = DataTable
then the plugin dumps to DataPool the following command:

Indexes[EURO STOXX 50 ]={    
	{
		Stocks=""EURO STOXX 50 "";
		Prices=""3131.39"";
		Volumes=""-37.84 (-1.19%)"";
	};
}; 
 
Indexes[CAC 40 ]={    
	{
		Stocks=""CAC 40 "";
		Prices=""4265.04"";
		Volumes=""-47.26 (-1.10%)"";
	};
}; 
 
Indexes[S&P TSX ]={    
	{
		Stocks=""S&P TSX "";
		Prices=""15476.77"";
		Volumes=""-48.05 (-0.31%)"";
	};
};  
 
NUM_RECORDS=3;

Note that the Key parameter is used to obtain the column from which the indexes of the table.

In case the Data Field is of type DataArray:

Indexes[0]={

{

Stocks=""EURO STOXX 50 "";

Prices=""3131.39"";

Volumes=""-37.84 (-1.19%)"";

};

};

Indexes[1]={

{

Stocks=""CAC 40 "";

Prices=""4265.04"";

Volumes=""-47.26 (-1.10%)"";

};

};

Indexes[2]={

{

Stocks=""S&P TSX "";

Prices=""15476.77"";

Volumes=""-48.05 (-0.31%)"";

};

};

NUM_RECORDS=3;
In case the Data Field is of type DataStructure, the Key parameter is used as the name of the column from which to take the names of the DataPool field:

EURO STOXX 50={

Stocks=""EURO STOXX 50 "";

Prices=""3131.39"";

Volumes=""-37.84 (-1.19%)"";

};

CAC 40={

Stocks=""CAC 40 "";

Prices=""4265.04"";

Volumes=""-47.26 (-1.10%)"";

};

S&P TSX={

Stocks=""S&P TSX "";

Prices=""15476.77"";

Volumes=""-48.05 (-0.31%)"";

};

NUM_RECORDS=3;
In case the DataField is of type DataFieldArray, then the output looks like:

Indexes[0-8]=””EURO STOXX 50 "",""3131.39"",""-37.84 (-1.19%)"",""CAC 40 ",""4265.04"",""-47.26 (-1.10%)"",""S&P TSX "",""15476.77"",""-48.05 (-0.31%)"";

NUM_RECORDS=9;
If the user selects to output the data to Shared Memory then the following value is dumped into the selected variable:

EURO STOXX 50|3131.39|-37.84 (-1.19%)|CAC 40|4265.04|-47.26 (-1.10%)|S&P TSX|15476.77|-48.05 (-0.31%)

The output is a string result of the concatenation of all the values. The separator is by default a pipe character (|), but it can be changed.

Input Type - SQL

In this mode, the plugin reads a sheet from a database that supports SQL. The parameters in this mode are:

Connection String: Connects to the database using the connection string. Each type of database has a different connection string. A good reference on connection strings for several different databases can be found in http://www.connectionstrings.com/.
SQL Query: Queries the database using SQL command.
Key: Determines the name of the column in the read query or table from which the keys to the DataTable should be taken.
From row: Determines the row number in the query/table from where to start reading the records.
Number of rows: Determines the maximum number of records to read, starting from the row stated in From row.
Fields to Read: Specifies by name the columns to read.
DP Field Name: Specifies the field name in DataPool where to write the output.
Field is a: Specifies the type of the target.
Post Update Action: Specifies commands (either DataPool or Viz commands) to be executed right after the data is read and sent to the target.
Show Data: Dumps the data the plugin reads to a DataPool variable called READER_CONSOLE when set to On. This is extremely useful for development and debugging purposes. This dump includes error messages too.
Return Variables Prefix: Specifies a prefix to add to the name of the DataPool variables that the plugin uses. For example, if the prefix is STV_, then the plugin dumps the debug information to STV_READER_CONSOLE.
Load Automatically: Reads the information automatically every certain amount of time. This amount of time is defined in Automatic Load Period (in seconds).
Automatic Load Period (in seconds): Specifies the period of time between consecutive automatic reads.
Load: Invokes the action of reading the data. The different types of outputs are exactly as in the case of Excel.

Input Type - XML

This mode allows for reading XML feeds. These can be read from a remote server or from a local file.

These can be read from a remote server or from a local file.

The parameters in this mode are:

File Name: Shows the full name of a file or a URI to a remote file to be read.
Use Authentication: Allows entering a user name and password In the case the File Name is a URI and the remote server requires Basic Authentication.
Avoid Cache: Avoids using cache mechanisms that avoid the refresh of data changing in the server side in the case of a remote file. This is useful when reading information that changes. The caching mechanism avoids the plugin to get the updates. When “Avoid Cache” is on the plugin gets the freshest information.
XPath: States what records to search for. For an explanation of the XPath syntax please refer to http://www.w3.org/TR/xpath/. For examples of how to use XPath please refer to https://msdn.microsoft.com/en-us/library/ms256086.
Namespaces: Specifies namespaces for use in XPath expressions when it is necessary to define new namespaces externally. Namespaces are defined in the XML style, as a space-separated list of namespace declaration attributes. You can use this property to set the default namespace as well. An example of the definition of namespaces could be: xmlns:na='http://myserver.com' xmlns:nb='http://yourserver.com'
Key: Determines the name of the field in the read records from which the keys to the DataTable should be taken.
From row: Determines he row number in the read record from where to start reading the records.
Number of rows: Determines the maximum number of records to read, starting from the row stated in From row.
Fields to Read: Specifies by name the fields to read from the records.
DP Field Name: Specifies the field name in DataPool where to write the output.
Field is a: Specifies the type of the target.
Post Update Action: Specifies commands (either DataPool or Viz commands) to be executed right after the data is read and sent to the target.
Show Data: Dumps the data the plugin reads to a DataPool variable called READER_CONSOLE when set to ON. This is extremely useful for development and debugging purposes. This includes error messages too.
Return VariablesPrefix: Specifies a prefix to add to the name of the DataPool variables that the plugin uses. For example, if the prefix is STV_, then the plugin dumps the debug information to STV_READER_CONSOLE.
Load Automatically: Reads the information automatically every certain amount of time. This amount of time is defined in Automatic Load Period (in seconds).
Automatic Load Period (in seconds): Specifies the period of time between consecutive automatic reads.
Load: Invokes the action of reading the data.

Input Type - JSON

This mode allows for reading JSON feeds. These can be read from a remote server or from a local file.