Source reader
A source reader conducts the following 3 roles:
- keeping "connection" (network connection, file handle) to a foreign source.
- reading arbitrary data from the foreign source.
- converting the data from into SpringQL's rows.
- pushing the rows into a source stream.
Let's look at an example source reader:
CREATE SOURCE READER source_reader1
FOR source_stream1
TYPE NET_SERVER OPTIONS (
PROTOCOL 'TCP',
PORT '12345'
);
This statement defines the following source reader (the edge in the diagram).
The source_reader1
does the following:
- Listen to the TCP port
12345
on the localhost. - Accept an connection from a client.
- Read a JSON object line like:
{"ts": "2020-01-01 00:00:00.000000000", "c1": 1}
- Convert the JSON object into a SpringQL row.
- Push the row into the
source_stream1
, and repeat from 3.
Source reader types and options
A source reader has a name, a type, and options. A type of a source reader determines the available options.
Here lists the currently available source readers.
IN_MEMORY_QUEUE
Reads data from an in-memory queue allocated in the application's memory space linked to SpringQL.
The application pushes rows from the queue using *push()
API.
See: Client API.
- Options
- (required)
NAME
: queue's name.
- (required)
Data format
*push()
API accepts SpringRow
.
See Client API to learn how to create SpringRow
s from client applications.
NET_CLIENT
Connects to a foreign source via TCP.
- Options
- (required)
PROTOCOL
: must beTCP
. - (required)
REMOTE_HOST
: host name or IP address of the remote host. - (required)
REMOTE_PORT
: port number of the remote host.
- (required)
Data format
Currently, only 1-line JSON objects are supported as an input datum from a foreign source.
{ "column_name1": <value convertible to source stream's data type>, "column_name2": <value2>, ... }
Say, you have the following source stream:
CREATE SOURCE STREAM ss (
ts TIMESTAMP NOT NULL ROWTIME,
c1 INTEGER NOT NULL
);
Then a valid JSON objects for an input into source readers are:
{ "ts": "2020-01-01 00:00:00.000000000", "c1": 1 }
{ "c1": 42, "ts": "1970-01-01 00:00:00.000000000" }
The following ones are invalid:
# column name
{ "timestamp": "2020-01-01 00:00:00.000000000", "c1": 1 }
# timestamp format must currently have 9-digit fractional parts
{ "ts": "2020-01-01 00:00:00", "c1": 1 }
# string cannot be converted into INTEGER type
{ "ts": "2020-01-01 00:00:00.000000000", "c1": "1" }
NET_SERVER
Accepts to a foreign source's connection via TCP.
- Options
- (required)
PROTOCOL
: must beTCP
. - (required)
PORT
: port number to listen.
- (required)
Data format
The same as NET_CLIENT
.
CAN
Read CAN frame data from a SocketCAN interface.
- Options
- (required)
INTERFACE
: SocketCAN interface name (e.g.can0
,vcan0
,slcan0
).
- (required)
Data format
CAN frame.