Skip to main content
Version: 0.16

Source reader

A source reader conducts the following 3 roles:

  1. keeping "connection" (network connection, file handle) to a foreign source.
  2. reading arbitrary data from the foreign source.
  3. converting the data from into SpringQL's rows.
  4. pushing the rows into a source stream.

Let's look at an example source reader:

Source reader definition
CREATE SOURCE READER source_reader1
FOR source_stream1
TYPE NET_SERVER OPTIONS (
PROTOCOL 'TCP',
PORT '12345'
);

This statement defines the following source reader (the edge in the diagram).

The source_reader1 does the following:

  1. Listen to the TCP port 12345 on the localhost.
  2. Accept an connection from a client.
  3. Read a JSON object line like: {"ts": "2020-01-01 00:00:00.000000000", "c1": 1}
  4. Convert the JSON object into a SpringQL row.
  5. Push the row into the source_stream1, and repeat from 3.

Source reader types and options

A source reader has a name, a type, and options. A type of a source reader determines the available options.

Here lists the currently available source readers.

IN_MEMORY_QUEUE

Reads data from an in-memory queue allocated in the application's memory space linked to SpringQL. The application pushes rows from the queue using *push() API.

See: Client API.

  • Options
    • (required) NAME: queue's name.

Data format

*push() API accepts SpringRow.

See Client API to learn how to create SpringRows from client applications.

NET_CLIENT

Connects to a foreign source via TCP.

  • Options
    • (required) PROTOCOL: must be TCP.
    • (required) REMOTE_HOST: host name or IP address of the remote host.
    • (required) REMOTE_PORT: port number of the remote host.

Data format

Currently, only 1-line JSON objects are supported as an input datum from a foreign source.

JSON object convertible to a SpringQL's row
{ "column_name1": <value convertible to source stream's data type>, "column_name2": <value2>, ... }

Say, you have the following source stream:

Source stream
CREATE SOURCE STREAM ss (
ts TIMESTAMP NOT NULL ROWTIME,
c1 INTEGER NOT NULL
);

Then a valid JSON objects for an input into source readers are:

Valid JSONs
{ "ts": "2020-01-01 00:00:00.000000000", "c1": 1 }
{ "c1": 42, "ts": "1970-01-01 00:00:00.000000000" }

The following ones are invalid:

Invalid JSONs
# column name
{ "timestamp": "2020-01-01 00:00:00.000000000", "c1": 1 }

# timestamp format must currently have 9-digit fractional parts
{ "ts": "2020-01-01 00:00:00", "c1": 1 }

# string cannot be converted into INTEGER type
{ "ts": "2020-01-01 00:00:00.000000000", "c1": "1" }

NET_SERVER

Accepts to a foreign source's connection via TCP.

  • Options
    • (required) PROTOCOL: must be TCP.
    • (required) PORT: port number to listen.

Data format

The same as NET_CLIENT.

CAN

Read CAN frame data from a SocketCAN interface.

  • Options
    • (required) INTERFACE: SocketCAN interface name (e.g. can0, vcan0, slcan0).

Data format

CAN frame.