ReTIN: Indexing Schema for Soft Real-Time Data Streams

The paper deals with the indexing of a complex type data stream where a portion of the stream that represents the content of its sliding window is stored in a database. The processing of this data must often meet some real-time constraints. We present here a novel indexing schema/framework referred to as ReTIN (Real-Time INdexing), the objective of which is to allow indexing of complex data arriving as a stream to a database with respect to soft real-time constraints for the maximum duration of insert and select operations.
In contrast to hard real-time constraints, the softness means that constraints violations are allowed but their number must be minimized. In ReTIN, soft real-time constraints are met with some level of confidence. The basic idea of ReTIN is a combination of sequential access to the most recent data to less recent data that has been indexed and stored in the database. The collection of statistics makes balancing the indexed and unindexed parts of the database efficient. We have implemented ReTIN PostgreSQL DBMS and its GIN index to store and index data. Experimental results presented in the paper demonstrate some properties and advantages of our approach.