|
The growth of
electronic commerce and the widespread use of sensor networks have
created the demand for online processing and monitoring
applications, creating a new class of query processing over
continuously generated data streams. Traditional database
techniques, which assume data to be bounded as well as statically
stored and indexed, are largely incapable of handling these new
applications, and so Continuous Query (CQ) Systems have appeared.
CQ systems must be adaptive to properly manage their available
resources in the face of data streams with widely varying arrival
rates, and a constantly changing set of standing user queries that
must be processed. Not a priori optimization algorithm can be
successful given such variability. The CAPE project aims to
propose a novel architecture for a CQ system that (1) incorporates
adaptability at all levels of query processing; and (2)
incorporates a dynamic metadata model used to help optimize all
levels of query processing.
The CAPE project aims to provide
novel techniques for processing large numbers of concurrent
continuous queries with required Quality of Service (QoS). Because
of the dynamic nature of query registration and stream behavior,
we are designing heterogeneous-grained adaptivity for CAPE and
exploits dynamic metadata at all levels in continuous query
processing, including the query operator execution, memory
allocation, operator scheduling, query plan structuring and query
plan distribution among multiple machines. We will (1) design an
extensible dynamic metadata model; (2) design adaptive algorithms
for use in each layer of query processing to exploit available
metadata; (3) develop QoS specification models for capturing
resource usage; (4) incorporate a hierarchical interaction model
for coordinating the adaptation at different levels within the CQ
system; and (5) design a family of metadata-exploiting
optimization techniques.
System Architecture [ppt]
|
|