The growth of electronic commerce and the widespread use of sensor networks have created the demand for online processing and monitoring applications, creating a new class of query processing over continuously generated data streams. Traditional database techniques, which assume data to be bounded as well as statically stored and indexed, are largely incapable of handling these new applications, and so Continuous Query (CQ) Systems have appeared. CQ systems must be adaptive to properly manage their available resources in the face of data streams with widely varying arrival rates, and a constantly changing set of standing user queries that must be processed. Not a priori optimization algorithm can be successful given such variability. The CAPE project aims to propose a novel architecture for a CQ system that (1) incorporates adaptability at all levels of query processing; and (2) incorporates a dynamic metadata model used to help optimize all levels of query processing.

The CAPE project aims to provide novel techniques for processing large numbers of concurrent continuous queries with required Quality of Service (QoS). Because of the dynamic nature of query registration and stream behavior, we are designing heterogeneous-grained adaptivity for CAPE and exploits dynamic metadata at all levels in continuous query processing, including the query operator execution, memory allocation, operator scheduling, query plan structuring and query plan distribution among multiple machines. We will (1) design an extensible dynamic metadata model; (2) design adaptive algorithms for use in each layer of query processing to exploit available metadata; (3) develop QoS specification models for capturing resource usage; (4) incorporate a hierarchical interaction model for coordinating the adaptation at different levels within the CQ system; and (5) design a family of metadata-exploiting optimization techniques.