XCQ - A Framework for XML Compression and Querying

Wai-Yeung Lam

HKUST

XML has already become the defacto standard for specifying and exchanging data on the Web. However, XML is by nature verbose and thus XML documents are usually large in size. The size problem of XML documents usually hinders their practical usage, since it substantially increases the costs of storing, processing and exchanging data.

In this talk we discuss the XML Compression and Querying (XCQ) Framework, which is designed for XML compression and compressed XML querying. We show that by utilizing the structural information from the input XML document and its associated DTD during the compression process, XCQ can achieve better compression ratio than that of gzip as well as generate more usable compressed data that is able to be queried without running a full decompression. We Also discuss in detail the techniques developed for implementing the XCQ Framework. These techniques include: (1) the DTD Tree and SAX Event Stream Parsing (DSP), which is designed for efficient compression of valid XML documents without involving user expertise, (2) the Partitioned Path-Based Data Grouping (PPG) compressed document format, which supports querying over compressed XML data by performing partial decompression, (3) the Block Statistic Signature (BSS) indexing, which is designed for indexing block-oriented compressed data that aims at using minimal system resources, and (4) the access method that is used for realizing partial decompression when processing selection, structural, structure-based aggregation, and path-based aggregation queries over a compressed XML document.