CHAPTER ONE
1.0 INTRODUCTION
The amount of documents in an online information database system usually grows rapidly overtime. How to store, manage and search these documents within the online information database system is a challenging problem. Documents in an online information database system are stored as semi-structured data, while in the traditional relational database it is stored as structured data. Relational database management system cannot manage semi-structured data efficiently and cannot satisfy the requirement of content-based text retrieval.
A lot of research works have been done about semi-structured data, such as data modeling, query language for text retrieval, index methods and text retrieval algorithms and similarity search algorithms. These research results have been used a lot in an online information database system systems. SSREADER an online information database system, the national an online information database system and wanfang database are popular referencing system in china. All the referencing system classify the documents into several classes and support querying inside a given class. Metadata search and full-text search through a single keyword or expressions are both supported in these referencing systems. other examples of referencing system are greenstone an online information database system, uc berkeley an online information database system, tufts an online information database system, acm an online information database system, ncstrl etc. similar functions are supported in these referencing system, such as metadata searching, full-text searching, documents classification and browsing. Greenstone an online information database system has a suite of software that provides management toOIDS for creating and maintaining a an online information database system. tufts an online information database system is for the integration of collections that exist or may be developed in the future. there is a system named lore developed by Stanford. it is a database management system for managing semi-structured data. The ncstrl at cornell university is a distributed technical report referencing method developed by the arpa-sponsored computer science technical report project. The ncstrl collection is distributed among a set of interoperating servers operated by participating national archivess. all of the referencing system described above do not support the following functions: structure and content-based queries, automatic entries of external documents and parallel document processing.
The OIDS system described in this study has the following features. (1) Generalization: It is essentially a general document database management system. It can be used to build referencing system for user needs and provides a suite of toOIDS to maintain it. (2) Parallelism. OIDS uses a lot of processors to execute queries and manage documents, which improves both storage capacity and query efficiency. (3) Structure and content-based retrieval. Users can query inside a document for an element, e.g. a chapter of a book, which not only allows users to propose for a more accurate query, but also reduce the information transmission workload in networks. (4) Personalization. OIDS can query according to user’s interest and recommend documents relevant to user. (5) Automatic external data entering. OIDS can combine with other search engines in finding and adding references automatically. (6) Multi-format supporting. DL collects a lot of document resources including books, journal papers, proceedings etc. and supports document information retrieval for a lot of document formats. (7) DLSQL query. OIDS defines a query language like standard SQL, named DLSQL. By using DLSQL, users can program and do all the operations in OIDS. (8) Automatic document classification. It creates a classifier according to the sample documents loaded by the system manager and automatically classifies documents.