Overview of the Zeos Database Objects Architecture

To address multiple and sometimes inconsistent requirements the development team had to completely rethink the new architecture. It is complex with respect to previous designs but not over-designed. Each module put into the new design was carefully considered and carries special unique features. In this article we'll try to present you the new architecture and explain its purpose.

From a top-down view the library is separated into three logical layers:

The Plain API implements low-level functions specific to each SQL server. The Component API encapsulates the main library interface that is utilized by application developers. The DBC API is the middleware that retrieves, stores and modifies data for the high level components.

Each layer has several horizontal and/or vertical sub-layers semi-independently from each other (see picture 1). Let's go through each layer and look how all that works.

ZeosDBO components do not communicate directly with SQL servers. Instead they use native client libraries provided with SQL databases. The Plain API layer provides an access to functions of native client libraries, constants and data structures (usually written in plain C) from the Object Pascal language or C++.

That functionality was the original basis for ZeosDBO. Support for multiple versions of client libraries and SQL servers was the main deficiency of earlier designs...

Native library calls (dynamic libraries in Windows and shared libraries in Unix) are represented in programming language as regular functions. For example:

ZPlainMySql323.pas:

function mysql_init(Handle: PMYSQL): PMYSQL; external 'libmysql.dll';

Usually database APIs do not significantly change between versions. But because function calls do not allow polymorphism adding support for new versions of the Plain API is coding intensive. Hard coded approaches are inflexible and error-prone, which limit long term feasibility.

if Version = 'mysql-3.23' then
  ZPlainMySql323.mysql_init(...)
else ZPlainMySql40.mysql_init(...);

To implement polymorphism, simplify source code, and provide insulation from changes in SQL server protocols a new extremely thin interface layer was added into ZeosDBO. That layer is called “Plain Drivers” and implemented as follows:

// Generic MySQL driver interface
IZMySQLDriver = interface ...
  function mysql_init(...)
end;

// MySQL driver for version 3.23
TZMySQL323Driver = class (TInterfacedObject, IZMySQLDriver)
  function mysql_init(...)
end;

// MySQL driver for version 4.0
TZMySQL40Driver = class (TInterfacedObject, IZMySQLDriver)
  function mysql_init(...)
end;

function TZMySQL323Driver.mysql_init(...)
begin
  Result := ZPlainMySql323.mysql_init(...);
end;

function TZMySQL40Driver.mysql_init(...)
begin
  Result := ZPlainMySql40.mysql_init(...);
end;

Using such thin class wrapper allows easy addition of new client interfaces. The specific functionality that requires overriding is encapsulated in the Plain driver. The rest of the code now has a uniform method of providing native database calls, without requiring specific knowledge about the database server.

// Initialize plain driver
PlainDriver: IZMySqlDriver;

if Version = 'mysql-3.23' then
  PlainDriver := TZMySQL323Driver.Create()
else PlainDriver := TZMySQL40Driver.Create();

// Use plain driver
PlainDriver.mysql_init(...)

With this approach it is possible to use the same API for different SQL servers, independently of version.

With native access provided to the database, Delphi database aware components can now expect database specific functionality in a uniform manner. However, each SQL server has different semantics that must be designed into the components to provide a universal generic interface. The main goal of the Zeos Database Objects is to provide this generic interface to the application developer.

In older versions of ZeosDBO, the intermediate interface was implemented as a class wrapper to the MySQL and PostgreSQL connection objects. These two databases have very similar capabilities so design of intermediate API was not complex. However, support for other SQL servers added new and very specific features. After several extensions to support these features the class architecture became was not clearly defined. Ultimately, the interface broke its encapsulation rules and the high level components were forced to make low level database calls.

To overcome this difficulty in the new version, the overall design was an abstract approach with an intermediate interface. Design of such an interface is not trivial. So to avoid new mistakes it was decided to avoid a proprietary design, and instead draw inspiration from a well-known database API.

As a prototype for the intermediate interface the development team chose JDBC 2.0. JDBC is one of the latest and more popular APIs in that database community. It covers various abstractions such as statements, result sets, stored procedures, blobs, and very rich metadata definitions.

JDBC API is implemented in Java through set of interfaces. Borland compilers support interfaces, so porting JDBC from Java to Object Pascal was straightforward. Data types and method names were generally preserved. Overloaded methods were avoided because of poor support in the C++ Builder family of compilers.

The main DBC interfaces are presented on picture 2.

Standard JDBC interfaces provide a uniform client API. To address functionality specific to different SQL servers, two general approaches were chosen.

1. Developers may initialize database connection object with list of string parameters. Each parameter switches server specific settings. For JDBC that method is not new.

Example: parameters can be defined in Connection URL -

zdbc:mysql://localhost/database?compress=true

Params.Values['compress'] := 'true';
Connection := DriverManager.CreateConnectionWithParams(Url, Params);

Additionally in ZeosDBO developers extended other DBC interfaces to initialize other object with specific parameters, particularly Statement object:

Params.Values['oidasblod'] := 'true';
Statement := Connection.CreateStatementWithParams(Params);

2. In Object Pascal each class is able to implement multiple interfaces simultaneously. We used that to extend standard JDBC interfaces with new methods specific for each particular SQL server. Now each class known not only a standard interface but a server specific interface as well:

IZMySQLConnection = interface (IZConnection)
  function Ping(...);
  function Kill(..);
end;

TZMySQLConnection = class (TInterfacedObject, IZConnection, IZMySQLConnection);
...
end;

The next step in porting JDBC to Object Pascal was an implementation of cached data access. Actually, many servers provide support for sequential data access only. Caching data on the client side is the important element to implement random data access for the retrieved result sets. On the other hand, many high-level database interfaces use even more sophisticated caching algorithms. So implementation of universal caching algorithms in one place can be a compact and efficient solution.

Cached DBC layer has only few classes:

The good thing about DBC interfaces that they are so generic, that additionally to regular SQL drivers it's easy to implement special adapters to other database interfaces such as Active Data Objects (ADO) for example.

DBC interfaces are generic, so providing support for additional databases or connectivity layers such as ADO is trivial.

The last top layer in ZeosDBO library implements dbware components. These components are used for development in Delphi, C++ Builder or Kylix. Currently Borland compilers support several standards for dbware components:

At the time of this writing this article ZeosDBO library only supported TDataset descendent components. Implementation of other component types is scheduled for future versions.

The component class diagram is presented on picture 3:

TDataset uses maximum the DBC layer functionality to read, modify and store data. Additionally it implements extra functions to filter, search and sort data, connect to visual components and many other things.

In this article we described the main ideas of the new architecture of Zeos Database Objects component library for native database access. This architecture was introduced in version 6.0 and demonstrated high flexibility and effectiveness. In version 6.1 the code was seriously revised and optimized, but the main principles of the architecture was not changed.

Functionality and flexibility incorporated in the architecture will meet adapt to new requirements in the future. Support for new SQL servers will be added as well as access to SQL specific functionality and support for other high level interfaces like dbExpress and Midas.

To familiarize with the library you may visit the project website at http://www.zeoslib.net or the development page on SourceForge athttp://www.sourceforge.net/projects/zeoslib