Distributed Transaction Processing using Oracle8^™

An Oracle Technical White Paper

June 1997

CURRENT APPLICATION ARCHITECTURES IN TRANSACTION PROCESSING(TP) SYSTEMS

Two styles of application architectures are predominantly found in current Transaction Processing (TP) systems. They are two-tier and three-tier client/server architectures. Both architectures have unique advantages and disadvantages with regards to development and deployment benefits. Therefore, both architectures have their place in TP systems in the enterprise, depending on benefits desired. A common trend in TP systems is the need to support transactional integrity across multiple servers in both architectures, i.e. Distributed Transaction Support. This trend has been fueled by an increasing adoption of the network computing paradigm.

The Oracle Server provides support for distributed transactions in either two-tier or three-tier TP systems. This paper first provides a description of both these architectures in an Oracle environment and how global transactional integrity is maintained across different servers. It then describes the X/Open DTP XA Interface, which is the industry standard for distributed transactions, and Oracle8’s support for the XA standard. The paper also presents a sample application using XA in an Oracle and TP Monitor environment.

TWO-TIER APPLICATION ARCHITECTURE FOR TP WITH ORACLE8

Introduction

The traditional two-tier TP architecture entails various clients communicating directly with one or more Oracle and other database servers. Here, the client is usually responsible for the presentation logic, i.e. the logic for screen handling. It usually handles some application and business logic. The server is responsible for the application and business logic, as well as data management.

In this traditional two-tier environment, each client is connected to an Oracle shadow process over a persistent connection. Consequently, the server has to host an equal number of processes as the clients, to respond to requests from each client. This requirement sometimes impacts the scalability of the overall system, as discussed in further sections.

Oracle Multi-Threaded Server

Oracle’s Multi-Threaded Server circumvents some of the scalability issues, especially the high process requirement on the server. In this environment, the database server starts up a configurable number of ‘shared server’ processes to handle client requests. The clients connect to a set of dispatcher processes via the Oracle Listener. The dispatcher process then posts the client requests in a queue and these requests are processed by any shared server that is currently free. Information is routed back to the client in the same manner. Thus, the server does not need to host per-client processes and connections. Connection management is handled by the dispatcher and the server can focus on data and business and application logic management. However, the connections are still persistent between clients and dispatchers.

Distributed Transactions Support in an Oracle Two-Tier Environment

Oracle-only environment

In a distributed environment with multiple Oracle servers, any Oracle server can access data and programs transparently from any other Oracle database using database links. Further, any Oracle Server can maintain transactional integrity across all these servers through Oracle’s optimized two phase commit mechanism. Such transactions are called "Server Managed Transactions."

Heterogeneous Environment

In a heterogeneous environment with a combination of Oracle and non-Oracle servers, the Oracle server can serve as the ‘integrating’ server using Oracle’s Heterogeneous Service and Oracle Open Gateways^™. The Oracle server accomplishes this by fetching data from other servers as appropriate and coordinating distributed transactions across these databases. Thus, an application can simply issue a COMMIT or a ROLLBACK at the end of the transaction, and the Oracle server along with the Heterogeneous Services and Gateways will coordinate the distributed transaction across various servers and ensure global integrity of the data.

The benefits and common problems with deploying client/server two tier architecture for transaction processing are described below.

Benefits from adopting Two-Tier TP Environments

• Flexibility of Development: A wider variety of choices is available to the application developer for developing and deploying client/server applications. The developer can deploy Oracle Applications^™, such as Financials or Manufacturing, or can use easy-to-use design and development 4 GL tools, such as Developer/2000^™ and Designer/2000^™. The developer can also use any GL database development language such as Pro*C, which is a part of Oracle Programmer^™.
• Easy to Understand: The TP marketplace understands client/server two-tier architectures very since it has been around for a long time. Hence, learning curves in designing and implementing this type of architecture are short.
• Low Cost of Development and Deployment: Because of the learning curve effects mentioned before, as well as high availability of development and management tools available, two-tier architectures can cost less to implement than their multi-tier counterparts.

Common Issues With Two-Tier TP Environments

• Inefficient Resource Utilization: An inherent problem with the traditional two-tier architectures is inefficient management of operating system resources. The resources in question are connections, processes and memory. The two-tier architecture imposes the need for a per-client process and connection and also leads to high memory utilization. For instance, 2,000 clients connected to a server may require 2,000 operating system server processes, 2000 persistent connections and a large number of file handles! In an online TP (OLTP) environment with a high think time, using persistent connections and processes per client can result in wastage of these expensive operating system resources.
• Multi-threaded Server reduces the overall number of processes in the environment, and provides more flexibility in configuring the environment according to the workload characteristics. It does not alleviate the persistent connection issue as mentioned before.
• Limited Scalability: Inefficient resource utilization can become a bottleneck in an environment supporting a large number of users. Thus, inefficient resource utilization does not allow users to exploit the full benefits of the hardware and operating system, and leads to limited scalability.

Oracle8 Enhancements for Scalability in the Two-Tier Architecture

Oracle8 has made significant enhancements in improving scalability for two-tier environments, such as, enhanced Multi-Threaded Server and improved memory management with Serial Reusability of PL/SQL^™ packages. Oracle’s Connection Pooling in Net8 allows idle operating system connections to be timed out and freed to be used by other clients. Thus, it allows an increased number of client/server connections across a fixed and limited number of physical server ports. Because of increased efficiency in resource utilization, Oracle8 can provide support for tens of thousands of users in the two-tier environment. Hence, in environments where the scalability offered by Oracle8 two-tier deployment is adequate, and where customers have made significant investments in developing applications using Oracle client-side tools, the two-tier environment may be more beneficial than a multi-tier deployment.

THREE-TIER APPLICATION ARCHITECTURES FOR TP WITH ORACLE8

Introduction

In a three-tier environment, the middle tier serves as a ‘concentrator’ between various clients and servers. It manages the number of persistent connections between the clients and servers, and may be extended to manage transactional integrity across various servers. The front-end and back-end processes have no knowledge of each other, and can be optimized for their particular task. This makes it easy to distribute both the front-end and back-end processes throughout the network on the most appropriate hardware for the task. Moreover, multiple server processes of the same type might be activated to handle an increasing number of users.

Oracle8 can be deployed in the three-tier environment, using either Oracle Connection Manager, or a TP Monitor at the middle tier, as described below.

Three-Tier TP Environment With Oracle8 and Connection Manager

Oracle8 and Net8 provide a three-tier solution to more efficiently utilize operating system resource than the traditional two-tier environments. Oracle’s Connection Manager provides higher scalability through increased efficiency in connection usage, and at the same time, enables protection of investment made by enterprises in their applications. It does this by multiplexing a large number of logical connections across limited physical connections, and thus enables a large client population to connect to a common server. This solution allows a high user population to access the Oracle server simultaneously over a limited number of operating system connections, and no rewriting of application code is necessary.

Oracle’s Connection Manager resolves the connection usage problems seen with the two-tier architecture described before. From an application programmer’s viewpoint, these solutions are similar to the two-tier model. Moreover, distributed transaction support is provided by the Oracle Server and potentially Heterogeneous Services in a heterogeneous environment as with the two-tier environment.

Three-tier TP Environments with Oracle and Transaction Processing(TP) Monitors

TP Monitors were designed to address the scalability issues with the two-tier architecture model described before. Further, they extend the ‘concentration’ function at the middle tier to provide support for managing transactional integrity across potentially heterogeneous servers. TP Monitors also provide added benefits by monitoring transactions and controlling workload control across various servers for better throughput and performance in the three-tier environment.

In the three-tier TP Monitor environment, the client usually handles the screen handling functions and requests for services by making calls to the TP Monitor. The TP Monitor controls the traffic between the clients and the Application Servers. The Application Servers are pools of prestarted application processes or threads, waiting for work. Each of these servers hosts one or more related services. They may have one or more connections to databases such as Oracle8.

When a client sends a request, the TP Monitor routes the request to the appropriate Application Server or invokes it, if needed. The Application Server processes the request and routes the reply back to the requesting client through the TP Monitor. Thus, TP Monitors remove the need for per-process persistent connections.

TP Monitors can also coordinate transactions across multiple and potentially heterogeneous databases. To ensure plug and play compatibility between different TP Monitors and database servers, most TP Monitors and database servers adhere to the XA interface standard.

In addition to the benefits listed above, TP Monitors offer many operational enhancements, such as, load-balancing across Application Servers, data dependent routing and managing failover of Application Servers.

Benefits from Three-Tier TP Monitor Environments

• Distributed Transactions Across Heterogeneous Databases
  A TP Monitor can coordinate transactions across different databases with the XA interface. Since XA is an open standard that has been adopted by most databases and TP Monitors, it is easy to mix and match between servers and TP Monitors. The next section describes the XA standard.
• Efficient Resource Utilization
  As with Oracle’s Connection Manager, TP Monitors remove the per-client connection requirement to the server, and thus, result in significant savings in operating system resource usage. Moreover, it facilitates separation of connection handling from data manipulation, specializing each component for its particular task.
• Flexibility in Development
  TP Monitors enforce service-oriented development, making it easy to reuse application code. In a TP Monitor environment, clients and servers do not need to be aware of each other, imparting flexibility in development applications. Further, each entity can be optimized for its particular functionality, as well as located on the appropriate hardware.
• Operational Benefits
  Operational benefits such as load balancing, data dependent routing and fail-over enhance performance and scalability of the overall system.

Note that some Oracle customers use a TP Monitor for the scalability and operational benefits, rather than to provide distributed transactions. In this case, the Oracle database server solely handles transaction management. If the transaction involves other Oracle databases via database links, then the Oracle Server will use its native two-phase commit to ensure transactional integrity.

Common issues with TP Monitor Environments

• Complexity in Development
  TP Monitor technology is yet to gain mass acceptance, hence information on techniques for developing applications is not widely available. This significant learning curve effect associated with the current TP Monitor development environments has been a factor in slower acceptance of TP Monitors in the marketplace.
• Protection of Investment
  It is difficult to merge a TP Monitor into an existing application environment. Taking advantage of TP Monitor entails a complete redesign and reimplementation of existing applications. Oracle’s connection manager and multi-threaded server offer improved scalability over client/server, while allowing use of existing applications.
• Performance Benefits
  Performance gains from using TP Monitors have only been documented for a very large user population, in the range of hundreds of thousands of users. Oracle8 can handle tens of thousands of users with client/server and optionally Connection Manager and Multi-Threaded Server.

STANDARDS FOR DISTRIBUTED TRANSACTION PROCESSING WITH TP MONITORS

Since TP Monitors coordinate the flow of transactional information between various entities, such as, applications and databases, the X/Open DTP standards body has defined a standard to facilitate distributed transaction processing among these entities. This standard, called the Distributed Transaction Processing (DTP) Model Specification, was published in late 1991. Subsequently, other publications and a revised model specification have been published.

The X/Open DTP model defines 3 components:

Resource Manager

The Resource Manager provides consistent access to shared resources. Examples of RMs include databases, transactional file systems and persistent queues. The Oracle database is an RM and provides consistency via its redo logs and undo segments.

Application Program

the Application Program defines transaction boundaries, i.e., start, end, commit and abort transaction. It also uses the resource manager APIs (i.e. Oracle OCI or Pro*C) to perform actions that constitute the transaction.

Transaction Manager

The Transaction Manager coordinates transaction across resource managers. It provides an API to specify the transaction boundaries ("Transaction Demarcation") and manages the commit and recovery procedures. It assigns identifiers to transactions, and monitors and coordinates their progress. Examples of TMs include distributed TP Monitors such as Tuxedo, Top-End, Encina and CICS.

Introduction to XA interface

The ‘XA’ interface is the industry standard interface defined by the X/Open DTP group, as part of their DTP Model Specification, for communications between a TM and an RM. This interface includes calls for opening/closing connections and two phase commit coordination. The Application Program is not involved, and the application developer does not need to know the details of this interface. A list of XA library routines is provided at the end of the paper.

To use the transaction management capability of the TP Monitor, an application would use a transaction demarcation API (called TX) provided by the TP Monitor, rather than the SQL transactional control statements like "commit work." For each TX call, the TM instructs all RMs using the XA commands to follow the two phase commit protocol. Figure 4 illustrates the interface within a monolithic application program model.

ORACLE AND XA

The Oracle Server, starting from Oracle7^™, can coordinate transactions across multiple, heterogeneous databases, using an optimized two phase-commit protocol, and thus ensure integrity of data globally. In most enterprise environments, the Oracle Server acts as a transaction coordinator and coordinates potentially distributed transactions. However, XA enables it to participate in a TP Monitor managed environment.

To support XA, Oracle, starting with Oracle7, has provided an XA library with the database server on all platforms that support an XA compliant TP Monitor. Further, Oracle’s XA implementation is fully compliant with the X/Open specifications. Since Oracle 7.3, Oracle’s XA implementation is also thread-safe. Oracle’s XA implementation, not only adheres fully to the requirements specified in X/Open’s XA guide, it also provides some of the optional features in the specification, as well as some optimizations, leading to significantly higher performance.

Features of Oracle’s XA Implementation

The requirements for an XA compliant resource manager include support for all specified xa routines and commitment protocol. Further, an XA compliant RM must be able to accept transaction identification from the TM and support TM driven recovery from failure. Lastly, the RM must publish specific information, such as the entry points and other information for the RM ‘xaosw’, the open string used with the xa_open() call, close string, libraries needed to link applications and any RM specific requirements in a well-published manner. Oracle’s XA implementation adheres to all these requirements completely.

Oracle’s XA implementation also supports the following optional features and enhancements:

Read-Only Optimization

This optimization alleviates the need for the Oracle server to participate in a two phase commit protocol, when no updates are involved, i.e., a read-only transaction. This optimization results in avoiding unnecessary overhead of roundtrips and I/O’s.

Dynamic Registration

Dynamic Registration allows the Oracle Server for just-in-time participation in a global or distributed transaction. Without dynamic registration, the TM initializes all RMs before any transaction. This results in needless overhead for RMs who do not play any role in a particular global transaction. To avoid this overhead, the Oracle server contacts the TM to check if it is part of a global transaction when it receives any work from an application. This just-in-time approach results in increased efficiency and improved performance for those servers which conditionally access Oracle in a transaction.

Higher Concurrency

Oracle8 supports loosely coupled transaction branches in the same instance. This allows for increased concurrency for transaction branches. Loosely couple transactions imply that transaction branches do not share resources, in this case, locks. Tightly coupled transactions branches, on the other hand, are designed to share locks, and the Oracle server guarantees that deadlocks do not occur within these branches. Tight coupling leads to serialization of work between branches, and thus reduce concurrency between them. In contrast, loose coupling offers more concurrency, but cannot provide any ‘no-deadlock’ guarantee. Thus, the application programmer must determine the degree of coupling depending on the application requirements.

Oracle7 supported only tight coupling in the same instance, and loose coupling among different instances. Oracle8 extends this support by providing the flexibility of loose or tight coupling within the same instance.

Strong Integration With Oracle8 Parallel Server

The XA specification requires session-based locking rather than the older process-based locking, so that transactional work can be moved across different Application Server Processes. This requirement meant that earlier, the Oracle XA library could only be used on platforms that provided the required session-based Distributed Lock Manager. However, Oracle8 Parallel Server’s integrated DLM enables support for Oracle’s XA library on all platforms. Further, strong integration with the Oracle Parallel Server allows for transaction recovery from any instance of the Oracle8 Parallel Server, resulting in improved recovery.

Putting All The Pieces Together: An Application Architecture Using Oracle and TP Monitors

A possible application architecture using TP Monitors and Oracle is illustrated below. This architecture comprises of different components, such as Application client program(s), TP Monitor, Application Server(s), and Database(s). The following paragraphs describe these components in more detail.

Application Client

The application client is responsible for presentation logic in this multi-tier architecture. It defines the boundaries of the transaction using the TX interface. It is implemented usually as a stand-alone process on a separate machine, and can be linked with the TP Monitor libraries. Most TP Monitors allow the client program to be coded in C, C++ or Cobol.

TP Monitor

The TP Monitor coordinates flow of transactional information between the application client and various services supported by the Application Server. It may be implemented as a set of libraries or as a set of one or more processes or threads. When implemented as a set of one or more processes, the TP monitor is usually installed on a highly available machine. For further information on TP Monitors, refer to your TP Monitor manual.

Application Server

The Application Server program implements one or more related services. For instance, a banking customer might have two categories of servers: one for the credit service, and the other supporting the debit service. The Application Server program is usually implemented with the Pro* languages such as, Pro*C or Pro*Cobol. It is linked with the TP Monitor libraries and Oracle’s client side library, libclient.a. It is implemented as a stand-alone process, and a client environment may have different Application Servers on different machines. This type of a service-based architecture allows for more flexibility and reuse in implementation.

The Application Server program may be connected to the shadow process of the Oracle database server directly using a dedicated Oracle shadow process or via Oracle’s Multi-threaded Server(MTS). Further, the Application Server may be connected to other Oracle databases using Net 8. This allows for the Application Server to be connected to a database that does not support a TP Monitor.

Existing Oracle applications connected to the Oracle server can coexist with the TP Monitor applications against the same server. The TP Monitor manages only the transactions that it has started. The Oracle server handles any concurrency issues between the transactions started by the server and those started by the TP Monitor.

WRITING AN ORACLE TP MONITOR APPLICATION

A key difference between writing an Oracle TP Monitor application with and without using XA, is the transaction semantics. This includes transaction demarcation in the Application Client program, as well as a possibly finer-grained approach to application implementation. Thus, the SQL clause "EXEC SQL COMMIT" would be replaced by a "txcommit()" call. Note that the actual API used for transaction semantics depends on the API’s provided by TP Monitor vendor. However, all TP Monitors furnish APIs to signal the beginning and end of a transaction, and for communication of requests and responses between the Application Client and Application Server.

The following table shows how some of the changes needed to enable a Pro*C program to use a TP Monitor's transaction capabilities. The key difference is that you use the TX API to logon, logoff and control transactions. Data manipulation operations remain in SQL.

Equivalent SQL and TX API Statements

Notes

1. txopen() is used once for all RMs accessed by the application program; the TP Monitor will call each RM in turn to establish the RM connection. It will do these connects based on information passed in an open string that is in a configuration file. Oracle8 allows the Pro* program to have more than one database open and to do updates to both in a single transaction. However, not all TP Monitors will allow this configuration.
2. Some TP Monitors may issue the txopen() and txclose() themselves so that the application program need not do so. The TP Monitor documentation will illustrate the correct method.
3. Since there is no implicit start of transaction in TX, the txcommit() or txrollback() are needed only if a txbegin() has been executed.
4. When running client/server, the txbegin() and txcommit() can be in the client even though all the SQL is in the server. This allows the client to talk to two or more servers in the same transaction and have their work committed atomically.
5. Use of ‘set transaction’ is not allowed in XA transactions.

Sample Program using TP Monitor and Oracle

The following sample program for a banking application involving transfer of funds, highlights some of the key application coding differences. Tuxedo’s/T verbs are used here for the sake of illustration, but most TP Monitor vendors provide similar constructs to construct application programs. The first example illustrates the use of a TP Monitor in conjunction with Oracle but without using XA. In this case, the Oracle server manages the transaction coordination. The second example depicts the same banking application, with the TP Monitor managing the transactions, using the XA protocol beneath the covers.

Sample Program: Oracle Managed Transaction

Application Client

tpcall("transfer_chk_to_sav …");

Application Server

transfer_chk_to_sav_service(TPSCVCINFO *input)

{

…

EXEC SQL UPDATE bal SET savings_bal = savings_bal + amount

WHERE account = acctno;

EXEC SQL UPDATE bal SET checking_bal = checking_bal - amount

WHERE account = acctno;

EXEC SQL COMMIT WORK;

tpreturn();

}

Sample Program: TP Monitor Managed Transaction

Application Client

tpbegin();

tpcall("debit_chk");

tpcall("credit_sav");

tpcommit();

Application Server 1

debit_chk _service(TPSCVCINFO *input)

{

…

EXEC SQL UPDATE bal SET checking_bal = checking_bal - amount

WHERE account = acctno;

tpreturn();

}

Application Server 2

credit_sav _service(TPSCVCINFO *input)

{

…

EXEC SQL UPDATE bal SET savings_bal = savings_bal + amount

WHERE account = acctno;

tpreturn();

}

TP MONITORS, XA AND PERFORMANCE ISSUES

TP Monitors may provide better performance in handling very large user populations in certain cases. However, with Oracle8 pushing the envelope on scalability for two-tier and multi-tier architectures, the load point at which TP Monitor solutions become a more effective solution has been pushed out and should be evaluated for each application.

The XA library imposes some performance penalty versus server managed transactions. This penalty stems from the extra bracketing calls that needed, such as xa_start() and xa_end() for each unit of work within each transaction. This is not required by native Oracle managed transactions. However, a benefit from the bracketing is that multiple processes can operate on the same transaction. Moreover, the code path using XA is longer because of the need to map XIDs to Oracle internal transaction identifiers and maintain a state machine to give XA protocol errors as appropriate. Thus, whether TP Monitor managed or server managed transactions provides the best performance will ultimately depend on the application characteristics.

Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.

Worldwide Inquiries:
+1.415.506.7000
Fax +1.415.506.7200
http://www.oracle.com/

Copyright © Oracle Corporation 1997
All Rights Reserved

This document is provided for informational purposes only, and the information herein is subject to change without notice. Please report any errors herein to Oracle Corporation. Oracle Corporation does not provide any warranties covering and specifically disclaims any liability in connection with this document.

Oracle is a registered trademark and Enabling the Information Age, Oracle 8, Oracle7, PL/SQL, Oracle Open Gateways, Oracle Applications, Developer/2000, Designer/2000, and Oracle Programmer are trademarks of Oracle Corporation.

All other company and product names mentioned are used for identification purposes only and may be trademarks of their respective owners.