Overview openPETRA architecture

(An overview of this was orignially written down after a presentation by Christian at the Petra Team Meeting on 25/01/2006 by Timotheus. Article updated and greatly extended in July 2006 by Christian. Reviewed/updated in May 2008 and November 2008 by Christian. Timotheus removed the Petra 2.x specific bits in September 2010. Following that, Christian reviewed/updated the article for openPETRA in September 2010.)

N-Tier Architecture Overview

PetraServer

openPETRA Server Architecture Diagram

• Platform
  • can run on Windows and Linux (fully managed code)
    • runs as a Linux Service (basically a background console application) on Linux for network installations
    • runs as a invisible console application on Windows and (potentially Linux) for standalone installations
      • we don't make it a Windows Service because of Linux compatibility!
    • we could make it a WinForms application for Standalones, if we want
      • won't have any WinForms, but a Tray Icon
• 1 to n Petra Clients connect to one PetraServer
• PetraServer accesses the database
  • opens a database connection for the PetraServer itself on PetraServer startup (used eg. for user authentication)
  • opens a database connection for each Client session
• creates an AppDomain for each Client session (see below: AppDomains)
  • reason for that: insulation!
    • program code executing separately for each Client session
    • prevent mixing/confusion/exchange/leaking of data between Client sessions
    • errors/exceptions in code only affect one AppDomain - namely the one of the Client session in which they occur!
      • it is therefore highly unlikely that an exception within a Client session can affect the stability of the PetraServer (to the best of our knowledge this has never happened in three years of production use of the PetraServer within OM)
• maintains a list of running Client sessions
• has got a Server Menu with several commands
  • listing details of connected/disconnected Client sessions
  • disconnecting a Client session
  • shutdown the PetraServer
  • plus other commands for debugging purposes
• provides two different .NET remoting connections
  • one for Petra Clients
  • one for the PetraServerAdmin application
• Logging (Screen, Logfile, ...)
  • setting in .NET Configuration file (XML) allows to set the level of logging/debugging
  • code that performs logging for debugging purposes needs to be enclosed with the conditional define DEBUGMODE (we don't want to ship that code in a production Petra installation)

AppDomains

• A Client connects once to the PetraServer, from then on it gets his own AppDomain and all future communication happens between the Client and its allocated AppDomain
  • Obtaining Security-related information for that client's user: Working_with_OpenPetra_Security#Programmatically_Accessing_an_OpenPetra_Users.27_Security_Details
• Intruders should not have a chance to run code unauthorised
  • only after a successful login of a Client...
    • ... code that can be called into from the Client side gets loaded (dynamic DLL loading)
    • ... remoted objects with unpredictable remoting URLs (cryto-lib generated) become available to that Client
      • this prevents the use of code that was loaded within an AppDomain by somebody that doesn't know the remoting URLs for that specific AppDomain (i.e. another Client could not access that code)

• Benefits
  • AppDomains still run in the same Process, less cost involved than starting a separate Process (.EXE) for each Client connection
  • makes sure no memory is used outside of the AppDomain (leaking is prevented by the .NET Runtime)
  • Memory that was used by the Client is completely deallocated by the .NET Runtime when it's AppDomain is unloaded (no server-side memory leaks!)
  • Threads can pass the AppDomain barrier, if we allow it

• Costs
  • not 'too fast' to start up an AppDomain and load DLLs into it (when a Client connects)
  • clearing up takes some time (when a Client disconnects)
  • 6 to 8 MB RAM required for each Client connection (on Linux with mono)

• Important:
  • Threads: we need to make sure that all running Threads are aborted if an AppDomain is to be unloaded sucessfully - otherwise the AppDomain cannot be unloaded!

Petra Client

• Platform
  • runs best on Windows
  • runs on other platforms as well. Layout of Forms isn't satisfactory yet, though.
    • Linux: with graphical desktop manager using mono's implementation of WinForms (therefore we need to stick to 'fully managed' code...)
    • MacOS X: using mono's implementation of WinForms (therefore we need to stick to 'fully managed' code...)
    • Browser-based (Operating System independent!): The PetraClient (or a part of it) could be implemented as a (rich) web application running in a browser
      • a basic reference implementation for a rich web application exists and is being improved on
• a single Petra Client installation can connect to different PetraServers
  • command line switch allows using several .NET configuration files (XML), which contain different parameter settings that specify a connection to a specific PetraServer

Database System

openPETRA Database Abstraction Layer Diagram

• Relational Database Management Systems (RDBMS)
  • With OpenPetra we have the choice of several OpenSource RDBMS's
    • Currently we support PostgreSQL, MySQL and SQLite.
• Platforms
  • The RDBMS server needs to be able to run on Windows and Linux, any future RDBMS server needs to be able to do this as well (for network/standalone installations of Petra)
    • Exception: if an individual or an organisation wants to extend the RDBMS support of OpenPetra to a RDBMS that runs only on a certain OS (e.g. MS Windows) then this is fine as well.
      • The OpenPetra Project will want to maintain a multi-platform approach, but might make such implementations available to the OpenPetra community to give the OpenPetra community a wider choice of RDBMS. For that to happen, the individual or organisation would need to make its RDBMS implementation in OpenPetra available to us (which we suggest).
    • Exception: SQLite doesn't need an RDMBS server - it is a file-based DB that is accessed directly through a native .NET driver which works on Windows and Linux, therefore this restriction does not apply.
• DB access driver model
  • we support several RDBMS's through native .NET drivers
    • native .NET drivers are ideal because they have less overhead and provide more functionality than ODBC
      • PostgreSQL, MySQL, SQLite are all accessed through native .NET drivers
      • the native .NET drivers are required to run on Windows and Linux (the ones for PostgreSQL, MySQL, SQLite do that)
    • ODBC DB access implementation works on Windows and Linux
  • Support for other RDBMS's can be added using either native .NET drivers or through ODBC
• DB Access Layer
  • PetraServer can deal with different ways of accessing data in DB's because all DB access goes through a DB Access Layer for RDBMS-agnostic DB access
  • only this DB Access Layer needs to be changed when changing RDBMS and DB access driver model or adding support for a new RDBMS!
• Constraints
  • Referential Integrity
    • PostgreSQL: RDBMS automatically enforces referential integrity
  • Other constraints
    • PostgreSQL supports field-level constraints (eg. NOT NULL) which are enforced by the RDBMS
• Transactions
  • can be used for Rollback of changes and for pessimistic locking
  • ADO.NET
    • nested transactions are not possible, since only one transaction can be running at any given time on one DB connection
    • this is a limitation of ADO.NET and not a limitation of the native .NET drivers or ODBC
• Pessimistic / optimistic locking
  • .NET
    • optimistic locking is the standard. This is in effect no real locking: everybody can read any record; when writing, data could have changed in meantime
      • to prevent two people writing shortly after each other
        • a special field is added to every DB table ('s_modification_id_c'). Whenever a record gets updated through the DataStore, this field gets assigned a unique value (unique in the whole DB!). Whenever a record is read through our DataStore, the contents of this field are automatically read, when writing back to the DB the content of this field is automatically checked to be identical before saving the record. If the content of this field has changed, we know that someone else has modified the record between reading and writing. As result we prevent the saving of the record and inform the user about that by throwing a specific Exception that is processed by the GUI.
    • pessimistic locking is possible using a transaction: a transaction is opened with the desired isolation level and must be kept open as long as the data must be protected from overwriting
• Triggers and Stored Procedures
  • Triggers and Stored Procedures are not used since different RDBMS have differing implementations and different built-in languages for defining those.
    • We would not be able to offer support for a choice of several OpenSource RDBMS, and the choice could not be extended as easily, would we want to support Triggers and Stored Procedures.
  • In the absence of these, program logic in Business Objects needs to do what could be done in Triggers or Stored Procedures.
    • The execution of program logic vs. Triggers and Stored Procedures is potentially slower since Triggers and Stored Procedures are run directly within a RDBMS while program logic is run externally. That is the cost we pay for the ability to support various RDBMS.
• DB Query Language
  • There are several things a software developer needs to keep in mind regarding DB Queries to keep OpenPetra working across various RDBMS's.
    • The 'least common denominator' we stick to is the core of the SQL-92 command set.
    • Also see SQL compatibility rules.

Server Management Application

• Platform
  • can run on Windows and Linux (fully managed code)
• provides a 'remote console' for the PetraServer
  • allows execution of all PetraServer menu commands on a remote console (e.g. list client sessions, disconnect client, shutdown server)
  • can be supplied with Command Line Arguments to run certain menu commands - this is used from within OM's SLS sysadm program (sysadm petra22 / sysadm petra23 command or menu).
• can be run on a Windows machine while PetraServer is running on a Linux server (and vice versa!)
• currently only available as a console application. A WinForms implementation could be made as well without too much effort since all functionality (except the text menu) is in DLL's, not in the Console EXE.

Business Object instantiation and data flow

openPETRA Business Object instantiation and data flow diagram

Client Tier

• a Petra screen does not contain logic to write or read the data from/to the DB, in fact it would not even know how to do that!

• instead, a Petra screen makes a call to the PetraServer to retrieve data when its loaded. Although this only one call, it may well load data from several tables in the Petra DB through this call.
  • slow connections (ie. via analogue modem, ISDN, slow ADSL) require that the least possible number of calls to the PetraServer are made because of restricted network bandwidth and network latency. For this reason we will make only one 'all-including' PetraServer call when loading the screen wherever possible. Exception: see bullet Optional Delayed Data Loading below.
  • if we know (or expect) that such a call could take some time to come back to the Client, either
    • use multithreading in the Petra Client in such situations to not 'freeze' the UI. A call to the PetraServer will then be made in a new thread and this thread will either update the UI when the call is finished or give the main UI thread a signal that the call is finished.
    • use multithreading in the PetraServer and Petra Client to be able to return the request quickly to the Client and then make the Client poll the PetraServer for a result in a separate Thread to not 'freeze' the UI. This gives also the option of showing progress/progress bar and that the user can click a 'Cancel' or 'Stop' button to stop a long running operation. However, this generates quite some network traffic and a slighlty higher PetraServer utilisation because of the (probably) frequent polling. Examples screens that use that technique: Partner Find screen, Add Subscriptions screen.
• such a Client call to the PetraServer is possible because the Client knows through Interfaces which Business Objects can be instantiated in which Namespace and what methods can be executed on them in the PetraServer. (In fact, the call goes first to an Instantiator Object [see below], but this is transparent to the Client.)
• a Petra screen makes also only one call to the PetraServer when it saves changed (or new) data (using a Typed Dataset in most cases), although data that stems from (or goes into) several tables in the Petra DB might have been changed (added) in the screen.
• Optional Delayed Data Loading
  • The PetraClient's 'Delayed Data Loading' functionality can be switched on for a certain user (either in the PetraClient.exe.config file or as a Command Line parameter) to reduce amount of data that is sent (and the time on slow connections) when opening a screen. The functionality is switched on automatically for 'Remote Client' installations (that aren't connected using a LAN, but a slower connection).
  • an openPETRA screen can choose to implement a functionality to load only limited information in the first call if 'Delayed Data Loading' is switched on
    • Example: Partner Edit screen
      • loads initially only the number of Addresses, Partner Subscriptions, ... to be displayed in the header of the Tab Page, but not the actual data)
      • only when the User changes to a Tab Page the Partner Edit screen loads the detail data that is to be displayed on the Tab page
      • Lists with details (Example: Partner Edit screen - Subscriptions): loads details for each list entry already when loading the list (when the user changes to the Tab page)

Shared between Server and Client: Interfaces

openPETRA Screen, Interfaces, Instantiator, UIConnector and Business Logic Objects Detail Diagram

• Interfaces represent a 'contract' between Server Objects and the Client side
• Interfaces are not Objects themselves
  • they get implemented by Server Objects
  • they are used by Client Objects to call methods on the server-side Objects (who implement the Interface). This is transparent for the PetraClient, ie. it uses the Interfaces like actual client-side Objects.
• Interfaces tell the Client (and also the Compiler) which public Methods are callable on Server Objects - without the need to have a reference to the actual Server Object.
• A reference to an Interface that is implemented by the specific Server Object is enough for the Client to execute the Server-side method that is specified in the Interface! The Client-to-Server call is transparently proxied by .NET remoting (the Client gets a 'Remoting Proxy' instead of a real server-side Object, and that 'Remoting Proxy' appears and works like a local Object to the PetraClient).
• By using Interfaces, the executables (DLLs) that contain the server-side Objects don't need to be installed on Client PC's for the Client to work properly. Only the executables (DLLs) that contain the Interfaces are needed by the Client at runtime.
• The Interfaces enable Code Completion and Code ToolTips in the IDE.

Two ways how Interfaces are used in Petra Client-Server communication

Interfaces implemented by Instantiators

These Interfaces create a virtual Petra Object Hierarchy. Through this they tell the PetraClient which (Business) Objects can be instantiated in which Namespace on the PetraServer.

• A call from the PetraClient on one of the public methods of an Instantiator (that are included in those Interfaces) causes the server-side Instantiator Object to instantiate (create) a server-side Business Object for the calling PetraClient and return a 'Remoting Proxy' to the PetraClient that represents that exact server-side instance of the specific server-side Business Object to the PetraClient.

Interfaces implemented by (Business) Objects

These Interfaces tell the Client which Methods can be executed - once a (Business) Object is instantiated (=exists) on the Server side.

• A call from the PetraClient on one of the public methods of an (Business) Object (that are included in those Interfaces) causes that method to be executed remotely (ie. that method executes within the PetraServer).

Server Tier

Instantiators

openPETRA Screen, Interfaces, Instantiator, UIConnector and Business Logic Objects Detail Diagram

• Instantiators instantiate (create) UIConnector objects that are tailored for a specific calling Petra Screen on behalf of the Client
• in many cases they return data to the Client (with a Typed DataSet in most cases) immediately after the UIConnector object is created.
  • this is done so that the Client doesn't need to make one call to create a UIConnector object and immediately after that another call to retrieve data, but only one call (less data transfer = faster on slow connections - also because of network latency!).
• Instantiators are 'Factory' objects in object-oriented Client/Server programming terms.

UIConnectors (User Interface Connectors) = Client-facing Business Objects of openPETRA

openPETRA Screen, Interfaces, Instantiator, UIConnector and Business Logic Objects Detail Diagram

• UIConnectors are Business Objects of openPETRA that form the client-facing 'Business Objects Layer' of the PetraServer. They are part of the 'Business Objects Layer' of the PetraServer.
• Characteristics
  • One UIConnector Object exists for one openPETRA Client screen (but see 'Shared UIConnectors' below).
  • UIConnector Objects are only instantiated on the Server side, never on the Client side!
  • UIConnector objects are created by the PetraServer only when a request from one of the openPETRA Client screens reaches the PetraServer. This request comes in via .NET Remoting and goes to the Instantiator of that specific UIConnector, and not directly to the UIConnector (the UIConnector would not exist at that time).
    • openPETRA screens don't directly create other Objects in the PetraServer other than UIConnectors (that is, other Objects in the PetraServer are just used or created by the UIConnectors, but those other Objects don't get accessed directly from the client-side via .NET Remoting).
      • Those 'other Objects' will usually be Business Logic Objects (see below)
    • Each screen of the openPETRA Client deals only with one Server-side Object (its UIConnector) on which it performs calls for reading and saving of data. The screen itself holds all data in a multi-table Typed DataSet whose changes can be easily sent to the Server.
• Functionality
  • UIConnectors know how to read data
    • UIConnectors only retrieve needed tables (eg. either Person Details [p_person DB table] or Family Details [p_family DB table] in the Partner Edit screen) using the using 'Load...' methods of the DataAccess Objects, put all data into one (potentially big) object (a Typed DataSet in most cases) and return it through the Instantiator to the openPETRA screen - this is the most efficient way to transfer data to the Client. They can also call functions (e.g. in Business Logic Objects [see below] that work on the data (if needed) before returning data to the openPETRA screen.
  • UIConnectors know how to write data
    • an openPETRA screen sends only changed (or new) data to the UIConnector (using a Typed Dataset in most cases). The UIConnector in turn coordinates the saving of the data (eg. when a new Partner is created: first save data to PPartner table, and after that to PPerson [or PFamily, PChurch, ...] and PLocation and PPartnerLocation tables). Saving is done by calling SubmitChanges functions of each of the DataAccess objects in the correct order (deletion would be handled in reverse order). The SubmitChanges methods in the DataAccess layer update/add/delete the data for each DB table that is involved. Finally the UIConnector returns a result value to the openPETRA screen.
  • UIConnectors know what Aggregator Objects should be used (eg. for the PartnerEdit screen: loading from/saving to PLocation and PPartnerLocation tables is done using a 'PartnerAddresses' Aggregate Data Access Object).
  • UIConnectors can optionally create custom DataRelations between Typed DataTables or remove DataRelations, or set up custom Constraints or remove Constraints in the Typed DataSet that is sent to the openPETRA Client.
  • UIConnectors enforce business rules. They will usually do this by calling Methods outside the UIConnector (mostly Methods of Business Logic Objects) to avoid duplication of business rules code, or by calling an internal Method if the business rule is specific to the screen and can't be re-used.
  • UIConnectors may perform special validation of data (eg. across several DB tables) before saving data. They will usually do this by calling functions outside the UIConnector (mostly Methods of Business Logic Objects) to avoid duplication of of validation code, or by calling an internal Method if the validation is specific to the screen and can't be re-used.
• UIConnectors are tailored for 'Rich UI experience' (.NET WinForms) Clients, other openPETRA Clients that might be developed (eg. a Web Client, [non-existing] external application using COM to access PetraServer) may have other Connectors (eg. WebConnectors, COMConnectors) to read/write data
  • this concept allows different data reading/writing strategies for differenct clients (eg. a Web Client will retrieve/save data for separate ASP.NET pages separately, whereas all this data might be presented in one WinForm and retrieved/saved in one go).
• Shared UIConnectors for distinct parts of screens (Tabs, UserControls, etc.) can be created if they are re-used in several openPETRA screens (Example: the Office Specific Data (aka. 'Local Data') UIConnector [TOfficeSpecificDataLabelsUIConnector] is used in the Partner Edit screen (Partner Module) as well as in several other screens in the Personnel Module).
• Examples of UIConnectors
  • Ict.Petra.Server.MPartner.Partner.UIConnectors.TPartnerEditUIConnector (for the Partner Edit screen [massive!])
  • Ict.Petra.Server.MPartner.Partner.UIConnectors.TPartnerFindUIConnector (for the Partner Find screen [special case as this is a Find screen])

Business Logic Objects = Server-internal Business Objects of openPETRA

• Business Logic Objects are Business Objects of openPETRA that can't be used directly from the Client side. Instead, they are used by UIConnectors to perform certain business logic on behalf of the Client. They are part of the 'Business Objects Layer' of the PetraServer.
• Characteristics
  • All Methods of a certain Business Logic Object are related to one specific business purpose or process.
  • Business Logic Objects are either static, or are created on request of one of the UIConnectors. Static Business Logic Objects are more common.
  • Business Logic Objects can be used from many Business Logic Objects or UIConnectors - even from Business Logic Objects or UIConnectors of different openPETRA Modules than the Module in which a particular Business Logic Object resides.
• Functionality
  • Business Logic Objects contain business logic for the processing, loading, or saving of data, or any combination of those.
  • Business Logic Objects enforce business rules. They will do this usually inside the Method that is called or by calling another internal Method. Alternatively, they could call a Method of another Business Logic Object, though this will be the case less often.
  • Business Logic Objects will often perform special validation of data to enforce business rules. They will do this usually inside the Method that is called or by calling another internal Method. Alternatively, they could call a Method of another Business Logic Object, though this will be the case less often.
  • Business Logic Objects know what Aggregator Objects should be used (see example given at UIConnectors).
  • Business Logic Objects can optionally create custom DataRelations between Typed DataTables or remove DataRelations, or set up custom Constraints or remove Constraints in the (Typed) DataSet that is returned to the caller of a certain Method (obviously only in case a [Typed] DataSet is returned).
• Examples of Business Logic Objects
  • Ict.Petra.Server.MPartner.Extracts.TExtractsHandling (Extracts handling in the Extract Sub-Module of the Partner Module)
  • Ict.Petra.Server.MPartner.TMailing (Mailing-related functionality in Partner Module)

Data Access Layer

openPETRA Data Access Layer Diagram

Typed DataTables

• Typed DataTables are automatically generated ...
  • for each DB table in the \db\petra.xml file (this file holds the complete definition of the OpenPetra DB);
    • These auto-generated Typed DataTables are all found in the DLL 'Ict.Petra.Shared.lib.data.dll'.
    • They are organised into Namespaces according to OpenPetra Modules/Submodules. These associations come from the \db\petra.xml file. Example Namespace: Ict.Petra.Shared.MFinance.Account.Data (found in file Account.Tables.cs)
  • for custom DataTables that are specified in Typed DataSets.
    • These auto-generated Typed DataTables are also found in the DLL 'Ict.Petra.Shared.lib.data.dll'.
    • They are organised into the same Namepsaces in which the Typed DataSet in whose context they were specified are found. Example Namespace: Ict.Petra.Shared.MPartner.Data (found in file Partner.DataSets.cs)
• They provide many advantages; see this page for details.
• Because Typed DataTables provide such vast advantages over normal, untyped DataTables we use the Typed DataTables wherever we can in OpenPetra!
  • Exception to that rule: temporary storage of data in tabular form, e.g. for sorting or for providing the underlying data for a DataGrid - when that data cannot be mapped to an OpenPetra DB Table (but consider next bullet point for such situations).
  • In case Typed DataTables are needed that aren't specified in the \db\petra.xml file, specify custom Typed DataTables in Typed DataSets.

Typed DataSets

• Typed DataSets are automatically generated from Typed DataSet XML files (not the \db\petra.xml file).
  • We maintain these XML files on our own and need to re-generate the Typed DataSets after each change (this is done by invoking 'nant generateORM').
    • Details about the format of the Typed DataSet XML files can be found here.
  • These auto-generated Typed DataSets are found in the DLL 'Ict.Petra.Shared.lib.data.dll'. They are organised into Namespaces according to the directories in which the Typed DataSet XML files are found. Example Namespace: Ict.Petra.Shared.MPartner.Data (found in file Partner.DataSets.cs). The Typed DataSet XML Files for that Namespace are found in directory \csharp\ICT\Petra\Shared\lib\MPartner\data.
• Typed DataSets are the main means by which we transfer data between PetraServer and PetraClient (and vice versa)
• Typed DataSets form a named set of 1..n Typed DataTables (either ones that already exist for OpenPetra DB tables, or self-specified DataTables). A Typed DataSet is only one Class which contains many Typed DataTables.
• We use our custom-developed 'ICT Typed DataSets' that extend the functionality of the .NET DataSets and .NET Typed DataSets
  • most exensions deal with things that Microsoft just didn't take care of
  • some extensions are shortcuts for powerful functionality which we would otherwise need to program over and over again
• An arbitrary number of Typed DataSets can exist for each OpenPetra module (and indeed for the whole OpenPetra application)
• Typed DataSets provide many advantages; see this page for details.
• Because Typed DataSets provide such vast advantages over normal, untyped DataSets we use the Typed DataSets wherever we can in OpenPetra!

Data Stores

'Data Store' is a high-level, conceptual term we use. Data Store objects as such don't exist.

A DataStore is made up of

• DataAccess Objects
• DataCascading Objects
• Aggregate Data Access Objects
• Dynamic Data Access Objects

A DataStore (or rather its Objects) exists for each of the OpenPetra Modules/Submodules.

DataAccess Objects

OpenPetra's DataAccess Objects form an Object-Relational Mapper (ORM [1]).

• One DataAccess Object exists for each database table.
  • DataAccess Objects are automatically generated for each database table in the \db\petra.xml file (this file holds the complete definition of the OpenPetra database).
  • DataAccess Objects use the RDMBS-agnostic Database Access Object for the execution of SQL commands (DataAccess Objects are therefore not tied to a specific RDBMS!)
  • The methods of DataAccess Objects are all static (for easy use and speed reasons) and return and accept Typed DataTables and, in certain circumstances, Typed DataSets as well.
• A detailed description of all the advantages and the functionality of DataAccess Objects, and how to use them, can be found here.
• Because OpenPetra's DataAccess Objects provide a whole host of advantages over Dynamic Queries (in which query arguments are simply turned into strings and are part of the SQL command string), we use the the DataAccess Objects wherever we can in OpenPetra!
  • Exceptions to that rule: see this section.

DataCascading Objects

• automatically generated for DB tables in the Petra XML file (this file holds the complete definition of the Petra DB). They are organised in Petra Modules/Submodules according to associations in the Petra XML file.
• Functionality
  • these Objects perform 'cascading' operations on a number of DataTables that are referencing each other.
  • cascading is top-down, ie. Start DB Table -> all referenced DB Tables.
    • Example: deleting a p_location record with its DataCascading objects...
      • 1) deletes every record in DB Tables that references the specified p_location record in referencing DB Tables (eg. p_partner_location, m_extract, s_group_location).
      • 2) deletes the specified p_location record.
• Limitations
  • currently only cascading Delete is implemented, cascading Updates not yet (but certainly doable).

Aggregate Data Access Objects

• need to be manually written
• work on several DB tables at once instead of dealing with them separately (e.g. PartnerAddresses aggregator: knows how to read, write, create, delete Addresses (involves both p_location and p_partner_location DB tables)
• may be used by several UIConnectors
• will usually work with a custom Typed DataSet that holds the involved tables.

Dynamic Data Access Objects

• need to be manually written
• used wherever we need to build SQL commands on-the-fly (eg. for Find Screens)
  • can work on any number of DB tables
  • can use nested SELECTs
  • can used the whole array of ANSI SQL-92 commands
• can use eg. the 'TPagedDataSet' object that currently resides in Ict.Petra.Server.MCommon to allow returning of large amounts of datarows in 'pages'.
• may encapsulate business rules (e.g. PartnerAddresses aggregator enforces loads of them)

Database Access Object (DBAccess)

• Designed to have RDBMS abstraction abilities (RDBMS-agnostic data access) in the PetraServer
  • Details of OpenPetra's support for various Database Systems can be found here.
• Contains functions that open and close the connection to the Database
  • Designed to support connections to different kinds of databases through native .NET drivers or ODBC (both on Windows and Linux!)
  • Connection Pooling is used if a RDBMS and its .NET driver support that (PostgreSQL does and we use it)
• Allows execution of SQL statements
  • SQL statements that return data (various options)
  • SQL statements that return only one value (eg. for 'SELECT COUNT(*) FROM ...' queries)
  • SQL statements that don't return data
  • some basic 'processing' of SQL statements is done to make them work on different RDBMS's
  • ANSI SQL-92 commands must be used that are understood by all RDBMS systems that should be supported - no 'translation' of the SQL commands is done to make them work across different RDBMS's
• Supports batch execution of a number of SQL statements that don't return data (currently not in use)
• Supports management of DB Transactions
• Raises custom Exceptions in case of database errors
• Provides an extensive and configurable logging functionality

Garbage Collected Runtime Environment

• Garbage Collection (GC) removes unused Objects and Variables from the computer's memory (RAM).
  • this relieves the developer from the need to actively manage memory and to release Objects properly, thereby reducing the risk of memory leaks
• GC works automatically in the background.
• GC in .NET is very similar to GC in Java.
• Pitfalls
  • don't take it for granted that Destructors of Objects are run!
    • they might never execute if the application is shut down
    • even if Destructors are run, it cannot be predicted when this happens
    • if you need some form of cleanup or releasing of resources, do this explicitly in code before you set an Object to nil or stop using it
  • Developers have no control when memory is freed exactly
    • Indirect control
      • by assigning 'Variable = null' the developer marks an Object as being eligble for GC.
      • GC can be forced if needed. This makes sense when the developer knows that he just released a huge number of objects and that freeing memory immediately would be beneficial.

• GC guarantees that non-referenced Objects will be freed at some point - at the latest when an application exits.

Effective use of Data Types with GC

• Array with 250 elements => one Object; ArrayList with 250 elements: 250 objects - all of which need to be Garbage Collected!
• small objects may fit into the 2nd level cache of the processor - which makes access to them much faster!
• mark Objects for Garbage Collection as soon as you don't need them any longer
  • 'MyObject = null' makes an Object eligble for GC (tells the GC that it can collect [=destroy] the Object)
  • any Object that is defined locally in a procedure/function gets marked for GC automatically as soon as the procedure/function is exited in any way

'Big' business objects, with lots of functionality vs. 'small' business objects with less functionality

• 'big' business objects will have a longer lifetime since many calls will be made to them over the time
  • benefit: less business objects need to be created, more powerful and stateful business objects
  • disadvantage: business objects will stay longer in memory and won't be Garbage Collected that soon after releasing them (.NETs GC is optimised for short-lived objects)
• 'small' business objects will have a shorter lifetime
  • benefit: will keep less information in memory for a shorter time, will be Garbage Collected immediately/soon after releasing them (.NETs GC is optimised for short-lived objects)
  • disadvantages: more business objects will need to be created (but .NET is optimised for creation of small objects), business objects are not that powerful and stateless
• we initially decided that we go with the 'small' business object approach; however, we found out that in doing that we cannot reduce the amount of data that is sent over the network as effective as we can with the 'big', stateful business objects. So we ended up with the 'big' business approach - at least for the UIConnectors for the Petra Client.

GC in Client-Server Scenario

The PetraServer needs to know whether a PetraClient still needs a Business Object that is instantiated in the PetraServer.

• Client needs to hold a reference to the Server Object to prevent it from being discarded and Garbage Collected (using TEnsureKeepAlive.Register)
• Client needs to tell the PetraServer when it doesn't need the Server Object any more so that it can be Garbage Collected (using TEnsureKeepAlive.UnRegister).
• When a Client disconnects, the PetraServer closes the database connection for the Client and the Client's AppDomain is unloaded (including all Objects that were ever instantiated in it) by the PetraServer.
• The Client sends a 'Keep Alive' signal to the PetraServer at regular intervals; if the Client crashes, the PetraServer closes the database connection for the Client and tears down the AppDomain after a configurable time, thereby releasing all the memory it used (including all Objects that were ever instantiated in it)
• 'Keep Alive' signal is sent on a separate thread in the Client .EXE (realised in the TPollClientTasks class)

References

Articles:

• openPETRA Project Structure and Namespace Descriptions
• openPETRA Technical Implementation Details

Diagrams:

openPETRA Server Architecture Diagram

openPETRA Database Abstraction Layer Diagram

openPETRA Data Access Layer Diagram

openPETRA Business Object instantiation and data flow diagram

openPETRA Screen, Interfaces, Instantiator, UIConnector and Business Logic Objects Detail Diagram