'Preparing' SQL Statements And Specifying SQL Statement Time-outs

Overview

This page deals with the advanced DB access topics of Prepared Statements and custom SQL Statement time-outs. Please note that both are used and needed only in special circumstances.

Prepared Statements

'Preparing' a SQL query lets the software engineer optimise the performance of frequently issued queries.

What a RDBMS does when it encounters a 'Prepared' SQL query is that it 'caches' the 'query execution plan' and uses this cached information in subsequent calls. Since the calculation of a query execution plan takes time, and since spending that time is no longer needed in subsequent runs of the same query, it makes subsequent runs of the same SQL query faster. While the caching of the 'query execution plan' takes a bit of time when the SQL statement gets 'prepared', this is one-off and pays off with subsequent calls to the same query.

A SQL query that uses Parameters and gets 'prepared' is treated as 'prepared' even with varying values for the Parameters and hence fully benefits from the speed-up that a Prepared Statement can yield!
The 'prepare' step itself is always performed before an SQL query is executed. As you don't want that step to be repeated each time, you need to 'prepare' the statement before you enter the loop.
As long as the cursor is not marked as 'finished or closed', the cached execution plan continues to be valid.
If 'finished/closed' SQL queries are called again, they are treated as 'unprepared' SQL queries and therefore are significantly slower, as each time the 'prepare' step must be performed again.
Therefore all SQL queries that are called within a loop or with changing parameters should always be 'prepared' statements with their 'finish' clause after the point where the loop has ended.

query->prepare()
start loop

query->execute()

end loop
query->finish()

Prepared Statements are not supported by all RDBMS, but should simply be executed in a non-prepared way in case a RDBMS doesn't support it. PostgreSQL definitely supports Prepared Statements.

It should be noted that there is no inter-relation between 'prepared' statements and custom SQL Statement time-outs.

Customising SQL Statement Time-outs

Every DbCommand that is run against the DB has got a time-out associated with it. If the command execution takes longer than the time-out, the ADO.NET provider for the RDBMS that we are connected to will throw an Exception.

In case of the PostgreSQL RDBMS we use the Npgsql ADO.NET provider, and this throws an Exception when a command takes longer than the default, which is 20 seconds: Npgsql.NpgsqlException: ERROR: 57014: canceling statement due to statement timeout

When a single SQL query takes longer than 20 seconds on a machine where both the PetraServerConsole.exe and the RDBMS are located on the same machine (which is the normal deployment scenario),

• it points to either a rather inefficient SQL query (perhaps it needs to use one or more Indexes for better performance), or
• the SQL query is simply attempting to do 'too much in one go'.
  There are of course plenty of ways to optimize queries - next to splitting the query into smaller parts - and make the DB’s optimizer choose the right method
  such as using indexes, optimizer hints or simply put the joined tables in the right order.

If you can't optimise the speed of the SQL query and can't break down what the SQL query does and you are running into Exceptions because of Command time-outs then the time-out should be manually increased for that one query. See below on how to do that.

It should be noted that there is no inter-relation between 'prepared' statements and custom SQL Statement time-outs.

Prerequisite for Prepared SQL Statements and SQL Statement Time-outs: Using the DBAccess.GDBAccessObj.SelectUsingDataAdapter Method

Call the special DBAccess.GDBAccessObj.SelectUsingDataAdapter Method to execute the query. It has got the optional Arguments APrepareSelectCommand and ASelectCommandTimeout.

• Pass in true into APrepareSelectCommand to make the query a 'prepared' query.
• Pass in an int value higher than 20 into ASelectCommandTimeout to increase the SQL statement time-out.

The Method will return the number of Rows successfully added or refreshed in the DataTable passed in with the AFillDataTable Argument.

Also, it will provide you with an instance of the TDataAdapterCanceller Class in the out Argument ADataAdapterCanceller. See Stopping of Running DB Queries for details.

Read the XML Documentation of the DBAccess.GDBAccessObj.SelectUsingDataAdapter Method and check where it is used so far for details on how to use this Method correctly!

Restrictions

Supported Only for Queries That Are SELECT Queries

The DBAccess.GDBAccessObj.SelectUsingDataAdapter Method can only be used for SELECT SQL queries, but not for other types of SQL commands, e.g. UPDATE or for scalar-type queries.

Only Available When Using the DBAccess.GDBAccessObj.SelectUsingDataAdapter Method

At the time of writing (March 2015) no other Methods of the TDataBase Class can prepare a SQL query or can alter a SQL statement time-out. This means that our 'Typed DataStore' Methods (eg. PPartnerAccess.LoadXXX) can't utilise those features either.

Historic Information: How this was done before Co-ordinated DB Access got implemented

While it has previously been possible to call the public PrepareNextCommand and the public SetTimeoutForNextCommand Methods of the TDataBase Class, that option had to be removed as the calling of these Methods is not working reliably in the light of multi-threaded DB Access (they each set a Field of the TDataBase Class to a certain value, and that value could get picked up by a Method that executes on another thread than the thread that set the values!).

To achieve thread-safety the DBAccess.GDBAccessObj.SelectUsingDataAdapter Method was written. This Method guarantees thread-safety as it 'wraps' the calls to the now private Methods PrepareNextCommand and SetTimeoutForNextCommand in a thread-safe scope.

Should the options for using Prepared Statements and custom SQL Statement time-outs be extended to other means of DB access, similar 'wrapper' Methods would need to be written.

Questions?

In case you have questions regarding the features that get explained on this page please contact ChristianK, the developer of those features.