In the Financial Service Industry, every kind of financial business transaction involves many complex nested transactions. An extremely wide range of applications depend on session reconstruction as a key first step, enabling everything from service level monitoring to ensuring regulatory compliance and detecting fraud.

Data Access and Formatting

For important processes such as service level monitoring, it is critical to have the data prepared and aggregated into a suitable format. Unfortunately, application and business transactions do not readily supply such data. Instead, the data have to be inferred or calculated from a number of accessible monitoring points, because even a single business transaction often comprises many sub-transactions and operations. These sub-transactions often involve many separate, distributed applications executing across a network of disparate servers, with different operating systems and application suites.

The input data for these processes come from a wide range of applications and related monitoring points. A common and valuable source is log file data, as most applications (and indeed many network or other service elements and devices) can be configured to generate log files.

Data Sources and RAMMS Monitoring

The SQLstream RAMMS solution continuously monitors all of the log files and combines the records in such a way as to perform session reconstruction for each service session.

The SQLstream transaction/session reconstruction solution reads and tails the log files continuously, processeing the data even while the log files remain open with records being appended. The SQLstream solution includes a log-file adapter to perform this task, configurable using the SQL:2003 SQL/MED standards-based adapter.

Other sources of session data include SNMP MIBs (Simple Network Management Protocol 's Management Information Base). MIBs offer a standards-based way for hardware devices (and, increasingly, software systems) to share their current status and usage information. SQLstream can poll MIBs at pre-configured time intervals, or the MIBs can be set to proactively generate messages, a method called SNMP Traps. A SNMP SQLstream plug-in can allow SNMP MIBs and traps also to become sources of messages to feed into session reconstruction. Other data might be available in external RDBMS databases, such as the application or subscriber details, such as allocation of IP addresses and ports to applications or subscribers.

Network probes can be another source of relevant data for the SQLstream session reconstruction solution. Such probes read the network packets promiscuously, meaning that every packet passing by the NIC (Network Interface Card) is read, rather than just those of the current TCP/IP connection. In one scenario, such a probe is deployed to parse the network packets and generate service records corresponding to the protocols observed (such as “http get”). Web-server proxy logs can also be mined to reveal ip-addresses and corresponding URL strings and timestamps.

SQLstream Results

SQLstream can combine all of the information from these disparate data sources by joining them within relevant time windows, thereby pulling together a complete view of each session. To do so, SQLstream performs fairly sophisticated SQL processing, such as the following tasks:

• Allocating session ids for each new session
• Managing and tracking sub-sessions of sessions
• Creating time-outs to cover cases of extended inactivity
• Joining IP addresses with web proxy logs to place the original URL string in those session records