The QDBase Solution

High-Performance Query

To achieve high-performance queries with a large volume of data, QDBase stored the current detail data in a physically ordered manner based on account numbers and created an account number index.

This approach enabled quick retrieval of all data for a specific account number by ensuring the data was read continuously, reducing disk I/O.

The business entity code table was stored separately and preloaded into memory, and then associated with the account data using an in-memory pointer reference method.

This real-time association eliminated the need to pre-associate the data, resolving the code refresh issue.

Enhanced Concurrency

By optimizing data storage and retrieval, and performing real-time association of data, QDBase was able to achieve a significant increase in concurrency.

QDBase efficiently handles concurrent requests by ensuring data is readily available and processing is optimized, resulting in a 6-fold increase in concurrent access capability compared to the previous ES system using the same hardware.

QDBase achieves this by loading only the relevant account data into memory and performing associations there, instead of loading the entire dataset to process the queries.

Flexibility and Reduced Maintenance

QDBase's real-time association of data provided flexibility by removing the need for pre-association of the business entity code table with the current details. This eliminated the need for data refreshes when business entities changed, and also solved the issue of slow restarts. QDBase can restart in 1-2 seconds, and code table maintenance takes less than 1 second.

These changes provide an easier to manage and more flexible system that can more quickly adapt to changes in data requirements.

QDBase Feature Highlights

• Physically Ordered Storage: By storing data after sorting them by certain fields, QDBase unlocks high-performance algorithms, and utilize the ordered feature of data to reduce computing complexity.
• High Performance Algorithms: Built-in high-performance support for many common and complex algorithms, including mechanisms such as fast file access, file index, and pointer reference association.
• Foreign Key as Pointer: A table is preloaded into memory, and associations with detailed data are done through in-memory pointer references, which provides faster association than traditional join operations.
• JDBC Driver: QDBase provides a JDBC driver that allows the front-end application to easily switch to QDBase from other systems such as ES with minimal changes, enabling easy integration with existing systems.

Conclusion

QDBase provided a superior solution by addressing the limitations of the existing systems with its high-performance, low-latency, and flexible design.

The system achieved significant performance improvements by using physically ordered storage and in-memory processing, along with other QDBase features.

The result was a system that is not only faster and more scalable, but also easier to maintain and adapt to change. The real-time data association capabilities of QDBase eliminated the need for time-consuming data refresh operations, and the flexible infrastructure reduced the maintenance overhead.

By leveraging the high-performance algorithms built in to QDBase, the bank was able to provide its customers with current data in real-time, while also taking advantage of reduced cost.

Read the full in-depth study, with code examples on the SCUDATA Blog