Recently, I was working on a project that was coming to a close. It was related to optimizing a database using a Java based in-memory cache to reduce the load. The application had to process up to a million objects per day and was characterized by its heavy use of memory and the high number of read, write and update operations. These operations were found to be the most costly, which meant that optimization efforts were concentrated here.
The project had already achieved impressive performance increases, but one question remained unanswered - would changing the JVM increase performance?
Continue reading "When optimizing - don't forget the Java Virtual Machine (JVM)" »
When decisions have to be made quickly? Access to real-time data is one of the key considerations in almost every corporation. Taking strategic decisions based on out-of-date data can produce painful results. Imagine a stockbroker who works using data that is ten minutes out-of-date – costly mistakes will occur as others react to market events quicker. Very strict requirements about the acceptable latency for decision-making data, where every second is important, force companies to find new solutions that can meet these expectations.
Data from DBMS can be extracted in many different ways using SQL, table dumps or use of application that sits over the database. These solutions are suitable in such scenarios, but the question to be asked is – can they really deliver data in near real-time? I doubt it. The high computation cost of processing large amounts of data and the time needed for data transfer make these solutions too slow.
Continue reading "Real-time data integration using Change Data Capture" »