I target customers who have large data processing needs. These come in
various forms, but generically look like this: the customer gets huge data
drops in some form or another and must process the data and output results in
a very specific time frame. The customer has written some scripts, maybe some
code and SQL. They have attempted some optimizations that helped a little,
but they're not meeting their timeline. They have opportunities to take on
even larger processing jobs, but don't have the capacity. They need help,
This is not an uncommon scenario. What to do? And what does this have to do
with Java? Good questions. Hold onto the Java question, I'll get to that
next. First, there are many products and frameworks for processing large
amounts of data (such as relational database management systems or RDBMSes).
But the vast majority of data that I see from d... (more)
There are two major drivers behind the need to embrace parallelism: the
dramatic shift to commodity multicore CPUs, and the striking increase in the
amount of data being processed by the applications that run our enterprises.
These two factors must be addressed by any approach to parallelism or we will
find ourselves falling short of resolving the crisis that is upon us. While
there are data-centric approaches that have generated interest, including
Map-Reduce, dataflow programming is arguably the easiest parallel strategy to
adopt for the millions of developers trained in serial... (more)
The multi-core buzz is everywhere. Pick up a newspaper and the local
electronics mega-store is advertising multi-core desktops and laptops to the
consumer. Interesting, but what does it mean to the everyday Java programmer?
Maybe nothing. If you live in the application server world writing EJB-based
applications your application server does most of the heavy lifting for you.
It handles concurrency just fine. But that doesn't cover all applications.
Multi-core technology will especially affect applications that must process
large amounts of data in a non-transactional (outside of ... (more)