Camel

Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. Apache Groovy is a Java-syntax-compatible object-orientedprogramming language for the Java platform. It is both a static and dynamic language with features similar to those of Python, Ruby, and Smalltalk. It can be used as both a programming language and a scripting language for the Java Platform, is compiled to Java virtual machine (JVM) bytecode, and interoperates seamlessly with other Java code and libraries....

Below the interested part of code using spring xml <bean id="ds-patriot-dw_ro" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="oracle.jdbc.OracleDriver" /> <property name="url" value="jdbc:oracle:thin:@//patriot.redaelli.org:1111/RED"/> <property name="Username" value="user"/> <property name="Password" value="pwd"/> </bean> <camelContext id="MyCamel" streamCache="true" xmlns="http://camel.apache.org/schema/spring"> <route id="scheduler"> <from uri="timer:hello?repeatCount=1"/> <setHeader headerName="ndays"> <constant>0</constant> </setHeader> <to uri="direct:start"/> </route> <route> <from uri="direct:start"/> <setBody> <constant>table1,table2,table3</constant> </setBody> <split streaming="true"> <tokenize token="," /> <setHeader headerName="tablename"> <simple>${body}</simple> </setHeader> <to uri="direct:jdbc2csv"/> </split> </route> <route> <from uri="direct:jdbc2csv"/> <to uri="direct:get-jdbc-data" pattern="InOut" /> <to uri="direct:export-csv" /> </route> <route> <from uri="direct:get-jdbc-data"/> <log message="quering table ${headers....

Bigdata projects can be very expensive and can easily fail: I suggest to start with a small, useful but not critical project. Better if it is about unstructured data collection and batch processing. In this case you have time to get practise with the new technologies and the Apache Hadoop system can have not critical downtimes. At home I have the following system running on a small Raspberry PI: for sure it is not fast ;-) At work I introduced Hadoop just few months ago for collecting web data and generating daily reports....

In the article Apache Camel: how to collect data from twitter, I’ll show howto save all tweets about Pirelli to files in .json format ..

In the article building system integrations with Apache Camel I’ll show how to create in 10 minutes an integration between two databases (without writing any lines of java or c# code): looking for uses in the database MOODLE (mysql) with missing attributes for each of that users retreiving the missing attributes from the database UPMS (m$ sql server) and then adding the missing attributes to the database MOODLE I’ll use...