Exporting database tables to csv files with Apache Camel

Below the interested part of code using spring xml <bean id="ds-patriot-dw_ro" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="oracle.jdbc.OracleDriver" /> <property name="url" value="jdbc:oracle:thin:@//patriot.redaelli.org:1111/RED"/> <property name="Username" value="user"/> <property name="Password" value="pwd"/> </bean> <camelContext id="MyCamel" streamCache="true" xmlns="http://camel.apache.org/schema/spring"> <route id="scheduler"> <from uri="timer:hello?repeatCount=1"/> <setHeader headerName="ndays"> <constant>0</constant> </setHeader> <to uri="direct:start"/> </route> <route> <from uri="direct:start"/> <setBody> <constant>table1,table2,table3</constant> </setBody> <split streaming="true"> <tokenize token="," /> <setHeader headerName="tablename"> <simple>${body}</simple> </setHeader> <to uri="direct:jdbc2csv"/> </split> </route> <route> <from uri="direct:jdbc2csv"/> <to uri="direct:get-jdbc-data" pattern="InOut" /> <to uri="direct:export-csv" /> </route> <route> <from uri="direct:get-jdbc-data"/> <log message="quering table ${headers....

May 24, 2019 · 1 min · 175 words · Matteo Redaelli

Querying public knowledge graph databases

You can query public knowledge graph databases (like wikidata.org and dbpedia.org) using SPARQL. For instance for extracting all “known” programming languages, you can use the query SELECT ?item ?itemLabel WHERE { ?item wdt:P31 wd:Q9143. SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO\_LANGUAGE],en". } } LIMIT 1000 There are also SPARQL clients for most of programming languages. With (swi) prolog you can easily run \[library(semweb/sparql\_client)\]. sparql\_query('SELECT ?item ?itemLabel WHERE {?item wdt:P31 wd:Q9143. SERVICE wikibase:label { bd:serviceParam wikibase:language "\[AUTO\_LANGUAGE\],en"....

August 18, 2018 · 1 min · 82 words · Matteo Redaelli

Sample microservice for exposing database tables via REST using node.js express.js in a docker container

I have started learning node.js: The first result is this github repository where you can find a basic project of a rest web service that exposes oracle data. How to run the service forever (with autorestart on failures)? I tested it both with a docker container and the pm2 tool

February 9, 2017 · 1 min · 50 words · Matteo Redaelli

Apache Spark howto import data from a jdbc database using python

Using Apache spark 2.0 and python I’ll show how to import a table from a relational database (using its jdbc driver) into a python dataframe and save it in a parquet file. In this demo the database is an oracle 12.x file jdbc-to-parquet.py:``` from pyspark.sql import SparkSession spark = SparkSession \ .builder \ .appName(“Python Spark SQL basic example”) \ .getOrCreate() df = spark.read.format(“jdbc”).options(url=“jdbc:oracle:thin:ro/ro@mydboracle.redaelli.org:1521:MYSID”, dbtable=“myuser.dim_country”, driver=“oracle.jdbc.OracleDriver”).load() df.write.parquet(“country.parquet”)

October 27, 2016 · 1 min · 66 words · Matteo Redaelli

About Cayley a scalable graph database

This is fast tutorial of using the Caylay graph database (with MongoDB as backend): Cayley is “not a Google project, but created and maintained by a Googler, with permission from and assignment to Google, under the Apache License, version 2.0” download and unzip a binary distribution edit cayley.cfg { "database": "mongo", "db\_path": "cayley.redaelli.org:27017", "read\_only": false, "host": "" } ./cayley init -config=cayley.cfg ./cayley http -config=cayley.cfg -host=“” & create a file demo....

August 3, 2015 · 1 min · 123 words · Matteo Redaelli