I’ve recently been playing around with the ELK stack (now officially known as the Elastic stack) collecting data from an IRC channel with Elastic’s Logstash, storing it in Elasticsearch and analysing it with Kibana. But, this isn’t an “ELK” post - this is a Kafka post! ELK is just some example data manipulation tooling that helps demonstrate the principles. As I wrote about last year, Apache Kafka provides a handy way to build flexible “pipelines”.

Continue Reading

On a family holiday in South Devon last week I had some good food experiences. Top of the pile was a Lardy Cake, a delicacy new to me but which I’ll be sure to be searching out again. It reminded me of an Eccles cake, but bigger and lardier! I got mine from Vinnicombes on the High Street in Sidmouth. Just down the coast from Sidmouth is a village called Beer.

Continue Reading

Timelion was released in November 2015 and with the 4.4.2 release of Kibana is available as a native visualisation once installed. It adds some powerful capabilities to Kibana as an timeseries analysis tool, using its own data manipulation language. Installing Timelion is a piece of cake: ./bin/kibana plugin -i kibana/timelion After restarting Kibana, you’ll see it as an option from the application picker There’s a bit of a learning curve with Timelion, but it’s worth it.

Continue Reading

OBIEE supports JDBC as a connection protocol, using the driver available on all installations of OBIEE, bijdbc.jar. This makes connecting to OBIEE from custom or third-party applications very easy. Once connected, you issue “Logical SQL” against the “tables” of the Presentation Layer. An example of logical SQL is: SELECT "Time"."T05 Per Name Year" saw_0 FROM "A - Sample Sales" To find more Logical SQL simply inspect your nqquery.log (obis-query.log in 12c), or Usage Tracking.

Continue Reading

OK, maybe that’s not entirely true. But my read-only client, certainly. I was perusing the Logstash input plugins recently when I noticed that there was one for IRC. Being a fan of IRC and a regular on the #obihackers channel, I thought this could be fun and yet another great example of how easy the Elastic stack is to work with. Installation is a piece of cake: wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/2.2.1/elasticsearch-2.2.1.zip wget https://download.

Continue Reading

One of the perks of my job is that I get to travel to some pretty nice places (hi, San Francisco, Bergen, Åland Islands), and get to eat some pretty good food too. If you’re looking for some techie content, then move along and go and read about Kafka, but if you enjoy food pr0n then stay put. I was working for a client in the centre of Manchester this week, staying as usual at a Premier Inn.

Continue Reading

I got this lovely failure during a fresh install of OBIEE 11.1.1.9. I emphasise that it was during the install because there’s other causes for this error on an existing system to do with corrupted credential stores etc – not the case here. The install had copied in the binaries and was in the process of building the domain. During the early stages of this where it starts configuring and restarting the AdminServer it failed, with the AdminServer.

Continue Reading

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0. The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov. Fortunately it’s still possible to run this setup out of the box in BDL 4.

Continue Reading

I’ve always defaulted to Slideshare for hosting slides from presentations that I’ve given, but it’s become more and more crap-infested. The UI is messy, and the UX sucks - for example, I want to download a slide deck, I most definitely 100% am not interested in “clipping” it…even if you ask me every. damn. time: Looking around it seems the other popular option is Speakerdeck. The UI is clean and simple, and I as both a user and uploader I feel like I’m there to read and share slides rather than be monetised as an eyeball on the site.

Continue Reading

#obihackers There’s a #obihackers IRC channel on freenode, where a dozen or so of us have hung out for several years now. Chat is usually OBIEE, Oracle, ODI, and general geek out. Bear in mind this is the equivalent of us hanging out in a bar; if you wanna shoot the shit with a geeky question about OBIEE go ahead, but if you’ve come to get help with your homework without even buying a round, you’ll probably get short shrift… ;-)

Continue Reading

InfluxDB is a great time series database, that’s recently been rebranded as part of the “TICK” stack, including data collectors, visualisation, and ETL/Alerting. I’ve yet to really look at the other components, but InfluxDB alone works just great with my favourite visualisation/analysis tool for time series metrics, Grafana. Getting data into InfluxDB is easy, with many tools supporting the native InfluxDB line input protocol, and those that don’t often supporting the carbon protocol (from Graphite), which InfluxDB also supports (along with others).

Continue Reading

Thanks to the power of twitter, I can look back on all the many and varied Full English breakfasts that I’ve (mostly) enjoyed: https://twitter.com/search?q=rmoff%20%23fullenglish&src=typd What makes a good Full English? Good ingredients, cooked well. Nothing worse than a limp pink sausage, as it were. Sausage, standard pork or Cumberland at most. Definitely no daft apricot and guava bean with a hint of foie gras nonsense. Must be cooked right well, crispy skin, almost burnt.

Continue Reading

It struck me today when I was writing my most recent blog over at Rittman Mead that I’ve been playing with visualising OBIEE metrics for years now. Back in 2009 I wrote about using something called JManage to pull metrics out of OBIEE 10g via JMX: Still with OBIEE 10g in 2011, I was using rrdtool and some horrible-looking tcl hacking to get the metrics out through jmx :

Continue Reading

Continue Reading

See talks

Continue Reading

You can find all my recent talks on Notist at https://talks.rmoff.net/. Older ones are on Speaker Deck. Below are links to particular talks I’ve done recently, along with recordings where available. From Zero to Hero with Kafka Connect 📖 Slides 👾 Code 📽 Recording The Changing Face of ETL: Event-Driven Architectures for Data Engineers 📖 Slides 📽 Recording Strata London 2019 (available soon) Paris Apache Kafka Meetup @ DataDog ATM Fraud detection with Kafka and KSQL 📖 Slides 👾 Code 📽 Recording (live @ London Apache Kafka Meetup) 📽 Recording (live @ Milan Apache Kafka Meetup) 📽 Recording (webinar) No More Silos: Integrating Databases and Apache Kafka 📖 Slides 📽 Recording Kafka Summit New York 2019 Online talk 👾 Code (MySQL) 👾 Code (Oracle) Embrace the Anarchy: Apache Kafka’s Role in Modern Data Architectures 📽 Recording Devoxx UK Devoxx Belgium Rated 32 out of all the conference talks Scored \( 4.

Continue Reading