Simple linux iptables configuration script

I had the necessity to firewall some VMs after installing a cluster of the awesome CockroachDB, since I couldn’t find out how to password protect the admin interface.

 

The idea is to drop all connections by default from all machines not in the cluster pool, while allowing unrestricted communication between machines within the pool from any port.

The script, found here, isn’t actually that complicated and since I’m not an expert I might have missed something, but it does includes many of the little details non-professional sysdamin like me tend to miss out:

  • ipv6 icmp ports
  • outgoing dns rules
  • ipv4:6 passtrough rules
  • a proper “reset all” option

 

The configuration is really simple and tuned toward making adding/removing machines from the pool easy across each of them without maintaining dozen lines for each input/output/protocol tuple, i.e

 

#allow input/output communication trough port 80
ALLOW 80
ALLOW 443
#allow all communication from the specific host
HOST 192.0.2.23
#same but for ipv6 addresses
HOST6 2001:DB8::4860:3242

 

 

Hope you find this useful! It’s already in init.d format. Please do tell if you see any improvement/suggestion to make it more robust/secure!

A word of warning: do not use this with Docker! The reset option will nuke its port forwarding rules.

SysUsage: visualize all the stats of your vms

There are plenty professional monitoring solutions that provide alerts and monitoring facilities for Linux servers.

They are mostly optimized for server administration and make it a little to hard for developers to understand their application resource usage under constrained environments.

It is especially annoying that many of them only fetch data from SMNP agents, making unnecessarily hard to have your own data plotted along other interesting system stats.

Enter SysUsage:

week

This monitoring tool available from here is easy to setup, immediate to use and support plugging in your own data easily.

Read more

Eclipse IDE configuration: TypeScript in a Java EE/Maven pipeline

Editor’s Note: I did this experiment to test out how putting together everything under a single IDE in a single project worked; I discourage this single project approach and I suggest instead to split the web front-end into a separate project, built with an appropriate toolchain.

FrankenIDE

Moreover, while Eclipse TypEcs works fine, you’d be probably better off with a web-first IDE, like JetBrains WebStorm

For the curious developer, this blog post describe what the setup and the development experience of a single Maven war/TypeScript project looks like, so you can try it yourself and see if it suites your personal tastes.

Read more

Apache Velocity for SVG

In our experiment on using Bluemix, we’ve created a crowd sourced knowledge repository of concept maps at conceptmaps.info.

One of the feature that was designed from the beginning was the ability to have automatically generated image from third party knowledge sources, as Wikipedia.

As proof of concept we’ve created a simple mechanism working off Wikipedia. The description of a page as returned from their API is very complex especially as we were more interested to the relationship of the page with other content, not just the knowledge contained by itself. As proof of concept we’ve created a simple mechanism:

  1. using the Wikipedia API we get the content of the page
  2. we extract the “see also” links
  3. we follow each link and get an abstract of the page
  4. we create a dynamic map containing the links and the extract as description of each topic

The result is a simpler version of the more complex Wikipedia knowledge, and while basic is easy to understand and easier to navigate. We used a Wikipedia-like palette for the colours and added a modified version of our logo in which one part of it is replaced by the wiki logo to make clear where the content comes from – including the appropriate sourcing link on the bottom on the page, as required form the Wikipedia guidelines on using their content.

conceptMap_Pict_03
Auto generated map on concept map.

 

To create such a dynamic map, we started creating a base version of the above sample image in SVG, with empty content. Since an SVG is also XML it was easy to modify and convert into a Velocity template: we used this generic empty image as our template, adding some logic in it to handle variable number of boxes and then adding all the necessary placeholders to hold the content.

Again logic implemented is very basic, the dynamic parts are:

  • number of topic and related content
  • size of the font of the text in the boxes
  • number of boxes and arrows ( 1 – 6 )

Since in conceptmaps.info the navigation and user interaction on a map is governed by JavaScript using a json descriptor, we had to generate that descriptor as well.

Dynamic logic in the templates

Font size

The size of the font is calculated to maintain the readability and the format of the image, without having to modify the boxes. Changing the size is needed because, being generated from an outside source, we had to handle some very long text, as in this map about naturally processed languages.

Here is the control logic we used:

#define( $fontsize )
#if ( $linksize >= 15 && $linksize < 20 )
18px
#elseif ($linksize >= 20)
16px
#else
24px
#end
#end

The linksize var is set at the beginning of the template like so:

#set ($linksize = $wiki.getLink(0).length())

Then the fontsize “function” is called like a normal variable:

<div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: $fontsize; ">
 <div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">$wiki.getLink(0)</div>
</div>

With this simple trick the image looks good even if the link is quite long and wouldn’t fit nicely in the box.

Dynamic adapt number of boxes and arrows

The map contains multiple boxes, one each topic, so we had to cull out from the image the boxes and link that weren’t in use. We do this in a specific order, so that the map looks ‘balanced’ visually.

Again the logic behind this is quite simple, just an if statement based on the number of links contained in the object passed in the context that guards each and every box:

#if ($wiki.getLinksSize() > 1 || $debug)

An interesting strategy we needed was to use a $debug variable, used to have them all displayed… Because at easy as it seems, it was quite difficult to correlate arrows and boxes from the SVG transformation alone, as when it got out of the editor we used the shapes were all over the xml in random order: we had to restructure the SVG into separate part, ordering and grouping together arrows and boxes.

Final thoughts

In this article we’ve briefly described how to use the velocity template engine, to create dynamic images, using the svg format, which is a very powerful and flexible image format.

To learn more about svg why not look at the svg map itself, dynamically generated on conceptmap.info using the tools described in this article 🙂

 

Node.js client-server deployment pipelines on Bluemix

On the road learning node.js I stumbled upon the awkwardness of having two engines serving http requests on different paths, say, for example, having calls to /html served by Express and to /api served by hapi.js

Since I’m just testing these frameworks out, I didn’t want to completely separate the source tree into independent repositories, especially because I’d need them to update in lockstep for as long as the client/server API is not stable.

Turns out the Bluemix deployment pipelines offered by hub.jazz.net  are quite flexible, allowing for custom commands to be configured, so I set out to create a split pipeline configuration out of a twin project folder setup.

I started with creating an additional node for the client code:

 

The Server and Client nodes

Read more

Writing testable code

As written in The Mythical Man Month you either plan half of your schedule for testing or spend twice that debugging.

 

However writing tests is not easy. Mocking strategies and full system tests allow one to easily orchestrate the parts of the application that cannot be unit tested, but they come at a significant increase in complexity and cost, especially when you need the full application stack up and running to test a couple lines of code.

Integration tests are slow and costly
Integration tests are slow and costly to develop and to run. Do you really need so many of them?

Wouldn’t it be better just to write testable code in the first place?

Read more

Creating a basic visual editor with svg.js – Part 1

 

Creating a browser based visual editor is not an easy task. The user workflow for editing the graphical elements, rotating them, selecting their colors and shape contains so many alternative paths and features to become easily overwhelming for developers that just want a quick and direct solution to get a project going.

This miniserie will describe our problem, our solution and the issues we had to solve while creating our own online svg editor.

Read more