About Maps and Data

This is an omnibus article on data visualisation using maps. I went back to map visualisation solution in the autumn of 2014. This maps tag retrieves the the articles on this wiki. My current interests is in web hosted maps visualisations and thus in wordpress. I have also found this article, 10 Free Tools For Creating Your Own Maps. A couple of these look very interesting. UMapper and HeatMaps look rather fun. I have also experimented with Tableau Software’s public aka community release. Continue reading “About Maps and Data”

Pidgin & TRM Runes

While trying to configure pidgin to act as a console for TRM once known as Reuters’ Messenger, I came across this page at their sourceforge site which documents the magic runes, critically the User Name.

Yes, me. Since version 1.3.4

Params: Anvanced tab: nothing basically. Connection type – auto. UNCHECK “Use Single Sign-On” (!) Otherwise it’ll use your Windows Domain credentials with Reuters network, which is rather incorrect.

Basic tab:
Username: first.last.company.com@reuters.net
Login: first.last@company.com

(Username is basically derived from the login name by replacing @ to dot and then adding @reuters.net)
Don’t forget that you may have unique password in Reuters network, not the same as your PC/Domain password.

They both tightened up on their SSL management , this page refers.

…..there have been considerable changes to the SSL handling in the NSS module. One part of them seems to be that the default for SSL cipher suites now follows the recommendation to disable known unsecure suites, e.g. that use RC4.

You can change the settings with Tools -> Plugins -> NSS Preferences. On my company account, which is a OCS2007 installation, I needed to enable at least TLS_RSA_WITH_RC4_128_SHA or TLS_RSA_WITH_RC4_128_MD5 before SSL connections went through again.

However, don’t forget to keep your SIPE libraries up to date!

Thunderbird, use thereof

This now seems pretty ubiquitous on my PC/Laptops since I moved my personal mail from Outlook to TB on my works machine to minimize errors. I have it installed on three systems, two windows and one Mac. I really need one set of filters, possibly most importantly to enforce deletions and filing decisions.

  1. Make some IMAP folders so that the file folders can be shared.
  2. Write some filters
  3. Share them accross machines

Writing complex filter condition and actions is not easy, but tags are our friend.

I found the following links about filter sharing,

Log Rotate and *NIX

This has changed since my days at Pyramid. There has been quite a lot of thinking and engineering about dealing with daemon logs. The places to look are via the syslogd, newsyslog project, you can see a the newsyslog man pages and using the keyword “rotate log”. The idea is that logs are rotated, although the names are not and that old logs are deleted. So the newest is called mypackage or mypackage.0 and the older logs increment the counter. Logs are generally kept in /var/log or /var/log/package_name and not in the package or product install tree. What a nob? There is a command ‘logrotate’; it can be installed using macports. It needs a a configuration file. Continue reading “Log Rotate and *NIX”

Scripting on the Mac

It’s UNIX, how hard can it be? I want to run some Macports uodate scripts regularly. To me the obvious programming language is bash, but I got excited and decided I want to notify that the script had run, and run OK.

There are two schemes, Growl and OSX Notification system. These both have language bindings.

See also the wikipedia Growl page and the Growl home page

So most roads lead to Applescript, and here is the Apple developer resource, and here is the man osascript page which allows applescript calls from the command line, and hence from within a script. I have bookmarked the applescript manual.

The shell call osascript takes a program file argument.

ooOOOoo

 

Adobe installs and updates

I went through a period where I am notified of a need to update an adobe product, usually flash and go through a loop of an install page and the home page; I was unable to update the tools.

I finally checked the Ghostery report, and paused the blocking since one of the objects blocked came from Adobe. This then allowed the download page to display the McAfee spamware opt-out check box and it all works properly. Mozilla Firefox is my default browser; I have installed ghostery

RDBMS theory

I feel the need to checkpoint this. Life is getting confusing.

Barry Morris, CEO of Nuodb, has written a series of artilces about the “Holy Grail”, which he published at the Cloud Computing Journal, and somewhere within the NuoDB site.

The most import ant contribution that Morris makes, in my mind is that there are four models of scale out RDBMS. (Shared Disk, Shared Nothing, Synchronous Commit and their own Durable Distributed Cache invented, (or maybe substantiated), by Jim Starkey.)

Unsurprisingly, Morris’ third article extolling the superiority of what he has to sell  does not, as far as I can see describe how the consistency property is met. I need to re-read the MVCC part of the article. MVCC is based on a file/item append model. MVCC obviates locks (How?) and thus removes a massive part of the seriality of a DBMS which is good because not only do we have Brewers Theory to deal with, but also Amdahl’s Law. The un-answered question to me is how does the relevant cache partition ensure that the page copy it gets from a remote node is the most recent and not required to be locked for update? He states the relationships are asynchronous between nodes, so we are back to eventually consistent, it would seem.

From Morris’ article we learn that NuoDB (like MarkLogic?) and in fact like MySQL where Starkey worked for a while consists of a Transaction Engine and a Storage Manager entity.

Morris mentions Google F1, which is used to support their ad keywords database.It is based on Google’s Spanner which seems pretty much their answer to the CAP theorum, we’ll have to see what the latency cost is like, but being Google it may not be publicly open source.

Morris’ article does not reference Brewer’s CAP theory. I have collected the following links tagged Brewer,

At some stage  I found the proof that the CAP theorem was a theory, I think the Barnes article above references it.

Can we break Brewer’s theory?

I need a, personally, accessible definition of Consistent, Available and Partition Aware. (The first two are easy). Although the wikipedia entry, CAP Theorum has a pretty good set of definitions

In theoretical computer science, the CAP theorem, also known as Brewer’s theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:[1][2]

  • Consistency (all nodes see the same data at the same time)
  • Availability (a guarantee that every request receives a response about whether it was successful or failed)
  • Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system)

It’s likely I suppose that we might engineer to ensure that the failing condition is so trivial it can be ignored.

The commonest compromise is between availability and consistency although eventual consistency is a relatively modern construction.

Shared disk clusters engineered for HA on a fail fast and recover algorithm are a solution that fails the Availability requirement of the CAP theorum although they have a zero RPO and can have relatively short RTOs.

Here’s the sponsored Bloor paper on NuoDB.

The Jim Starkey wikipedia article references a 2012 patent that patents “A multi-user, elastic, on-demand, distributed relational database management system.” We’ll see? Probably the patents that protect the Nuodb products.

ooOOOoo

The NHS have decided to replace Oracle with RIAK for the “spine”. This claims partition tolerance and availability.

http://www.aerospike.com/ is another hi-performance, scale-out database.

When considering XML/RDF optimised databases, I have been pointed at Virtuoso, which has a wikipedia page here. and a white papers page here.

RCS

Revision Control System, one of the grand daddies of them all.

I still use it because its simple and does not have a network interface.

Links

rcs -i creates the repo, rcs -l will turn locking on, ci -l checks the file in. If locking is not on, no version message is prompted. If locking is not on, the file if checked in is deleted from the home directory. rlog will display the version history. So


cat $file | rcs -l | ci

$Header$ is an omnibus with most of what one needs and $Log$  displays the log messages.

I need to experiment with the $Revision$ keyword, can this be subsituted into code.

Upgrading MacOS X

And on to 10.9

There was a rumour that it was best to stop at 10.8, Mountain Lion but by the time I got round to it htis wasn’t an option. The Appstore only permitted an upgrade to Mavericks; is this a cat?  The Appstore upgrade works fine.

And now onto Macports which is a pain, and I document on my Macports page.

Filevault is a constraint as well. You’ll need a better answer than I used if any of your pre 10.8 users had Filevault encrypted home directories.

Continue reading “Upgrading MacOS X”

ERP

Obviously ERP has been around a while, in my researches, I came across these two open source ERP packages; they obviously run on distributed systems platform.

This includes CRM, Accounts Payable and PoS. Also Project Accounting, Social Networking and an Issue Tracker, think I was looking at Project Accounting and the limited opportunity for innovation in accounting systems.

originally from the U.of Navarre. I have always found this hard to get my arms around, but then I am not an accountant. This is now on SourceForge which is good. If you poke around the reviews, to quote Captain Rum, “Opinion be divided”.

Here, though is a planet of Open Bravo blogs.

Snipsnap and the ‘html’ macro

One of the add on features contributed  by the community to the snipsnap project was an {html} macro.

This is here… at the snipsnap site, and I have enclosed it as part of my mirror site. Obvioulsy best use snipsnap.org while it remains on-line.

This escaped the enclosed code and rendered it in raw HTML. The snipsnap macro defintion syntax was to use the embraced keyword twice, so {html}<strong>Important!</strong>{html}

I used it to include the google map widgets, the delicious tag cloud widget, getsatisfaction, which worked, see below, and Disqus which broke the backup. One of its most important extensibility features, as you can see above was it allowed Javascript.

The Disqus bug was because the backup format was XML, and snipsnap struggled with recursive CDATA elements. The Disqus scripts held CDATA elements, which snipsnap tried to enclose in the same tags. This failed on important.

I have also used this technique to get sharethis, addthis, flickr badges and pictures, wordle and big huge labs web badge to work. ( I think that’s it). It was useful if you wanted, needed, additional parameters beyond those available in the snipsnap img macro.

Get Satisfaction

An example of the get satisfaction code is as follows;

 
<div id="gsfn_list_widget">
  <a href="http://getsatisfaction.com/plazes">Active customer service discussions in Plazes tagged "davelevy"</a>
  <div id="gsfn_content">Loading...</div>
  <div><a href="http://getsatisfaction.com/">
<img alt="Favicon" src="http://www.getsatisfaction.com/favicon.gif" style="vertical-align: middle;" BORDER=0 /></a> <a href="http://getsatisfaction.com/">Get Satisfaction support network</a></div>
</div>
<script src="http://getsatisfaction.com/plazes/widgets/javascripts/c178c17/widgets.js" type="text/javascript"></script><script src="http://getsatisfaction.com/plazes/topics.widget?callback=gsfnTopicsCallback&amp;length=0&amp;limit=5&amp;sort=recently_active&amp;tag=davelevy" type="text/javascript"></script>

It is used on the Plazes article as well.