Blog

Removing unused Kernel files via command line

regular kernel updates are great, but having 3 years of kernel updates on one hard disk is not. chances are you are never going to use any of the 10 or 20 that you installed last year. if everything continues to work fine, why would you?

you can obviously use a package manager, such as synaptic, to uninstall unwanted programmes. quite often you might find a few other programmes that you didn’t know you had still and want to get rid of. but it’s generally easier to just use the command line to remove things quickly.
use this command:

dpkg -l ‘linux-*’ | sed ‘/^ii/!d;/'”$(uname -r | sed “s/(.*)-([^0-9]+)/1/”)”‘/d;s/^[^ ]* [^ ]* ([^ ]*).*/1/;/[0-9]/!d’ | xargs sudo apt-get -y purge 

i use this command in ubuntu and i’m pretty sure it will work in derived versions, but results may vary!

Bulk encode MP3s with Lame and avconv

to be honest, when i am in the car or listening to music at home, i don’t really hear the difference between “lossy” mp3s and lossless formats such as flac. flac is just really huge compared to MP3 and i wouldn’t care if hard disk space was unlimited.
even if buying 2-3 nat storage devices, i think i’d still run our of space eventually.

so i do compress flac files after a while. but doing a whole album that is correctly labelled and tagged can be tricky. you don’t want to use some software that renames the files or changes the tagging…

linux is very supportive for doing things like that via the command line:

what you need is avconv and lame. avconv is a very fast video and audio converter. it can use libmp3lame to encode to mp3. because avconv can also convert and optimise video files, you can use it the same way to strip mp3s from video files, such as music videos or stuff you downloaded from youtube. but this post is about mp3s.
lame ain’t an mp3 encoder, but it simply is the best of them.

the command line argument to convert all flac files to mp3s in one folder is:

for file in *.flac; do n=$(basename “$file” .flac); avconv -i “$file” -codec:a libmp3lame -qscale:a 2 “$n”.mp3; done

if you want to convert from a different format, simply exchange the “.flag” to a “.wav”, “.mp4”, “.swf” or whatever the source file might be.

Exporting Sound with XDMCP

Some of you might be connecting to your remote linux box using XDMCP. While that would export the display,what would one do if one wanted to play a song on the remote box but hear it on the present system? well that can be easily done using the arts server on the remote machine setup the arts daemon by typing 

artsd -n -u -p 5001

once the server is started change to the X display of the remote machine and type in 

export ARTS_SERVER=ipofcurrentsystem:5001

and then

xmms &

Voila you will have a complete remote desktop including sound just like in windows xp.

Keyword Categorisation for PPC Search Engine Marketing

sometimes it’s strange that you never get to think about creating a solution for a manual task that you have to do, even though it would be totally easy to do.
such for me was building a tool that categorises keywords… i have probably categorised keywords manually for way too long, probably 10s of thousands of keywords. often i cut corners, often i would just wrap keywords into an ad group by the way that i generated them, running a quick sense check if the keywords are in the right place.

it turns out that the way that i was doing this was actually quite straight forward: i look at the keyword and decide, based on a few key phrases, what category i would map that keyword too. let’s take an example. the company i (hypothetically) work for is an airline with the name paddyjet:

  • i have brand terms, ie: paddyjet, paddy jet and so on and so forth.
    everything that has these words or ‘paddy’ in them will be brand terms, with the possible exception of keywords that have the word ‘review’ or ‘compare’ in them. maybe i also want to exclude the words ‘contact’ and ‘customer service’, as they too should be in another category called ‘paddyjet cs terms’
  • i have competitor terms that i may or may not want to bid on: british airways, ba, delta, – i would probably want to create a rule with these other brand terms and put them in another category, again with the exception of ‘review’ and ‘compare’ terms
  • there are the ‘review’ and ‘compare’ type terms
  • budget type terms will probably have the terms ‘cheap’, ‘discount’, ‘bargain’, ‘low cost’ and so on
  • cs terms should be our brand name and the words like ‘customer service’, ‘contact’, ‘complaints’…
in short, there are rules in your mind that make you decide quite instantly where to put terms. these rules are:
  • must include this word
  • must exclude these words
  • must include these words and these words
so i wrote a little python script to categorise keywords based on your rules. click on the link to go to my website and get the file and also samples! 
it needs the list of keywords in a text file (default keywords.txt), a list of noise words to remove (default: noiseWords.txt – it is optional to provider content in this file, the file itself has to be there) and the categorisation rules file (default keywordCategories.txt). 
the rules file needs to be marked up and structured in an ordered way:
  • more important rules need to go at the top
    ie: paddyjet tickets – if you rather want this in the ‘brand terms’, then put the brand rules first. if you rather want this to be categorised as ‘ticket terms’, then put this one first
  • one line per rule – no exceptions
  • start the line with the category name, followed by a colon (:)
  • words that mean the keyword does not fit in this category need to be in curly brackets ({…})
  • words that need to in combination with the term need to be in hard brackets ([…])
  • you can create combinations of words that need to be included and the search term.
    ie: you have vanity urls that you wish to treat different from brand terms (‘flypaddyjet’, ‘premierpaddyjet’ or paddyjetbusiness). simply put a tilde at the required position. ie:
    vanity urls: [premier~, ~premier] paddyjey, tickets, bookings, paddy
    will create the word combinations:
    premierpaddyjet, premiertickets, premierbookings, premierpaddy, paddyjetpremier, ticketspremier, …
finally, anything that can’t be categorised as ‘unknown’.
at the end a file will be generated with three columns: the original keyword, the noise words removed and the category based on the rules. the default name of the file would be date-time stamped and look like: YYYYMMDD_HHMMSS_categorisedKeywords.txt
i hope this tool could be useful for some people in digital marketing who currently struggle to categorise thousands of keywords manually. contact me if you have any questions, i’m happy to help.
it’s a very quick and dirty solution, so if you have problems, find bugs or would like more features, feel free to contact me. 

Using AJAX to read XML

arguably not the wisest of all implementations of an AJAX, i have prepared a page on my personal website (http://www.matenaers.com/chris/chris-matenaers.html) to put my CV online – at least a basic version.

i finished with the code, but still have to fill in the rest of the xml file to complete this task. this will change the content of a div on hovering over another div…

finally, this here is code only, apologies. ask questions if you have to.

here is a copy of the solution:

xml part:

<?xml version=”1.0″ encoding=”UTF-8″?>
<lvl1>
    <lvl2 id=”aaa”>
        <lv3a attrib=”test_attribute”>node_text</lv3a>
        <lvl3b><![CDATA[
            Some data not parsed
            ]]></lvl3b>
    </lvl2>
</lvl1>

and the html & javascript:

<script>
function getXML(url) {
var xmlhttp;
var txt = “”;
var x,xx,i;
if (window.XMLHttpRequest) {
xmlhttp=new XMLHttpRequest();
}
else {
 xmlhttp=new ActiveXObject(“Microsoft.XMLHTTP”);
}
xmlhttp.onreadystatechange=function() {
if (xmlhttp.readyState==4 && xmlhttp.status==200) {
x = xmlhttp.responseXML.documentElement.getElementsByTagName(“lvl2”);
for (i=0;i<x.length;i++) {
txt = txt + “<br />Attribute lvl2 : ” + x[i].getAttribute(‘id’);
xx = x[i].getElementsByTagName(‘lv3a’);
txt = txt + “<br />Node value lvl3a : ” +  xx[0].firstChild.nodeValue;
txt = txt + “<br />Attribute lvl3a : ” +  xx[0].getAttribute(‘attrib’);
xx = x[i].getElementsByTagName(‘lvl3b’);
txt = txt + “<br />Node value lvl3b : ” +  xx[0].firstChild.nodeValue;
}
document.getElementById(“change_me”).innerHTML=txt;
}
}
xmlhttp.open(“GET”,url,true);
xmlhttp.send();
}
</script>
<h1>Chris’ Sandbox</h1>
<div id=”mainpage”>
<div id=”content” class=”text”>
<h4><a href=”#” onmouseover=”getXML(‘http://www.matenaers.com/chris/code/sample_xml.xml’)” onclick=”return false;”>Hover here!</a></h4>
<hr/>
<div id=”change_me”> test </div>
</div>
</div>

PHP Image Resizer Class

class imageResizer {
var $newImage;

function getSquareImage($size, $image) {
//getting the path and resetting to the new path of the image
$oldPath = explode(‘/’, $image);
$t = sizeof($oldPath) – 1;
$i = 0;
while ($i < $t) {
$newPath = $newPath . $oldPath[$i] . ‘/’;
$i++;
}
$filename = str_replace(‘.gif’, ”, end($oldPath));
$filename = str_replace(‘.jpg’, ”, $filename);
$filename = str_replace(‘.png’, ”, $filename);
$filename = str_replace(‘.JPG’, ”, $filename);
$filename = str_replace(‘.jpeg’, ”, $filename);
$newPath = $newPath . ‘cache/’ . $filename . ‘_sq’. $size .’.jpg’;

//if that image does not exist already (we haven’t resized it before):
if(!is_file($newPath)) {
list($width, $height) = getimagesize($image);

//check if the image has the right propotions or do:
if ($width > $size || $height > $size) {
if ($width > $height) $ratio = $size / $width;
else $ratio = $size / $height;

$new_width = round($width * $ratio);
$new_height = round($height * $ratio);

$newImg = imagecreatetruecolor($new_width, $new_height);
$imageTmp = imagecreatefromjpeg($image);
imagecopyresampled($newImg, $imageTmp, 0, 0, 0, 0, $new_width, $new_height, $width, $height);
imagejpeg($newImg, $newPath);

$this -> newImage = $newPath;
}
//If the original image is the correct size:
else $this -> newImage = $image;
}
//set this orignal imaget the image returned:
else  $this -> newImage =  $newPath;
return $this -> newImage;
}

function getImageDimensions($sizeX, $sizeY, $image) {
//getting the path and resetting to the new path of the image
$oldPath = explode(‘/’, $image);
$t = sizeof($oldPath) – 1;
$i = 0;
while ($i < $t) {
$newPath = $newPath . $oldPath[$i] . ‘/’;
$i++;
}
$filename = str_replace(‘.gif’, ”, end($oldPath));
$filename = str_replace(‘.jpg’, ”, $filename);
$filename = str_replace(‘.png’, ”, $filename);
$filename = str_replace(‘.JPG’, ”, $filename);
$filename = str_replace(‘.jpeg’, ”, $filename);
$newPath = $newPath . ‘cache/’ . $filename . ‘_’. $sizeX .’x’. $sizeY .’.jpg’;

//if that image does not exist already (we haven’t resized it befoe):
if(!is_file($newPath)) {
list($width, $height) = getimagesize($image);

//check if the image has the right propotions or do:
if ($width > $sizeX || $height > $sizeY) {
if ($width > $height) $ratio = $sizeX / $width;
else $ratio = $sizeY / $height;

$new_width = round($width * $ratio);
$new_height = round($height * $ratio);

$newImg = imagecreatetruecolor($new_width, $new_height);
$imageTmp = imagecreatefromjpeg($image);
imagecopyresampled($newImg, $imageTmp, 0, 0, 0, 0, $new_width, $new_height, $width, $height);
imagejpeg($newImg, $newPath);

$this -> newImage = $newPath;
}
//If the original image is the correct size:
else $this -> newImage = $image;
}
//set this orignal imaget the image returned:
else  $this -> newImage =  $newPath;
return $this -> newImage;
}

}

PHP Image Downloader Class

a simple PHP class with just one method to download images from a remote server. checks for supported types (gif, jpg, png) and will throw an exception if the file is not supported.

usage:

  1. create instance if ImageDownloader
  2. call function downloadImageFrom with parameters: url, destination in local file-system, image name, image type
explanation of download image:
  1. check if the image type is supported
  2. get width and height of original image
  3. create a new image on local machine with width and height
  4. check the filetype and load correct image from imagecreatefrom~ function, parsing the image remote url
  5. resample the image
  6. save the image as the correct filetype using img~ function, parsing the image in memory and the new filename

code:

class ImageDownloader {
var $supported = array(“png”,”jpg”,”gif”);
function downloadImageFrom($url, $to, $fn, $img_type) {
if (in_array($img_type, $this -> supported)) {
list($width, $height) = getimagesize($url);
$newImg = imagecreatetruecolor($width, $height);
$imageTmp = ”;
if ($img_type == ‘png’) {
$imageTmp = imagecreatefrompng($url);
}
elseif ($img_type == ‘jpg’) {
$imageTmp = imagecreatefromjpeg($url);
}
elseif ($img_type == ‘gif’) {
$imageTmp = imagecreatefromgif($url);
}
if ($imageTmp != ”) {
imagecopyresampled($newImg, $imageTmp, 0, 0, 0, 0, $width, $height, $width, $height);
$newPath = $to . $fn . ‘.’ . $img_type;
$this -> imgLoc = $newPath;
if ($img_type == ‘jpg’) {
imagejpeg($newImg, $newPath);
}
elseif ($img_type == ‘gif’) {
imagegif($newImg, $newPath);
}
elseif ($img_type == ‘png’) {
imagepng($newImg, $newPath);
}
}
}
else {
throw new Exception(‘Not supported file-type’);
}
}

UK and the EU – exit or not

in response to an article on the bbc website:

in my opinion, the referendum is quite a bad idea for the country, but popular in terms of pr. It also distracts from the otherwise poor performance of the coalition… (although i think a labour government would have been much more disastrous) 
the referendum puts the decision of an eu membership into the hands of people who do not know all the details and consequences of such a move. It’s easy to blame the eu for some of the less welcome legislations they force on it’s member states and we should all fight against that, but an exit of the eu might have a lot of unintended consequences when the uk won’t receive it’s benefits any more.
given that this country has a trade deficit, it is, in the long term, advisable to be part of a bigger community of countries where some of the countries have an export surplus. the eu is definitely a long term investment, which may only start to pay dividends in 25+ years. 
we might not like some laws that are imposed by eu, but we can and must diligently influence the decision making on eu laws and how eu laws will effect member states laws when there is conflict. a general rule of ‘eu law supersedes country law’ and the idea common employment laws, just causes grievance. we should aim to align laws and resolve conflicts on a case-by-case basis. over years we will iron out the issues between the laws, in the meantime we should just make humanitarian decisions to resolve conflicts.
But an eu exit is dangerous and would be reversed within living memory, no doubt. i’m trying to find an analogy to this problem, but it proofs difficult…

Hate when people “sign” emails, but you can’t reply

i really think it’s the worst!

the conversation about whether or not to sign emails by an officer of your company has been going on for some time. in principal i don’t have a strong opinion about this, but i have certain expectations from such ‘signed’ emails:

  1. the person that signs this message has read the content and approved it’s content
  2. the email does not come from a ‘no-reply’ address (an address that only sends and all replies are discarded)
  3. when a reply is received, the person that signs the email is aware of the response
i have worked in a few online organisations and i’ve seen the good, the bad and the ugly. surprisingly it’s the smaller organisations that seem to get it right. your emails aren’t signed by anyone, but when you reply a bunch of people, including the customer service and marketing teams still see your reply and answer to it.
the bigger companies however get it completely wrong: singing the communication, sending from a ‘no-reply’ address and then don’t even make the customer aware of the fact that the email won’t be received by any human being!
two of these companies are british gas and british telecom. funny enough, you can find the bt-guy on twitter: warren buckley. he actually replies there and told me that he signs it so people can find him and he does hear what the customer comments are. i have an idea for you warren: put your email address (or a secondary mailbox, if you want) on the communication you crooks send out – i’m sure that people would be happy to reply!