Oct 17, 2012

Sonar book is out, free chapter inside

Having read quite a few development books i am so happy to have written one,
especially for a platform as important as Sonar.

SonarSource Sonar Code Quality Testing Essentials Book Charalampos Arapidis
...have been following and using
the platform since its birth...


It all started about one year ago, when i was contacted
by packt's editor aquisition team. A year later the book is out, and... a dream has become true :)

Head on to packt's site to download the first chapter for free from
the Sample Chapters section, and why not place your order :).

Finally, i have to say that the moment the book published,
the reviewing process and the feedback from developers
caused some of the warmest and fuzziest feelings
in my work life. This, was  my turn to give back to the community.






To view sonar in action visit Nemo, a public running instance of Sonar.

Oct 15, 2012

How to change default DNS server in Linux

Just a quick note ( to self ) of the cleanest way to set the DNS servers in Linux.

DNS entries stored in /etc/resolv.conf are overriden with every boot.

There are ways to prevent NetworkManager from overriding the file, altering dhcp hooks, or manually editing /etc/network/interfaces.

A simpler solution would be to add the following line to your /etc/dhcp/dhclient.conf

prepend domain-name-servers 8.8.8.8,8.8.4.4;

Add the DNS server IPS separated with a comma [,].

Oct 9, 2012

How to convert Mac line endings to Unix

Just a small note because a bunch of parsing scripts stopped working.

To convert MAC CR line endings to UNIX LF:

cat mac.txt | tr "\r" "\n" > unix.txt

We use the translate tr command to replace \r to \n.

Sep 25, 2012

How to generate UML Diagrams from Java code in Eclipse

UML diagrams compliment inline documentation ( javadoc ) and allow to better explore / understand a design. Moreover, you can print and bring them to table to discuss a design.

In this post, we will install and use the ObjectAid plugin for Eclipse to produce jUnit lib class diagrams. Then, we will be able to generate UML diagrams by simply dragging and dropping classes into the editor. We can further manipulate the diagram by selecting which references, operations or attributes to display. 
  • Open Eclipse and go to Help > Install New Software
  • Click on add to add a new repository
  • Enter name ObjectAid UML Explorer
  • Enter Location http://www.objectaid.net/update

Next, select the ObjectAid Class Diagram plugin - it is free - and click Next. The Sequence Diagram one requires a paid license.


Click Finish to confirm your choices and start the instalation process.
  • Click Ok on the security warning
  • Select Restart Now after the installation completes to restart Eclipse.

To create a new UML diagram we start the ObjectAid wizard with File > New > Other... and start typing in the textbox Class Diagram to filter the desired wizard. Click Next, and enter a directory and name for the diagram.


Drop java source or compiled class files into the visual UML editor to start populating the diagram. Right click on a generated class to bring up a context menu and adjust visibility, operations attributes, etc as you like.
Below, we see the Assert class from the jUnit library with all operations and fileds hidden.


From the context menu, we can add implementations and associations for a selected class. In the following screen, we add the interface Test implemented by the TestCase class.


This is how part of the jUnit UML class diagram look after adding some more classes.


To auto layout the diagram right click anywhere within the editor and select Layout Diagram. From the same menu, you can export the diagram to an image ( gif png jpeg ) by clicking the Save As Image... menu item




Have fun with the plugin, leave a comment if you like :)




Sep 22, 2012

How to find the old youtube player after upgrading to iOS6 in iPad (video).

The original hidef and fullscreen YouTube player it is still inside the (new) iPad, even after upgrading to iOS6. This is not about the prelaunch iPhone app, which is useful by the way, but not specifically designed for iPad.

The old player is available via the free Flipboard app. You can connect flipboard to your youtube account and browse favorites,subscripions,channels from the app. I hope that they won't release any updates removing the player from there too.  It is not necessary to have a Flipboard accountto use the app or watch youtube videos.

Here is a crap video i shot today showing the old player in action within Flipboard in iOS6.






Jul 24, 2012

How to batch identify similar images in Linux

After restoring two separate 1 Tera hard drives, i ended up with a new single directory loaded with duplicate photos, screens, icons etc. Many of the photos were different in size, saturation resolution but were still...duplicates; so i wanted an efficient way of identifying them and removing them from the archive. Notice that the goal of the bash script is to identify similar photos (e.g. similar size, content, distribution ) and not exact duplicates.

How it works:


  1. execute the script inside a folder
  2. the script recursively scans all directories for any image files and stores them for further processing
  3. compares each image with the rest of the set
  4. if the images differ in size, scale them down to a 300x300 square and compare
  5. if the images are of equal size, compare them and produce the Root Mean Square Error of their pixel values
  6. continue with next image in the set
Sample Output of the script
The script outputs results in 3 columns in ascending RMSE difference, similar images are first. First column is the RMSE pixel difference, second column the source image file and the third the target image file.
To get a measure of how different are the images, multiply by 100 the numeric RMSE value in parantheses to express the difference in 100%.

We can execute the script and redirect the results to a txt file:
imgcompare.sh > results.txt
RMSE                    Source image            Target image
1547.18 (0.0236085) ./1338951104.png ./1338951159.png
2128.89 (0.0324848) ./1338951279.png ./1338951159.png
2201.63 (0.0335947) ./1338951104.png ./1338951279.png
2280.55 (0.0347989) ./1338951221.png ./1338951159.png
2320.65 (0.0354108) ./1338951221.png ./1338951104.png
2346.09 (0.035799) ./1338951221.png ./1338951279.png
--- images below this line are entirely different, RMSE too high ---
16285.4 (0.248499) ./1338951221.png ./1338950993.png
16304 (0.248783) ./1338950993.png ./1338951279.png
16359.5 (0.24963) ./1338950993.png ./1338951159.png
16365.7 (0.249724) ./1338951104.png ./1338950993.png
Next, we will go through the script step by step. Of course you can modify it as you like, according to your needs. You can find the complete script at the ed of the post.
The script depends on the excellent imagemagick multi purpose tool suite. On ubuntu / debian, install it via the following command:
sudo apt-get install imagemagick

Scan and filter image files

If the script is invoked with a directory argument then scanning starts from that directory. Else, it scans for image files from the current dir. The following code uses the identify command in quiet mode to check if a file is of a known image type. All images will be stored in the images array.
######################################################
# FILTER IMAGE FILES
# get all files in the directory
# and filter out files that are not images
# store images in the $images array
# If user did not supply a  directory
# start searching from the working directory pwd
if [ -z "$1" ]; then
        directory=$pwd
else
        directory=$1
fi
files=(`find $directory -type f`)
length=${#files[@]}
count=0
for (( i=0 ; $i < $length; i=$i+1 )); do
        file=${files[$i]}
        identify -quiet -ping $file >/dev/null 2>/dev/null;
        if [ $? -eq 0 ]; then
                images[$count]=$file
                let count++
        fi
done
######################################################

Perform image comparison

Imagemagick cannot compare images that differ in size. After some research, i found out that while there were some internal builds supporting different size comparison, due to lack of performance and accuracy these changes never were officially released. ( If you are an imagemagick developer / contributor please let me know if this has changed )

So we need a way to verify that the two images are of equal size. To achieve this, we use identify to extract the dimensions in a %h%w format and compare the results:
#find the dimensions of the images in format height,width and compare the result
srcDim=`identify -format "%h,%w" $source`
trgDim=`identify -format "%h,%w" $target`

if [ $srcDim == $trgDim ]; then
...perform comparison...
fi
If sizes are equal we perform the comparison using the compare command, and append the output to the result variable. The result variable will hold the output of our script. We will just keep appending to this variable the results of each comparison. Notice that we use \n to break a new line per comparison and \t for column spacing.

Compare command example
compare -metric RMSE $sourceImage $targetImage null
( we do not want to create a difference file so redirect to null)

Compare all images in a nested loop
and append the output for
matted in 3 columns to the result variable.
######################################################
# COMPARE IMAGES
# compare all pair of images in the directory
# and store the results of each hit / comparison
# in the results variable in the following format:
# RMSE|source|target
length=${#images[@]}
count=0
for (( i=0 ; $i < $length-1; i=$i+1 )); do
        # set source to next image
        source=${images[$i]}
        start=$i+1

        # in a nested loop compare the source image starting from its
        # index to the length of tha array
        for (( j=$start ; $j < $length; j=$j+1 )); do
                # get target image
                target=${images[$j]}

                # find the dimensions of the image in format height,width
                srcDim=`identify -format "%h,%w" $source`
                trgDim=`identify -format "%h,%w" $target`

                # compare the dimensions of the images
                # if are of equal size proceed with the comparison
                if [ $srcDim == $trgDim ]; then
                        out=`compare -metric RMSE $source $target null: 2>&1;`
                        if [ $? -eq 0 ]; then
                                result=$result"\n"$out"\t"$source"\t"$target
                        fi
                fi
        done
done
######################################################

Finally, we short the results by the first column, ( RMSE value ) , and output to the console:

echo -e $result | sort -n

The complete script

You can save the following script e.g. imgcompare.sh and modify it as you like:
# imgcompare.sh
# imgcompare.sh
#
# Simple bash script to identify similar images
# in a directory. The script uses the great imagemagick
# tool suite to identify image formats, rescale images to same
# sizes before comparing and finally performs comparison
# and calculates an RMSE pixel error value for each image pair.
#
# Charalamapos Arapidis
# arapidhs@gmail.com
# 7/12/2012
#
#!/bin/bash


######################################################
# USAGE FUNCTION
# Usage imgcompare /path
# display usage function
function usage () {
   cat <<EOF

Usage:
$scriptname [directory]
example:
imgcompare.sh ~/images

EOF
   exit 0
}
######################################################


######################################################
# VARIABLES DECLARATION
scriptname=$0
directory= # the directory that contains image files to compare
threshold=10 # option -t, RMSE values below this percentage threshold are hits
source=  # source file
target=  # target file
images=  # array of image files
result=  # store the results in a three column format RMSE|source|target
OIFS=$IFS
######################################################


######################################################
# FILTER IMAGE FILES
# get all files in the directory
# and filter out files that are not images
# store images in the $images array
# If user did not supply a  directory
# start searching from the working directory pwd
if [ -z "$1" ]; then
 directory=$pwd
else
 directory=$1
fi
files=(`find $directory -type f`)
length=${#files[@]}
count=0
for (( i=0 ; $i < $length; i=$i+1 )); do
 file=${files[$i]}
  identify -quiet -ping $file >/dev/null 2>/dev/null;
 if [ $? -eq 0 ]; then
  images[$count]=$file
  let count++
 fi
done
######################################################


######################################################
# COMPARE IMAGES
# compare all pair of images in the directory
# and store the results of each hit / comparison
# in the results variable in the following format:
# RMSE|source|target
length=${#images[@]}
count=0
for (( i=0 ; $i < $length-1; i=$i+1 )); do
 # set source to next image
 source=${images[$i]}
 start=$i+1

 # in a nested loop compare the source image starting from its
 # index to the length of tha array
 for (( j=$start ; $j < $length; j=$j+1 )); do
  # get target image
  target=${images[$j]}

  # find the dimensions of the image in format height,width
  srcDim=`identify -format "%h,%w" $source`
  trgDim=`identify -format "%h,%w" $target`

  # compare the dimensions of the images
  # if are of equal size proceed with the comparison
  if [ $srcDim == $trgDim ]; then
   out=`compare -metric RMSE $source $target null: 2>&1;`
   if [ $? -eq 0 ]; then
    result=$result"\n"$out"\t"$source"\t"$target
   fi
  fi
 done
done
######################################################

echo -e $result | sort -n

Improving the script to compare images of different sizes

If the images differ in size, then we will have to scale them down to equal size before invoking the compare command. We can modify the script as to scale the images down to a fixed 300x300.
We will use the convert command to perform the size conversion, and store the converted images to temporary files.

######################################################
# COMPARE IMAGES
# compare all pair of images in the directory
# and store the results of each hit / comparison
# in the results variable in the following format:
# RMSE|source|target
length=${#images[@]}
count=0
for (( i=0 ; $i < $length-1; i=$i+1 )); do
 # set source to next image
 source=${images[$i]}
 start=$i+1

 # in a nested loop compare the source image starting from its
 # index to the length of tha array
 for (( j=$start ; $j < $length; j=$j+1 )); do
  # get target image
  target=${images[$j]}

  # find the dimensions of the image in format height,width
  srcDim=`identify -format "%h,%w" $source`
  trgDim=`identify -format "%h,%w" $target`

  # compare the dimensions of the images
  # if are of equal size proceed with the comparison
  if [ $srcDim == $trgDim ]; then
   out=`compare -metric RMSE $source $target null: 2>&1;`
   if [ $? -eq 0 ]; then
    result=$result"\n"$out"\t"$source"\t"$target
   fi
  # if the images differ in size scale them down 
  # to fixed size 300x300, we store the scaled down
  # images to temporary files
  else
   `convert $source -resize 300x300! $source".tmp"`
   `convert $target -resize 300x300! $target".tmp"`
   imgSrc=$source".tmp"
   imgTrg=$target".tmp"
   # perform comparison between the scaled images
   out=`compare -metric RMSE $imgSrc $imgTrg null: 2>&1;`
   if [ $? -eq 0 ]; then
    result=$result"\n"$out"\t"$source"\t"$target
   fi
   # delete the temporary image files
   rm -f $imgSrc
   rm -f $imgTrg
  fi
 done
done
######################################################

I hope you find the script useful and serve its purpose as a starting point of your own modifications and improvements. 
Let me know!

Jul 23, 2012

Back in action

Finally, after full 6 months i have been able to find some time and write again.
A lot of things have happened in the past 6 months including

Things have settled down now, the book is complete, and my life started adjusting back to normal :)
So i will be able to dedicate some time each week to post software / technical findings here at blogger.