Oct 21, 2011

Visualizing the schooX dataset with Gephi, a colorful poster.

So,schooX and Gephi.
First of all, a few words about these two platforms.



schooX - the academy for Self Learners - is a newfound startup which helps you collect and share content (wikis,videos,slides, etc), and optionally organize them into courses always accessible within schooX eliminating the need to visit again different media hubs tracking down knowledge you already found in the past.

Gephi is a visualization tool based on graph theory and the gexf file format. Feed it with data, select a layout algorithm, tweak colors and settings and voila (sic): a beautiful and informative image.


The Model
In schooX each user has a set of collected articles and each article is described by user assigned tags. The superset of all articles' tags forms the user's tag cloud. In graph theory terms, a user is a node and his tags are the node's edges. We will put Gephi to layout all user nodes and their respective edges, but first we have to get the dataset.


The Process
This first part of the article will detail steps 1,2,3 of the process.
  1. Extract the model in csv format (user-tags)
  2. Parse schooX user profiles with cURL
  3. Store html profile pages locally.
  4. Grep or XPath user tag clouds from the htmls.
  5. Create the csv file and import into Gephi.
  6. Perform cluster analysis and graph drawing layout algorithms.
  7. Export and Gimp the final image with different fx. 

Final Result
Before detailing the process, some images of the final result.
We can identify user concentration around some topics such as social media, software development, medical sciences etc. Interesting...the startup's dataset has already started getting in shape forming communities around some hot topics. On to the images (available in flickr too):
schooX Network Graph
schooX Network Graph with Labels
Glass Clustering Effect
Network Painting - qualifies as an abstract poster?
Dark Neon Abstract


Parsing data, assessing the model
Since we haven't got direct access to any dataset or database we can only do the...unthinkable :) Create a schooX account, parse all users' public tag-clouds and create our model in csv format. Each row of the csv file will contain two columns and will be imported to Gephi:
  • tag name
  • unique user name
schooX' entry point is:
http://www.schoox.com/login/index.php
User profile pages containing the user's tag cloud follow this URL pattern:
http://www.schoox.com/user/7969/tag-cloud/
For example 7969 is my user id. After some trial and error we can find the last user id since an error comes up in place of a user profile page simply because there is no user with such an id.

The cURL (see-url) command
We will use curl (see-url) to login to schooX and store the login cookie, parse all user's profile pages and store them locally as html files. Later, we will extract from pages the tags for each user.
Login to schooX and store cookie:
 curl --cookie-jar cjar --data 'username=arapidhs@gmail.com' --data 'password=********' --output /dev/null http://www.schoox.com/login/index.php
Starting from id 1 to the user max id we consecutively execute curl ,(see-url), to parse all user profiles:
#!/bin/bash
for i in {1..200000}
do
curl --cookie cjar --output /home/arapidhs/tmp/i.html           http://www.schoox.com/user/i/tag-cloud/
done

Our tmp local directory now stores all user profiles and tag clouds in separate html pages. Each html page is named after the corresponding user's id.
Have a look:


In part 2 we will parse these pages with Tidy to generate a csv model file to import into Gephi.
The format of the csv will be something like with each row representing an edge of the network:

coefficient,Charalampos Arapidis
paok,Charalampos Arapidis
neural,Charalampos Arapidis
desktop,Charalampos Arapidis
minimal,Charalampos Arapidis
analog,Charalampos Arapidis
Fourier,Charalampos Arapidis
Java,Charalampos Arapidis
visualization,Charalampos Arapidis
software,Charalampos Arapidis
subversion,Charalampos Arapidis
social,Charalampos Arapidis
processing,Charalampos Arapidis


See you soon, and happy collecting!



Aug 29, 2011

Oracle Certified MySQL Associate uCertify prepkit review

Back from vacation, feeling refreshed and ready for new things:)

Here is a review of a preparation kit i received from uCertify  for the Oracle Certified MySQL Associate exam. I found it worthwhile to share my my experience with the kit for developers interested in training to get a technical certification.

The kit is by uCertify a company which specializes in the certification training domain providing
many kits for various technical certificates.
uCertify has developed a prepkit engine which runs al the available certification preparation kits.
Exploring the kit i found three types of tests, two of them were approximates of the final test while 
the final one was a little bit more difficult than the actual certification test.

The User Interface is elegant, responsive and i liked the extensive reporting after having taken some tests.
Yes, i love stats. They cover all the aspects of database usage, from terminology to transactions.
It is also great that you can clear your doubts about a question  or actually learn from the tests since the feedback is immediate and articles, extensive explanations etc are included within the kit.


performance report
sample question
The tests can be taken in two distinct modes. Learning and exam mode. 
Learning mode is the feautre i liked the most providing feedback and insight immediately after each question  thus helping to not repeat the same mistakes again and really understand what went wrong and why.

Do they actually help?

I think they do.
The kit identified my SQL weaknesses correctly: transactions and import/export procedures. Could i improve using the prepkit? To test this i created a custom test - you can do that - with questions concerning the above fields. I ran it in learning mode following up the explanations after each question. Took a break for a day and retook the simulated tests and yes! my stats were improved.

Most important is that the improvisation does not feel technical. I actually gained knowledge from the tests
and helped get the feeling of the final exams.

smaple ER question
articles and concepts included in the kit






Jul 29, 2011

Do not use Java 7 yet

The 7th release of Java today seems that introduced some nasty bugs caused by hotspot compiler optimizations miscompiling some loops. Code containing loops will propably be affected by this bug.


If you use Java 7 use this switch when starting the JVM:
-XX:-UseLoopPredicate

Uwe Schindler an Apache Lucene PMC Member tweeted a warning earlier today as the bug affected Apache Lucene and Solr project causing wrong compilation of some loops.

The warning mail and the full story from LucidImagination:

From: Uwe Schindler
Date: Thu, 28 Jul 2011 23:13:36 +0200
Subject: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler
optimizations, which miscompile some loops. This can affect code of several
Apache projects. Sometimes JVMs only crash, but in several cases, results
calculated can be incorrect, leading to bugs in applications (see Hotspot
bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are
affected by these bugs, namely all versions released until today. Solr users
with the default configuration will have Java crashing with SIGSEGV as soon
as they start to index documents, as one affected part is the well-known
Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
miscompiled, too, leading to index corruption (especially on Lucene trunk
with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release,
so Oracle had no time to fix those bugs, affecting also many more
applications. In response to our questions, they proposed to include the
fixes into service release u2 (eventually into service release u1, see [6]).
This means you cannot use Apache Lucene/Solr with Java 7 releases before
Update 2! If you do, please don't open bug reports, it is not the
committers' fault! At least disable loop optimizations using the
-XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the
unicode version shipped with Java 7 changed and tokenization behaves
differently (e.g. lowercasing). For more information, read
JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project,
Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
[4] https://issues.apache.org/jira/browse/LUCENE-3335
[5] https://issues.apache.org/jira/browse/LUCENE-3346
[6] http://s.apache.org/StQ

Better play safe and expect an update form Oracle i guess...

Jul 11, 2011

Google plus Invites

I have some Google Plus invites so contact me or add your email to the comments to invite you :)
The comments are moderated and will never be published to prevent exposure of your email. Thank you.

Jul 9, 2011

Snail mail from Google AdSense, how does it look?

So Google sends snail mail to verify address details. I wasn't expecting that but then again how else could verify ones' physical address? Hmm...

Some pics of the mail...behold:






Jul 7, 2011

Google plus interface a new trend for rounded corners?

Google with the launch of Google Plus has introduced  a series of user interface changes. Probably you will have noticed the big black bar on top of your Google account.

To me the new interface looks cleaner and...more edgy around the corners. Rounded buttons and insets have been replaced by smooth rectangles or semi-transparent drop out shadows.

Will these changes have a greater overral effect on how things might start to look in the future?
Will we forget about the all-rounded (heavy)corners and go for tiny 1px roundings or none at all?

What do you think?



For us Gmail users, there is a new theme available going along with the default Google Plus look, (i like how google tries to prevent alienation of its user base and introduces small changes at a step by step rate).


As Google puts it:
Gmail is getting a cleaner, more modern look over the newxt few months.
It looks like this:

Jun 29, 2011

Greece's Public Sector merges its data, how it was done

Kallikratis is the codename of the largest project ever conceived in the later years (2010-2011) of Greece's public sector digitalisation / computerisation effort and was founded by the Ministry of Interior and Decentralization.

The goal? To migrate different datasets spread among different applications and databases within the public sector's infrastructure.
That accounts to million rows of citizens' and public organizations' data / datasets distributed among different applications, structures, databases etc. having to be migrated in a homogenous, valid, normalized and commonly accepted data structure. In other words an ETL - Extract Transform Load process was required.

imho the talks and negotations to define the details of the final homogenous structure for each dataset was the hardest part, requiring cooperative thinking and strategizing among different tech companies and the ministry.

Datasets were categorized by domain: for example citizens' demographic data, public sector's and prefectures' economical/budgeting data, document management systems' data, and the list goes on.

The final process took place in the beggining of the year and lasted about a month, the merge happened in the first five days with corrections following until the end of January.

The project involved 2 vital factors:

1. had to be fast - ( execution, implementation,debugging, ability to change how things work on site wihtout the need to alter code / or recompile enabling tech support to operate)
2. and opensource


The plan was to extract and transform the data to a common XML file, validate it against the corresponding XSD and finally load it into the new environment. We ended up with a set of numerous transform processes and one generic load process that accepted as input the transformed xml files. 
Codename of the process was Datapair,  thumbs up for the...original name i found are welcome - sarcasm ensues :)

Various apis / platforms were investigated with some of them being:
For reasons that i would not like to present here and now, we chose the pentaho solution.
(briefly, CloverETL came at a cost for advanced features, while Talend's performance was found subpar 
at least at that time)

In the end everything worked better than expected and i am especially  happy about our tech support department beeing able to learn and finally operate the Pentaho Data Integration suite. That was a great relief enabling more people to take part in the process.


Someone could consider this,(and i mean the project as a whole from conceptualization to implementation), a default practice or a standardized ,(sic), solution but for Greece's Software industry was well...groundbreaking.

Expect more articles to come,  further analysing Pentaho's components as a minimal tribute to the platform that did the job for us.

Extra thank you for reaching the end of this post :)


Jun 20, 2011

JSF and Seam Security - custom login in 3 steps

Seam framework provides a standard IdentityManagement API to manage user management and authentication.The Credentials class includes the two basic properties needed to perform user authentication. The username, and the password, (sic).


But some times extra information is needed to authenticate a user. For example a company or an organization idenntification number, etc.

What we can do is simply extend the Credentials class including these properties.In the following scenario an extra company field is added to the login page.



1. Create a new class - CustomCredentials - extending Credentials, which includes the private Company property.
/*
 *@(#)CommCredentials.java  1.00  26/05/2011
 *
 */

package com.comm.security;

import com.comm.entities.Company;
import org.jboss.seam.ScopeType;
import org.jboss.seam.annotations.Install;
import org.jboss.seam.annotations.Name;
import org.jboss.seam.annotations.Scope;
import org.jboss.seam.annotations.intercept.BypassInterceptors;
import org.jboss.seam.security.Credentials;

/**
 *
 * @author Charis Arapidis - arapidhs@gmail.com
 * @version 1.00, 26/05/2011
 * @since v1.00
 */
@Name("org.jboss.seam.security.credentials")
@Scope(ScopeType.SESSION)
@Install(precedence = Install.APPLICATION)
@BypassInterceptors
public class CommCredentials extends Credentials {

    /** */
    private Company company;

    /** @return the company */
    public Company getCompany() {
        return company;
    }

    /** @param value the company to set */
    public void setCompany(final Company value) {
        this.company = value;
    }

}
Notice that CustomCredentials is a Seam Component since it extends Credentials and is installed at Application Scope.

2. In the login page add the Company field refering to it with the expression #{credentials.company}
<h:outputLabel for="company" value="#{messages.login_company}:"/>

<h:selectOneMenu id="company" value="#{credentials.company}">
    <s:selectItems value="#{CompaniesList.companies}" var="company"
        itemLabel="#{company.title}"
        noSelectionLabel="#{messages.general_select}"/>
    <f:converter converterId="CompanyConverter" />
</h:selectOneMenu>

<h:commandButton action="#{identity.login}"/>
3. Finally, in the Authenticator class inject the CustomCredentials comopent created earlier providing access to the user submitted Company field.
/*
 *@(#)Authenticator.java  1.00  11/05/2011
 *
 */
package com.comm.security;

import com.comm.entities.Company;
import com.comm.entities.User;
import com.comm.sessionbeans.SecuritySessionBeanLocal;
import org.jboss.seam.ScopeType;
import org.jboss.seam.annotations.In;
import org.jboss.seam.annotations.Name;
import org.jboss.seam.annotations.Out;
import org.jboss.seam.annotations.Scope;
import org.jboss.seam.international.StatusMessages;

/**
 * @author Charis Arapidis - arapidhs@gmail.com
 * @version 1.00, 11/05/2011
 * @since v1.00
 */
@Name("authenticator")
@Scope(ScopeType.EVENT)
public class Authenticator {

    @In
    private CustomCredentials credentials;
    /** */
    private User loginUser;
    /** */
    @In(create=true)
    private SecuritySessionBeanLocal service;

    /** @return true if user is authenticated */
    public boolean authenticate() {
        
        final String username = credentials.getUsername();
        final String password = credentials.getPassword();
        final Company company = credentials.getCompany();

        final boolean login = service.authenticate(...);

        if (login) {

        }

        return login;
    }

    @Out(value = "loginUser", required = false,
    scope = ScopeType.SESSION)
    public User getLoginUser() {
        return loginUser;
    }

    public void setLoginUser(final User value) {
        this.loginUser = value;
    }

}

May 27, 2011

Visualize this, software bugs as coloured dots

Some developers, many many issues.
How does it look?

This comes straight out from our RnD department and shows how developers are connected to the various issues as they are submitted in our bug tracking system.

The issues and the developers are colour coded:

  • The Teal circles are the developers and they are connected to their assigned issues.
  • Green dots represent resolved and finally closed issues
  • Red dots are issues which are only resolved but not confirmed - quality checks still pending.
  • Purple dots - the fewer - are open issues.
Of course this is just a subset of our system to test a custom plugin under development for Gephi - (gephi is an opensource visualization platform following graph theory ).

The subset contains  issues that are mostly resolved or closed - that's why the purple dots are sparse :)

More to come as time allows - and this does not mean never :).

Developers and their issues...

May 18, 2011

Resign Patterns

The article titled Resign Patterns by Michael Duell was recently posted in a Linkedin discussion by Sergey G. , (Systems Architect).

The discussion  was about:

What's your favorite design pattern? Why? 

and among the very useful and informative comments the Resign Patterns one hit me the most. It is a humorous take on design patterns and how they are applied under development pressure.

I reformatted the article in pdf format and is now available from Google Docs.
You can read the original text below.



Resign Patterns
Ailments of Unsuitable Project-Disoriented Software
by

Abstract
Anyone familiar with the book of patterns by the Gang of Four [1] knows that the patterns presented in the book represent elegant solutions that have evolved over time. Unfortunately, extracting these patterns from legacy  code is impossible, because nobody knew that they were supposed to be using these patterns when they wrote the legacy code. Hence, this work is a catalog of patterns for the masses. The patterns presented here represent abundant solutions that have endured over time. Enjoy reading the patterns, but please don't use them!



1. Cremational Patterns
Below is a list of five cremational patterns.

1.1 Abject Poverty
The Abject Poverty Pattern is evident in software that is so difficult to test and maintain that doing so results in massive budget overruns.

1.2 Blinder
The Blinder Pattern is an expedient solution to a problem without regard for future changes in requirements. It is unclear as to whether the Blinder is named for the blinders worn by the software designer during the coding phase, or the desire to gouge his eyes out during the maintenance phase.

1.3 Fallacy Method
The Fallacy method is evident in handling corner cases. The logic looks correct, but if anyone actually bothers to test it, or if a corner case occurs, the Fallacy of the logic will become known.

1.4 ProtoTry
The ProtoTry Pattern is a quick and dirty attempt to develop a working model of software. The original intent is to rewrite the ProtoTry, using lessons learned, but schedules never permit. The ProtoTry is also known as legacy code.

1.5 Simpleton
The Simpleton Pattern is an extremely complex pattern used for the most trivial of tasks. The Simpleton is an accurate indicator of the skill level of its creator.


2. Destructural Patterns
Below is a list of seven destructural patterns.

2.1 Adopter
The Adopter Pattern provides a home for orphaned functions. The result is a large family of functions that don't look anything alike, whose only relation to one another is through the Adopter.

2.2 Brig
The Brig Pattern is a container class for bad software. Also known as module.

2.3 Compromise
The Compromise Pattern is used to balance the forces of schedule vs. quality. The result is software of inferior quality that is still late.

2.4 Detonator
The Detonator is extremely common, but often undetected. A common example is the calculations based on a 2 digit year field. This bomb is out there, and waiting to explode!

2.5 Fromage
The Fromage Pattern is often full of holes. Fromage consists of cheesy little software tricks that make portability impossible. The older this pattern gets, the riper it smells.

2.6 Flypaper
The Flypaper Pattern is written by one designer and maintained by another. The designer maintaining the Flypaper Pattern finds herself stuck, and will likely perish before getting loose.

2.7 ePoxy
The ePoxy Pattern is evident in tightly coupled software modules. As coupling between modules increases, there appears to be an epoxy bond between them.


3. Misbehavioral Patterns
Below is a list of eleven misbehavioral patterns.

3.1 Chain of Possibilities
The Chain of Possibilities Pattern is evident in big, poorly documented modules. Nobody is sure of the full extent of its functionality, but the possibilities seem endless. Also known as Non-Deterministic.

3.2 Commando
The Commando Pattern is used to get in and out quick, and get the job done. This pattern can break any encapsulation to accomplish its mission. It takes no prisoners.

3.3 Intersperser
The Intersperser Pattern scatters pieces of functionality throughout a system, making a function impossible to test, modify, or understand.

3.4 Instigator
The Instigator Pattern is seemingly benign, but wreaks havoc on other parts of the software system.

3.5 Momentum
The Momentum Pattern grows exponentially, increasing size, memory requirements, complexity, and  processing time.

3.6 Medicator
The Medicator Pattern is a real time hog that makes the rest of the system appear to be medicated with strong sedatives.

3.7 Absolver
The Absolver Pattern is evident in problem ridden code developed by former employees. So many historical problems have been traced to this software that current employees can absolve their software of blame by claiming that the absolver is responsible for any problem reported. Also known as It's-not-in-my-code.

3.8 Stake
The Stake Pattern is evident in problem ridden software written by designers who have since chosen the management ladder. Although fraught with problems, the manager's stake in this software is too high to allow anyone to rewrite it, as it represents the pinnacle of the manager's technical achievement.

3.9 Eulogy
The Eulogy Pattern is eventually used on all projects employing the other 22 Resign Patterns. Also known as Post Mortem.

3.10 Tempest Method
The Tempest Method is used in the last few days before software delivery. The Tempest Method is characterized by lack of comments, and introduction of several Detonator Patterns.

3.11 Visitor From Hell
The Visitor From Hell Pattern is coincident with the absence of run time bounds checking on arrays. Inevitably, at least one control loop per system will have a Visitor From Hell Pattern that will overwrite critical data.

References
Gamma, E., Helm, R., Johnson, R., Vlissides, J., Design Patterns -
Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

Michael Duell is an Engineer at AG Communication Systems, where his
Resign Patterns have been rejected in favor of the Gang
of Four Design Patterns.
"Resign Patterns: Ailments of Unsuitable Project-Disoriented Software,"
The Software Practitioner, Vol. 7, No. 3, May-June 1997, p. 14.



May 10, 2011

Epic Q&A submission

Jira submission from a Quality and Assurance department :

JPEG screenshot inside Internet Explorer showing a screenshot of MSPaint editing a screenshot
of Firefox browser runnng the application.


Epic. 


Must go...deeper...






Source from reddit.

Netbeans 6.5 JDK Not Found

If you are trying to install an old Netbeans version under Windows 7 64bit ,(Netbeans 6.5), you may encounter the following error message:
Java JDK Not Found
This is a known issue and filed as a bug under Netbeans' tracker but there is a solution.


The workaround is to manually extract the exe and execute the bundle.jar installer.
From the commnand prompt execute the following two commands:
netbeans-6.5.1-windows.exe –extract
java -jar bundle.jar
The first one extracts the installer from the exe while the other executes the installer.



May 6, 2011

How software companies die

An essay by Orson Scott Card, found it lurking in my articles folder but i lack the source.
A very true and entertaining read. Hope you enjoy it.

The environment that nutures creative programmers kills management and marketing types - and vice versa. Programming is the Great Game. It consumes you, body and soul. When you're caught up in it, nothing else matters. When you emerge into daylight, you might well discover that you're a hundred pounds overweight, your underwear is older than the average first grader, and judging from the number of pizza boxes lying around, it must be spring already. But you don't care, because your program runs, and the code is fast and clever and tight. You won. You're aware that some people think you're a nerd. So what? They're not players. They've never jousted with Windows or gone hand to hand with DOS. To them C++ is a decent grade, almost a B - not a language. They barely exist. Like soldiers or artists, you don't care about the opinions of civilians. You're building something intricate and fine. They'll never understand it.

BEEKEEPING
Here's the secret that every successful software company is based on: You can domesticate programmers the way beekeepers tame bees. You can't exactly communicate with them, but you can get them to swarm in one place and when they're not looking, you can carry off the honey. You keep these bees from stinging by paying them money. More money than they know what to do with. But that's less than you might think. You see, all these programmers keep hearing their parents' voices in their heads saying "When are you going to join the real world?" All you have to pay them is enough money that they can answer (also in their heads) "Geez, Dad, I'm making more than you." On average, this is cheap. And you get them to stay in the hive by giving them other coders to swarm with. The only person whose praise matters is another programmer. Less-talented programmers will idolize them; evenly matched ones will challenge and goad one another; and if you want to get a good swarm, you make sure that you have at least one certified genius coder that they can all look up to, even if he glances at other people's code only long enough to sneer at it. He's a Player, thinks the junior programmer. He looked at my code. That is enough. If a software company provides such a hive, the coders will give up sleep, love, health, and clean laundry, while the company keeps the bulk of the money.

OUT OF CONTROL
Here's the problem that ends up killing company after company. All successful software companies had, as their dominant personality, a leader who nurtured programmers. But no company can keep such a leader forever. Either he cashes out, or he brings in management types who end up driving him out, or he changes and becomes a management type himself. One way or another, marketers get control. But...control of what? Instead of finding assembly lines of productive workers, they quickly discover that their product is produced by utterly unpredictable, uncooperative, disobedient, and worst of all, unattractive people who resist all attempts at management. Put them on a time clock, dress them in suits, and they become sullen and start sabotaging the product. Worst of all, you can sense that they are making fun of you with every word they say.

SMOKED OUT
The shock is greater for the coder, though. He suddenly finds that alien creatures control his life. Meetings, Schedules, Reports. And now someone demands that he PLAN all his programming and then stick to the plan, never improving, never tweaking, and never, never touching some other team's code. The lousy young programmer who once worshiped him is now his tyrannical boss, a position he got because he played golf with some sphincter in a suit. The hive has been ruined. The best coders leave. And the marketers, comfortable now because they're surrounded by power neckties and they have things under control, are baffled that each new iteration of their software loses market share as the code bloats and the bugs proliferate. Got to get some better packaging. Yeah, that's it.

May 5, 2011

Code Fitness - Visualize your java classes and identify complexity

Before refactoring a class, package or library  weak spots have to be identified. There are many code analysis tools providing insight, hints and alerts as to where someone should focus.

But before reading these metrics i'd prefer to have a quick look, an overview of the structure and the complexity underneath especially in a new project. Something visual would be ideal.

So jSizer comes to the rescue. It is a small swing-based app which reads a jar file and produces a graph depicting
packages' - classes' node hierarchy while effective coloring and shaping  helps to differentiate each node based on some vital metrics.

Some examples:






To interpret the graphs :

  • The Width of a class-node represents the number of fields.
  • The Height of a node represents the number of methods.
  • The Color represents LOC - lines of code.
So, quoting jSizer's author Robert Olofsson:
What you want to do is to look at the graph, find the big dark boxes and possibly refactor them.

Apr 24, 2011

Reasons for Slow Database Performance

Great article focusing on the reasons for slow database performance.

Reasons for Slow Database Performance

Notice the interesting comments about over normalization at the end of the article.

Apr 19, 2011

Continuous Integration in 40 screens

These screens describe our continuous integration setup using Atlassian's software tools and Hudson


About one year ago, i was asked to research various products and setup a  continuous integration test environment. The plan was to migrate part of our development to the new setup.
These screens which are also available from my public Picassa album were taken during the evaluation process of the Atlassian platform and finally the setup was adopted by our development team with some additions ofcourse.










 Fisheye - Percentile source code distribution between real and test code - the test code's path is parameterized.

 Fisheye - Percentile distribution of each commiter to the codebase.

 Fisheye - Diff presentation and history of a file.

 Fisheye - Source code filtering.

 Fisheye - LOC ,( lines of code), chart.

 Fisheye Crucible - Review status of a source file.

 Fisheye Crucible - The review's author receives a pending review warning.

 Fisheye Crucible - Pending activities and reviews are displayed on the dashboard.

Fisheye Crucible - Review and comment on source code at line level.

 Fisheye Crucible - Each reviewer can read the author's comments and respond to complete the review process.

 Fisheye Crucible - A reviewer responds to author's code comment.

 Fisheye - DIFF history.

 Fisheye - Full source code view. Changes are color-coded including the name of each commiter.

 Fisheye - Recent activity for favorite projects and files.

Fisheye - Overall user's repository activity  and statistics.

JIRA - dashboard.

JIRA - activities' stream, netbeans integration.

JIRA - project's issues summary

Fisheye JIRA integration. Inspecting issue's details from Fisheye's interface. Source code changes correspond to Jira issues.

JIRA Fisheye integration. Source code changes for an issue are accessible from JIRA.

Netbeans IDE JIRA integration.

Netbeans IDE JIRA integration - submitting an issue from Netbeans IDE.

Netbeans IDE JIRA integration - commiting code including the project's JIRA key in the comments


.
Netbeans IDE JIRA integration - creating a new issue.

JIRA - project summary including Bamboo build activity.

Bamboo - projects' build status.

Bamboo - executing a maven build.

Bamboo - display of all the available build plans. The user can actively execute a plan.

Bamboo JIRA Fisheye integration

JIRA Bamboo integration -  JIRA issues include Bamboo's build information.

Bamboo - a build's plan summary

Bamboo JIRA integration. The issues tab displays information about the builds they are related to.

Bamboo JIRA  integration - details of a build including all the JIRA issues that are related to it.

Bamboo - project's bulid history.

Hudson - Status of the project's latest build. RSS feeds and email notifications also available.

Hudson - The workspace of a new hudson project.

Netbeans IDE Hudson integration - integrating a hudson server instance with Netbeans.