A Retrospective Review

     After the events of my group project: hale-aloha-cli-backr, I gained enough knowledge to review another hale-aloha-cli project created by another team. Before I can get into this review, let's go back and see how we got here.

  In August, in a blog entitled, "Three Important Pixels", I've introduced three important rules under open source software engineering called the Three Prime Directives. I used these directives to evaluate a program called Pixelitor and so on to gain some experience using Java-based open source systems, and to gain an appreciation for the difficulties involved.

The Three Prime Directives are as followed:

• Prime Directive 1: The system successfully accomplishes a useful task.
• Prime Directive 2: An external user can successfully install and use the system.
• Prime Directive 3: An external developer can successfully understand and enhance the system.

  Moving up a month to September, I've introduced an "Awesomely Nifty Thing" called ANT. Developed by the Apache Software Foundation, Ant is a tool for automating software build processes. Along with ANT came a project in October involving Robocode which introduced myself to the Automated Quality Assurance tools: Checkstyle, PMD and FindBugs. Checkstyle is a development tool to help programmers write Java code that adheres to a coding standard. PMD scans Java source code and looks for potential problems. FindBugs uses static analysis to look for bugs in Java code. By implementing ANT build files to adhere to one of these automated quality assurance tools, it was a simple task to build, test and compile a project to distribute to other users.
  At the dawn of November sparks the introduction of WattDepot which is a system for collecting and storing data from electricity meters for smart grid research and experimentation. To better understand the aspects of WattDepot, I've undergone some software Katas to familiarize myself to the WattDepot API manipulating various energy data and displaying them in a given fashion.
Not found in my blog, but from personal experience came the introduction of Continuous Integration (CI). What may CI be? Well, tying all the aspects of what I've introduced to you is an important technique for high quality software development. CI implements continuous processes of applying quality control — small pieces of effort are applied frequently. Continuous integration improves the quality of software, and reduces the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.
  Finally this led to the hale-aloha-cli-backr project highlighted in the previous blog entry which also brought up the topic of Issue Driven Project Management (IDPM). IDPM breaks up a project worked on by several members and is divided into small tasks or "issues". Each person completes their issues and updates frequently to adhere to the CI standards.

  Okay, so you're still here if you're reading this line. Let's go back to the review…

  As stated before, I got the opportunity to review another hale-aloha-cli project created by another group of people. In this review, I will evaluating the software incorporating all the things learned throughout these past months and basing them off the first thing I introduced to you: The Three Prime Directives.

Review question 1:  Does the system accomplish a useful task?
     This project is just like the project I've worked on with my group, so it's basically our project envisioned by another group. I obtained a copy for this project and ran as many commands to view the feature set of this team's project. It seems that the group implemented all the feature commands:

•      current-power [tower | lounge]
• Returns the current power in kW for the associated tower or lounge.

• daily-energy [tower | lounge] [date]
• Returns the energy in kWh used by the tower or lounge for the specified date (yyyy-mm-dd).

• energy-since [tower | lounge] [date]
• Returns the energy used since the date (yyyy-mm-dd) to now.

• rank-towers [start] [end]
• Returns a list in sorted order from least to most energy consumed between the [start] and [end] date (yyyy-mm-dd)

• quit
• Terminates execution

• help
• Displays a menu of all possible commands.

With all of these commands, they provide useful tools for displaying various energy data gathered from the Hale Aloha towers at the University of Hawaii at Manoa.

Review question 2: Can an external user can successfully install and use the system?
     Now lets get down to testing this project. The home page of this project describes briefly what the project is about. Also on the home page is a featured link to the user guide. The user guide written provides simple instructions to install the program and what available commands the program offers (as described in the help command). The downloads section of the project website provides a distribution with the current version number of the project. Unzipping this file outputs the source and an executable jar file.
     After an extensive testing continuing from the first prime derivative, I've tested this program thoroughly and noted these errors and findings:

• Program crashes when entering any number of space characters without any characters entered.
• If a correct command is called without any parameters, it gives the same help information.
• The help command with parameters still gives help and doesn't catch error.
• The output of the help dialog needs more spacing between information to be easier to read.
• Some commands catches invalid date format, but needs to be more clear on the specific error.
• rank-towers does not have energy rating units.
• energy-since without a tower goes through but brings up a generic error message, needs to be more specific.
• Calling daily-energy without a tower or lounge but having the date parameter in a correct format gives an error specifying that there was an error accessing the blank tower.
• Calling rank-towers with a future date in the end parameter crashes the program.
• Calling rank-towers with the end date lower than the start date gives the correct error.
• Given a date far in the past gives a generic error that should be more specific.

Review question 3: Can an external developer successfully understand and enhance the system?
     For this part, we'll take the shoes of a developer. On the website for the project, it also provides a developer guide along the user guide. It provides a short sentence on how to build the system and lists all the types of automated quality assurance it runs. Though it doesn't mention if there are any manual testing done. Provided is a coding standard used for the project and a link to it as a plugin of the Eclipse IDE. The remainder of the guide describes the packages of the program and their functionality. There is no mention if this project is under continuous integration, no descriptions on how the project was implemented and contains no explanation on how to generate javadoc.
     After examining the website, I move on to checking out a copy of the project from the repository. I was able to find an ant build in the project that generates javadoc. The javadoc of the system seemed clear and simple, but maybe a bit short. However, the names of the components in the project are clear and provides a sense on what they do. Within the processor class are hard coded command class, so there is some information hiding since one could update the processor class to add in additional functionality.
     Upon building the system, there we're no errors. Generating test coverage from the jacoco ant build provided an overall 85% coverage for instructions and 65% coverage for branches. The manual test cases in the program tests one java source according to the name. If any additional enhancements to the project should occur, it would not disturb these test files since enhancements should come in the command package as specified by the developer guide on the project's website.
     Overall after reading through the source code, I would say that it was fairly easy to read, but the comments in the code was on the low side. Some additional comments would be helpful to explain some algorithms such as get the user's input date and parsing it.
     Back to the website to do more analysis. This project was developed under Issue Driven Project Management, so there were a see of issues assigned to each member who completed that assigned task. The issues page clearly shows on which member worked on what and would give a clear sense on who to question if a new developer works on this project. The system does show that one developer did more work than the fellow members but most of the issues this user committed were testing issues.
     Even though it was not specified in the developer guide, this project was ran under continuous integration. I was given the link to the CI build site and did some analysis on it. Failed project builds seemed to be corrected promptly, though there was a lengthly correction for build #4 - #7 which took 11 hours and 52 minutes to correct. The project did seem to be worked on consistently with build every day or two though not all builds were associated with an issue. Here's a breakdown of the builds with no issues:

• #1-14 no issues associated with upload.
• #16-19 no issues associated with upload.
• #24 no issues associated with upload. Started by user.
• #29 no issues associated with upload.
• #32-35 no issues associated with upload. Started by SCM change.
• #43 no issues associated with upload.
• #47 no issues associated with upload.
• #49 no issues associated with upload.
• #51 no issues associated with upload. Started by SCM change.
• #53 no issues associated with upload.
• #55 no issues associated with upload.
• #56 no issues associated with upload.
• #57 no build.
• #58 no issues associated with upload.

     Overall, the project seemed easy to read and with some time someone would understand it. Since much of the program to call the command package from the run class is hardcoded into the processor class, an external developer would have some ease on adding more elements to this system. All the developer needs to do is hard code the new command class into the processor, produce a working new command class and add tin the description into the help command. It will be a simple task, but since the developer will need to update 2 classes and provide a new one, it will also be a lengthy task.

Links:
Three Prime Directives (Blog Entry) : http://elmarcarillo.blogspot.com/2011/08/three-important-pixels.html
Robocode : http://robocode.sourceforge.net/
Robocode (Blog Entry) : http://elmarcarillo.blogspot.com/2011/09/kode-cut-row-bow-code.html
Apache ANT : http://ant.apache.org/
ANT (Blog Entry) : http://elmarcarillo.blogspot.com/2011/09/awesomely-nifty-thing.html
CheckStyle : http://checkstyle.sourceforge.net/
PMD : http://pmd.sourceforge.net/
FindBugs : http://findbugs.sourceforge.net/
WattDepot (Blog Entry) : http://elmarcarillo.blogspot.com/2011/11/shopping-for-electricity-at-wattdepot.html
hale-aloha-cli-backr (Blog Entry) : http://elmarcarillo.blogspot.com/2011/11/command-line-in-back-row.html

Command Line in the Back Row

     Most young programmers like myself start off programming solo, but we eventually have to learn how to program and work with others since this is a more realistic approach to a programming type of job. Over the months, I've introduced many things such as the Ant Build System and Configuration Management that enables working in a group with ease. With these tools, anyone in a group can make a change in the project and update it to the other members, but would you say it's organized? Over the past weeks, I participated in a group to develop a program using a style called Issue Driven Project Management (IDPM). Simply, as a group, we break down the project into tasks and we update the project according to these small tasks that we are assigned to do rather than just writing code till it works and putting it together. IDPM organizes what each member of a team should do and is efficient since it acts as the guideline to finishing up the project.
     So let's get back to the project my group and I created. The project is called hale-aloha-cli, but for our group, it's entitled "hale-aloha-cli-backr". (Note: backr is or group name. Derived from "Back Row" since our group was formed in the back row of a classroom.) This project extends what I've been working on the more recent blog entry of WattDepot. This project is a command line interface program to understand various aspects of energy use in the Hale Aloha residence halls at the University of Hawaii at Manoa. Other information on our project can be found here: http://code.google.com/p/hale-aloha-cli-backr/
     As a group, we developed this program using Google Project Hosting to host a repository of our system online and we also used Jenkins to automatically build the project and check for errors. We divided the project into tasks which we individually had to update the status on (or Issue as it's called). At first, our group didn't start with much issues. We just builded the base system and didn't create any issues on them. After a formal meeting on IDPM, out group started fresh and created issues for all tasks from revision 1 of the project. After more issues, we finally accepted what these issues did; it directed each group member what they needed to do rather than asking someone what needs to be done. If an issue is found, anyone can create an issue for it where anyone could take to address it.
     We completed the project fulfilling the guidelines and implementing all what needs to be in our system. Our project takes in 6 basic commands:

• help

             Shows the user a list of commands, the parameters for them and a description.

• current-power [tower | lounge]

             Returns the current power in kW for the associated tower or lounge.

• daily-energy [tower | lounge] [date]

             Returns the energy in kWh used by the tower or lounge for the specified date (yyyy-mm-dd).

• energy-since [tower | lounge] [date]

             Returns the energy used since the date (yyyy-mm-dd) to now.

• rank-towers [start] [end]

             Returns a list in sorted order from least to most energy consumed between the [start] and [end] date (yyyy-mm-dd)

• quit

             Terminates execution

     Our project contains a main program which we named HaleAloha. This class is the main interface which does most of the printing on to the console, takes user input and connects to the WattDepot server. The HaleAloha class calls the CommandProcessor class which decodes the user's input in which this processor class makes a call to the CommandManger. The CommandManager invokes one of the command class that matches with the command input from the user using the Java Reflection API to create instances of these command classes. This seems to be more robust that hard coding these classes onto the file. This would make adding future command classes simpler.
     While developing the program, we also needed to test the program at the same time. In sync, we developed some test files for the classes we created. Using jacoco to see how much our test covered our code, we tried to make test cases that would cover most of our code, if not 100%. A simple program that is well tested will probably perform with no issues than a complex program with little or no testing. As we completed our test files, the entree project was hit with a curve ball. The WattDepot server crashed and we were unable to connect to the server, rendering our program useless for a couple of days. When WattDepot revived, our program ran as before, but our test files failed. This was because all old sensor data from WattDepot was no longer available. We could only access data since the revival. Our team acknowledged the problem and fixed it to get it up and running again; clearing up the many failed builds in Jenkins.
     Overall, I think our group produced a great quality program, though there will always be room for improvements.

Link:
Project Home: http://code.google.com/p/hale-aloha-cli-backr/
Download: http://code.google.com/p/hale-aloha-cli-backr/downloads/detail?name=hale-aloha-cli-backr-v1.zip&can=2&q=

Shopping For Electricity at WattDepot

     With the previous introduction of energy comes the idea of utilizing the data centered around it. Why do such a thing you may ask? Well interestingly, energy data could be used to research different areas while gathering data on the amount of energy consumed or produce. With this information, there could be a way to use it such that in the future when a hypothetical smart grid is developed, energy could be dispersed based on the need of a certain area. Could this be possible? Most certainly, but maybe not today.

     Today on the other hand, I introduce to you WattDepot. This web service collects electricity data from meters and stores it in a data base. This data can then be retrieved and analyzed then possibly become the basis to developing applications for a future smart grid.

     Like previous applications introduced to you in my blog, before we decide to use an application, we must learn how to use it ourselves. See the pattern? Here we go… Code Katas. The following code katas were designed to help one become proficient in utilizing the WattDepot API.

Kata 1: SourceListing

     Implement a class called SourceListing, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and their descriptions, sorted in alphabetical order by source name.  Use the System.out.format method to provide a nicely formatted list. 

• Using the wattdepot-simpleapp distribution as a basis, I built this kata off of it. I took out some unnecessary code such as retrieving sensor data and just left it pruning out the sources and descriptions.

• Time to completion: 10 Minutes.

Kata 2: SourceLatency

     Implement a class called SourceLatency, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the number of seconds since data was received for that source, sorted in ascending order by this latency value.  If no data has every been received for that source, indicate that.  Use the System.out.format method to provide a nicely formatted list.

• This kata was a bit tricky. I could calculate the latencies and print them along with the list, but this kata states that the output must be sorted by the latency values. My first though was to create an array and then sort it, but it might take too much time and wouldn't be efficient. Ultimately, the best solution was to create a class that implements comparable. I called this call WattDepotUtility and created it such that I could benefit from using it for the next katas, hopefully. I also note there is a delay in getting the latencies before showing the list. A user might think the program might be stalled, but it's actually doing the calculations.

• Time to completion: 2 hours (Estimate. Most of time was spent on researching how to create my WattDepotUtility class).

Kata 3: SourceHierarchy

     Implement a class called SourceHierarchy, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a hierarchical list of all sources defined on that server.  The hierarchy represents the source and subsource relationship between sources.

• Building off from the previous katas, I discovered that there is a method in the API to retrieve sub sources of a source. The problem is that this method returned the path of the sub source instead of only the name. What I did to solve this problem was to split the string at the last index of the '/' character to get the name. I also created a method that would generate a tab character (2 spaces) when the hierarchy needs to be branched out.

• Time to completion: 45 minutes.

Kata 4: EnergyYesterday

     Implement a class called EnergyYesterday, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the amount of energy in watt-hours consumed by that source during the previous day, sorted in ascending order by watt-hours of consumption.  If no energy has every been consumed by that source, indicate zero.

• Most of my time spent building this kata was learning how to retrieve the energy data and getting yesterday date. I know I need to use the getEnergyConsumed method, but I always kept getting errors. What I discovered with some testing was that if the XMLGregorianCalendar being passed it had a time with seconds and milliseconds that were not zero, it would produce a resource not found error, so these time always had to be set to zero. First problem solved. For getting yesterdays date, I went through many methods such as Calendar and SimpleDateFormat. After browsing through the API, I could my answer in using the Tstamp class.

• Time to completion: 3 hours.

Kata 5: HighestRecordedPowerYesterday

Shopping For Electricity at WattDepotImplement a class called HighestRecordedPowerYesterday, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the highest recorded power associated with that source during the previous day, sorted in ascending order by watts.  Also indicate the time when that power value was observed. If no power data is associated with that source, indicate that.  Use the System.out.format method to provide a nicely formatted list.

• In this kata, it's very similar to the previous one. I just added in a method that would get all the power values from the previous date and get the highest value. My utility class only holds a string and data (double), but the output also wants the time. I could have added another field to my WattDepotUtility class, but instead, I concatenate the time with the name and the deconstruct it before printing the result. Same like the second kata, there is a delay in showing the list, but is much, much longer to the point most users would think it froze. So I added in print statements that would tell users when it is getting data from a certain source, in this test, there where 64 sources, so there were 64 messages printed to ensure the program is doing something in the background.

• Time to completion: 35 minutes.

Kata 6: MondayAverageEnergy

     Implement a class called MondayAverageEnergy, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the average energy consumed by that source during the previous two Mondays, sorted in ascending order by watt-hours.  Use the System.out.format method to provide a nicely formatted list.

• For the final kata, I think the trickiest part is determining when the previous two Mondays were. Playing with the Tstamp class and writing a switch statement product the correct result.

• Time to completion: 30 minutes.

     Overall with my experience using WattDepot I think it's very simple, though what you do with the data however, can get really complex. I completed all the katas listed above and most where not easy to complete as described. WattDepot is straight forward on getting energy data with many methods that probably sound what they will do such as getEnergyConsumed (getting the data for the amount of energy consumed). The katas can teach you how to sort data in a way that is easy to compare data from one source to another. In the end, this could be applied to any type of data and is not limited to energy data. Data manipulation may take some work to produce the correct output, but what really maters is how one will use this information.

Links:

WattDepot Home Page: http://code.google.com/p/wattdepot/

Introduction to WattDepot: https://sites.google.com/site/ics314fall2011/modules/introduction-to-wattdepot