IDE's: is there a tool for every occasion?

IDE, Integrated Development Environment, a programmer's best friend.  The best ones run unit tests, pre-compile code, do code completion, and run the code in a demo environment.

The value of a second pair of eyes

A long time a go in an office far far away I had to sit for four hours a day watching someone else write code.  It was slow, it was boring, and it drove me nuts.  It was pair programming and in retrospect I've noticed that it did make both of us better programmers and the code itself better.  I often get little reminders of one of the important reasons for pair programming.

A second set of eyes

This weeks example, I received an IRC message asking if I could help out with a CSS issue.  I'm not a "CSS guy" I'm more a "CSS thief" but I was game.  So, we looked over his Django template and a solution eventually presented itself.  I mocked it up, uploaded a patch and he merged it in.  Great!  Now he states if he could only figure out one other little issue.  

I couldn't help myself, I was in programming mode.  I asked "what was wrong?"  A ManyToMany object wasn't printing out on his page and had him stumped.  It stumped me to, I barely know Django and after going through all the documentation everything looked correct.  I threw in a "{{ forloop.counter }}" and nothing showed up.  Found sites that were doing exactly what he was trying to do and they matched.  Finally I saw he had one little element "related_name" in the foriegnkey attribute.  I had read about it, and it wasn't necessary so I thought "why not try removing it and seeing what happens."  His code worked.  Code he and then I had poured over for hours that was technically correct and one little "un-necessary" element was the cause.

Programming: Putting all of the Pieces together (Part 4)

While teaching the developers the proper process of coding and testing and committing will make them better developers, I need something to help me monitor the process.  So far all I have is labor intensive reviewing individual commits and running the tests by hand.  Pulling back in things learned from cornell while working on VIVO I can automate my tests with Jenkins.

Jenkins is amazing, I've spent part of the previous week putting together a POC installation on a Virtual Machine and the tests that I can run.  There are plugins for Selenium and Grails and code review like codenarq and code coverage.  With this we can automate tests to be performed before code is merged with develop.  If our developers are following the appropriate steps, their code will pass with flying colors.  If they are not following the appropriate steps, I save time and effort by having a robot (jenkins) remind them to test their code and that the code they are trying to submit fails.

How do I put this into our typical development process?  We're using in addition to Git, Git-Flow.  Basically it formalizes our branching structure so that between projects we are all following the same methods for defining the production branch (master) the development branch (develop) the working branches (/feature) and branches for hot fixes, version tags, etc.  Inserting jenkins into my development process isn't as easy as targeting a branch.  If I select master, I'm not testing often enough as its only receives release candidates from develop.  I could select develop, however, I want to insert it before the code comes into the develop.  I want to tigger it when my developer in his branch calls

git flow feature finish featureName

Also I don't want to only rely upon automated review.  What if my developer didn't add the appropriate tests for his or her new feature.  What if the poor code is syntactically perfect, but lacks good design decisions.  The code review tools will catch a lot of these errors, but not all of them.  Re-enter the original concept of code reviews.  When your peers see your code, you code cleaner, smarter, more efficiently.  Enter Gerrit, a code review system that automates the process of reviewing code.  It sets up review tickets, and can be integrated with jenkins to allow jenkins to weight in a pass/fail based on the scripted tests.  Reviewers can be setup for each developer and I can get emails when there is code for me to review.  Upon successful review the code can be automatically merged back into our main development trunk.

So this is the task this week; put it all together.  I have the proper pieces falling into place, now I need to make them match up.  How do I interrupt the git-flow process to insert Gerrit?  How do I make Gerrit preform the rest of the merge with develop if the reviewers and Jenkins approve?  

RIP Anne Mccaffrey

Before my mother began reading romance novels (yuck), she read fantasy books.  More specifically, she read Anne Mccaffrey.  My mother pulled from her collection my first fantasy book, Dragonsong.  It was a quick read, but it took me to a far away world.  The threat of thread, ancient lifestyle, dragons it all took hold.  I quickly moved from DragonSong to DragonSinger and DragonDummer finishing up the Harper Hall Trilogy.  From there the other Pern books which took on a slight scifi bent, then the "Ship Who Sang" and many other series by Mccaffrey.  My cousin suggested one of my other favorite authors, David Eddings, because I liked Anne Mccaffrey.

Anne Mccaffrey died this past week from a stroke.  Anne, you brought music into a genre of of swords, magic and mythical beasts.  Your books took me out of childrens literature and opened me to hundreds of authors and worlds.

Thank you

NPR Article on Anne Mccaffrey

Programming: Putting all of the Pieces together (Part 3)

Testing the individual units of the code is great for making sure each unit works.  Integration testing is fantastic for making sure the code fits together as intended.  What about testing the interface from the users prospective?

It all starts with plan, when I was a developer at the library we spent over a year combining 4 disparate ticket tracking and inventory systems into a single system that could link library hardware to the request for its purchase, any trouble tickets, etc.  As we were finishing the project our boss asked for a test document.  When complete it was a hundred and fourteen pages complete with step by step tests to be performed from the user interface.  Running through every page would take us about two to three hours.

A few years older and a few years smarter and I still make my team write those test plans.  However, today I'm making them create parallel selenium tests for every item in the document.  The current project is around 50 pages and the tests take 5 minutes, on slow.  The best part about this, selenium doesn't get bored and skip tests.  I don't know how many times, my own boredom caused me to miss an error in a function that I hadn't anticipated would be affected by my change.

So at this point I have tons of tests being generated.  I'm working on changing the workflow of my team so that they analyze the problem, start a new branch, build tests, code and comment, run tests, then push to our main branch.  They are starting to get better, but how do I take this from "you should do this" to "this happens every time". . .

Programming: Putting all of the Pieces together (Part 2)

Lets take a look at what I have to start with:

• Bug and Feature Tracking using Fusion Forge
• Git Repository
• Git Flow
• Grails Framework (Built on Groovy, Built on Spring, Built on Java)

What did I learn from VIVO:

• Style enforcement through eclipse
• Hudson/Jenkins for Continuous Integration
• Seleninum for Acceptance Testing

The tools are there, now its time to start putting them together.  Our old projects, our non-VIVO projects, have never used unit tests.  Its a change in style that even our more senior engineers are just now starting to do.  What is a unit test?  According to Wikipedia it is "a method of testing an individual unit of a software project."  In our case, it is a test of each class in our Grails project.  Its interesting teaching developers to test.  If a model (or domain) accepts null values, do you have to test that it will save and validate when given a null value?  Of course you do, but until that's a part of your programming knowledge base it might not occur to you.  If you test that a model properly fails validation when given a null value for a field that is null, do you have to test for the error that is returned?  Yes, just testing for the validation doesn't allow you to confirm that a the view will be given the proper error.

Lesson 1, when changing the culture of development at your group you may encounter slackers.

Lesson 2, be specific when you say "fix the tests", explain that means fixing either the code or the test depending on the reason the test is failing.

So, we are learning as a group to test more.  Unit tests lead to integration tests, which thankfully in Grails are built similar to unit tests.  We've stumbled, found tests that are basically integration tests in unit tests and vice versa.  Found items that should have been tested for and failed later.  Next step, acceptance tests!

Programming: Putting all of the Pieces together (Part 1)

It struck me the other day as I attempted to navigate my project's git log to review code that my team had pushed into the repository, something was missing.  It's taken a long time to pull myself out of the student programmer headspace and towards a full fledged developer whose code is sacred and must retain its history.

A Little History
I didn't get started with computers until around 2000.  Of course we had one at home, but my father is a programmer and for him at the end of a long day working for a bank he didn't want or need a computer at home.  Until 1994 we had a little black and white dos box.  That was replaced by a 386 with windows 3.1 until 1999 when I would get my own computer for college.

Fast forward through 5 years of school 2 spent as pre-med and 3 as computer science and enter my first programming job at the UF Computing Helpdesk.  It was a part time job with-in a part time job.  When items were finished they went to production and that was that.

From there I went to work at Smathers Libraries a Microsoft shop.  We used Visual Studio and Visual Source Safe at first.  For those who are rabid against MS, set aside your anger. Visual Studio is quite nice even though it is proprietary.  Visual Source Safe on the other hand, absolutely abysmal.  It would claim files were locked by one user only to find that person didn't have the file locked and could not unlock it.  It would over write newer files with older files when a person saved.  In summary, it stunk.

So we switched to the next best thing, in our uneducated opinions, versioned out file folders.  Basically a by hand version control system.  We were pair programming a rather large project so every morning the code would get copied to the file share and appended with the date and prefix AM.  In the afternoon the same would occur with the prefix PM.  It worked fairly well, the file share was backed up on tape and lived on a SAN that had a redundant system across campus.  Sure we had to navigate folders to find old versions of code, but we hardly ever looked back.  Who would need to do that?

Back to Today
Since VIVO, I've been immersed in SVN, Mercurial, CVS, Bazaar and Git.  I've learned the ins and outs of my old pal from undergrad, Eclipse.  I've used various bug and feature tracking systems; SourceForge, Jira, Launchpad, and FusionForge.  Its all coming together, the lifecycle that is software development.  The code is simply the result of all the pieces falling into place and without the right pieces, the code usually suffers.

Back to my current delima.  I have very junior programmers.  They come from a mixture of environments and educational levels, and for all of them I'm deeply concerned with what they took away from those experiences.  For example, perhaps its the sum of all of my experiences or perhaps its the first school project that nearly missed a deadline due to a hard coded value but I strive not to do these things.  I found, when helping a masters student fix a table filtering issue, hard coded drop down lists for items that are stored in the database.  The breakage was due to spelling differences between the database and the dropdownlist, obviously.  Their code is scary and the worst part I am putting my and the departments reputation behind it.

So how do I help them learn and avoid accepting bad code.  My brain slowly starts up, the hamster rising from his nap, and I think "Review their code".  So I open the code and before I begin to browse the first spark hits, use the gitg explorer so you can review what they are changing.  "Duh! How can I help them improve if I don't know who committed the error."  Now my hamster is not yet up to speed and when we finally got up to speed I processed that something was missing.  Why am I doing this code review by hand?  I have standards documents and with VIVO I've used code coverage reports and unit tests and acceptance tests.  How do I put this all together?