How a good day turns bad

I’m in Seattle for a 4-day training course on AWS at Amazon and the weather here is a sunny 75 degrees. Today was the first day of training, I learned a lot of good information, had a good lunch and walked back to hotel. For dinner, coworkers and I decide to walk to the nearest Thai restaurant. Everything was going great. I ordered a Tum Kha soup. After eating the worst Thai food I’ve ever had, we walk back to hotel while foolishly looking at South Lake Union and not at the path ahead.

I entered the hotel elevator and another guy in the elevator walks away from me to the back wall of the elevator. Then I look down at the floor and notice something there. I lift my shoe up and aaah, there is what can only be human feces all over it. It couldn’t be dog shit cause there was just too much of it. I limped over to my room and spent the next half hour cleaning up the shoe.

Now I sit here washing down this day with some good lemon tea and musing about the piece of shit that’s still lying on the floor of the hotel elevator. I should really call the front desk and tell them about it. I now also regret not paying tip yesterday at the dirty Indian restaurant where I ate dinner; at least their food tasted okay.

Here’s hoping the rest of the trip will be more uneventful - Manchester United vs. Sounders!!

WikiPublisher plugin for jenkins

Sometimes, folks other than the developers are interested in the finding out about the different builds and the components (upstream projects) that are in a build. These other folks might be managers or non-programming types that don’t have access to Jenkins or are not interested in looking up things on Jenkins. They prefer to just see a website with all builds listed. This plugin attempts to satisfy that need.

We have an internal wiki site for sharing useful information and that seems as good a place as any to share information about current builds of projects. So, I’ve written the wiki publisher plugin that publishes your build name along with the names of all of its upstream builds to a wiki page you’ve configured. The names are also linked back to their corresponding build on Jenkins so that you can go from wiki to Jenkins to download artifacts or lookup changes.

This plugin is modeled after the Confluence Publisher plugin that is available for Jenkins. There are two separate configuration pages. In the global configuration page, you need to setup your list of wiki sites that you would like to publish to. I expect most will only have one site setup here but its nice to know you can add more. If your wiki site uses authentication, then you need to setup your username, password, domain for the user that Jenkins will use to publish to the wiki site. Make sure that this user has edit permission on the wiki pages. Also, I’ve only tested it against a wiki that authenticates against LDAP server. So there’s the possibility that it won’t work on other setups. One additional note of caution. Your wiki site may be using https but only use a self signed certificate. In that’s the case, then do not check the “use https” link or it won’t work.

Now that you have your wiki site setup, its time to configure your project to publish to the wiki site. Open up your project configuration page and you will see a new section to publish build results to a wiki. Here, select the wiki site your setup previously and enter the name of the page where the results are to be published. The plugin won’t create the page itself and you must go to your wiki and setup an empty page before running any builds.

This plugin uses the jwbf library for publishing to the wiki and that library has a dependency on log4j 1.2.14 or above. Unfortunately, Jenkins uses an older version and so on your Jenkins machine, you must setup the “java.endorsed.dirs” environment variable to point to a directory that contains log4j 1.2.14 or above.

Let me know if this plugin has been useful to you and thanks to the excellent Jenkins build tool.

major.minor plugin for Jenkins

Recently, I’ve been tasked with setting up a build environment at work and we decided to go with Hudson and now Jenkins. I think the best part of Jenkins is its extensibility in the form of writing plugins to make it do exactly what you want. So my first plugin was to change the build numbers that Jenkins uses like #1, #2 etc. to something more meaningful for our scenario and so was born the major.minor plugin.

The premise of the plugin is quite simple. In our projects, we would like to see build names of the form major.minor.revision. We use subversion for source code management and so the revision number here refers to the svn revision number. The major and the minor numbers are configured when the build is initially setup and are initialized to 1 and 00 respectively. So, the first build name might be something like 1.00.23 if the svn revision at that point happened to be 23.

At some point down the line in the project cycle, we might like to increment the minor or major number and this can be done manually after a build by using already Jenkins feature to edit the build name. Since the plugin picks up the major and minor part of the build name from the previous build of the project, subsequent builds will now have the correct major and minor sections.

Now that the build name has been changed, when you browse through the builds directory for your project, it will be hard for you to figure out which build is what because Jenkins only creates directories using the build number like #2, #3 etc. and the date/time when the build was executed. To make it easier, the plugin also creates a new symbolic link with the build name in the builds directory. Now the Jenkins UI and builds directory matches and its easier for you to get to the right build.

When you enable this plugin, you get a new configuration section for your project. In this section, you will configure a regex to match build names against. I’ve made it a regex so that it is a little bit more flexible and can be still be used even if your organization does not use a major.minor.revision format for naming builds. The important thing is that you add a capturing group in your build name regex which is used to insert the new revision number into the build name. Note that if you don’t use svn for scm, the plugin still works and inserts the build number in place of the revision number. The other configuration is for setting the initial build name for your project like 1.00.00.

The plugin also makes available a BUILD_NAME environment variable for any scripts that might need it.

Hope this plugin helps anyone looking for nicer build names and thanks to the Jenkins project for a great build tool.

The plugin source is available on Github and you are free to fork and tinker with it suit your needs.

Hard drive fail!

Update:So it turns out that it wasn’t a windows update problem after all. My Samsung hard drive was faulting and had gone bad. Even though the Hard Drive S.M.A.R.T monitor was telling it was bad, there was no monitor on this machine and so I never saw it. Well, I got another WD hard drive over Christmas on Amazon for cheap. Samsung hard drives are no good, it seems. Lasted only as long as the warranty, 3 years.

So, about 1 month ago, I log into my server and see that I got about fifty windows update waiting to be installed. I think, oh what’s the harm and goes ahead and clicks update. And there began my month long attempt to bring this site back up.

First, windows would not boot back up anymore. No safe modes and windows repair couldn’t fix it either. And I think, this is as good a time as any to upgrade to windows server 2008 r2. So I go ahead and install it. I was doing periodic backups of data to another drive and so I didn’t I would lose anything.

But of course, I had never thought to test any of my backups. So, I got myself a brand new windows installation and my sql server database backup is messed up. All of my personal data was safe though. So, what happened was that sql server uses some sort striped multi-file backup mechanism and I only had 1 of 2 files required to restore my blog database. Great! You’ll get a nice little error message talking about having only 1 file out of a media set. You can read a good explanation here.  I’m not sure where the other file got backed up to.

Anyways, so I’ve scraping google and bing cache to get my blog posts back. Lots of fun! Luckily, I didn’t have much posts to begin with, so it isn’t too much work. It just took a month mainly because this has been a pretty tough semester. Machine Learning is not an easy subject and I’ve started interviewing for full-time jobs. Better start early than late. Only 1 more semester to go! Hope to have everything backup in order in 2 weeks. Until then, things are probably going to look a little messy.

Format of SVN access file for Path Based Authorization

Fine grained permissions on SVN repositories can be provided by creating an access file. Following is a sample access file explaining the basic format of the file. More details can be found by reading the SVN book.

#groups are defined here. simply list the names of members of the group delimited by comma
[groups]
admin = rudy, batman, admin

# to create access control for a repo, start with the directory that you are
# defining the permissions for
# to define permissions for all directories within a repo, use /
# for example, to define permissions for all directories in the gui project
# i've used the following [gui:/]
# the syntax is [reponame:directorypath]
# to define permission for a specific directory within the repo, specify the path
# example: [gui:/branches/2.0/bug/140] defines the permissions for the
# /gui/branches/2.0/bug/140 directory
# the syntax to define the permission itself is
# username = access
# where username is the username of the user you are writing permission for
# and access is one of r or rw, r meaning read permission and rw meaning read and
# write permission
# username could also be the name of a group. in that case, the group name is written
# as @groupname
# for example, to give the admin group rw permission on all directories in gui
# use,
# [gui:/]
# @admin = rw
# if you need to give all users permission on a directory, use the special
# character '*' to denote all users
# NOTE: The longest matching directory path permission is always applied first
# so if you have two permission rules like so,
# [gui: /]
# rudy = rw
#
# and
#
# [gui:/branches/2.0]
# nithin = r
#
# then, nithin will only have read permission on the /gui/branches/2.0 directory
# because that is the longest matching path
# More examles below:

[/]
* = r

[gui:/]
@admin = rw
nithin = rw
#
[moregui:/]
@admin = rw
matt = rw
john = r
chuck = rw
#
[hardcore:/branches/2.1/broke]
@admin = rw
#
[test:/]
@admin = rw
* = rw

Create SVN repositories using a standard folder structure

This is shell script that you can use to create all of your repositories using the same standard folder structure. You need to replace the following variables for it to run:

1. Replace [name] with the correct username of a user who will have permissions on the repository.
2. Replace [password] with the correct password of the user
3. Replace yourdomain.com with the path to where your svn repositories are stored

call the script: ./scriptname reponame and it will create a new repository with the name reponame.

Here’s what the script does:

1. The script creates the new repository folder
2. It gives apache permissions on the folder. If you are not using apache, uncomment the line.
3. If you are using an access file to provide fine-tuned directory permissions, you may want to uncomment the two lines that refer to access.txt. Those two lines add the new repo to the access file and gives the admin group rw permissions on the repository.
4. It then goes and creates the standard folder structure, ie, trunk, branches adn tags folder.

#!/bin/bash
svnadmin create $1;
chgrp -R apache $1;
chmod -R g+rw $1;
#echo "[$1:/]" >> access.txt
#echo "@admin = rw" >> access.txt
svn –username [name] –password [pass] mkdir "http://yourdomain.com/svn/"$1"/trunk" -m "ADD: /trunk";
svn –username [name] –password [pass] mkdir "http://yourdomain.com/svn/"$1"/branches" -m "ADD: /branches";
svn –username [name]—password [pass] mkdir "http://yourdomain.com/svn/"$1"/tags" -m "ADD: /tags";

echo "------------------------------------------";
echo "Standard Repo Layout Created";

jQuery Formhints plugin

A very simple jquery plugin for showing hints right next to form inputs. This is a jquery adaptation of the excellent javascript & css form hint script here.

I’ve made it where the form hints show up when you mouse over the input elements. You can also make a small edit to the script so that the formhints only show when the user actually clicks on an input element. Just change the “input.mouseover” to an “input.click”.

(function($){

	$.fn.formhints = function(params){
		$(":input", this).each(
				function(i)
				{
					var input = $(this);
					if(input.next("span").length > 0)
					{ // if this input element has a span next to it
						$(input.next("span")).hide();// initially hide all
						input.mouseover(function() {
							var pos = $(this).position();
							var left = pos.left + $(this).width() + 30;
							$(input.next("span")).css({'display':'inline', 'top':pos.top+'px',
								'left':left+'px'});
						});
						input.mouseout(function() {
							$(input.next("span")).css('display','none');
						});
					}
				}
		);
	};

})(jQuery);

In order to use it, the form html needs to be modified. Each input element with a hint must have an adjacent “span” element which contains the hint text that will be displayed to the user. Here’s a sample input element:

<div class="input text required">
	<label for="ScriptName">Script name</label>
	<input id="ScriptName" maxlength="200" name="data[Script][name]" type="text" />
	<span class="hint">
		Name of the script
		<span class="hint-pointer"> </span>
	</span></div>

Note the span element after the input element. It will contain the hint that will be displayed to the user on mouseover. You can apply css styles to the hint to make it look nice. I’m just reusing the hint class developed by askthecssguy for his script which looks quite nice.

Once you’ve gotten the span elements added to your form, add this one line to your page load function to get your form ready for action.

$(document).ready(function(){
	$("#formidhere").formhints();
});

And that’s it. Make sure to replace the “formidhere” with the id of your form.

Here’s the css I used for my form hint and the final result.

.hint {
  position: absolute;
  width: 200px;
  margin-top: -4px;
  border: 1px solid #c93;
  padding: 10px 12px;
  background: #ffc url(../img/hint_pointer.gif) no-repeat -100px -100px;
}

.hint .hint-pointer {
    position: absolute;
    left: -10px;
    top: 5px;
    width: 10px;
    height: 19px;
    background: url(../img/hint_pointer.gif) left top no-repeat;

}

<images here>

Installing SVN server

1. Install latest Apache http server
2. Install SVN
   1. Make sure to install mod_dav_svn and mod_authz_svn. mod_authz_svn is only needed if you plan on doing fine-grained directory permissions on your SVN repositories
   2. Load the mod_dav_svn and mod_authz_svn modules in httpd.conf. You may also need to load the mod_dav module if you don’t have it yet:

   LoadModule dav_module modules/mod_dav.so
   LoadModule dav_svn_module modules/mod_dav_svn.so 
   LoadModule authz_svn_module modules/mod_authz_svn.so 
   

   3. Modify apache httpd.conf to add the following:

    
   <Location svn> 
   DAV svn 
   SVNParentPath D:/svn 
   SVNListParentPath on 
   AuthType Basic 
   AuthName &quot;Subversion Repository&quot; 
   AuthUserFile passwd.txt 
   Require valid-user
   </Location>
   

   4. You should now be able to browse to your SVN repos at www.yourdomain.com/svn

   5. If you would like to enable anonymous checkouts, replace the Require valid-user above with the following. Check-ins will still require authentication.

    
   # For any operations other than these, require an authenticated user. 
   <LimitExcept GET PROPFIND OPTIONS REPORT>
   Require valid-user
   </LimitExcept>
   

   6. If you are not going to be using per-directory access control, you might as well turn off path authorization (it’s on by default). This improves performance of your server, since its not checking every single path requested to see if the user is authorized to access it. Add this line to turn it off:

   SVNPathAuthz off
   

   7. Install PHP

   8. Install an SVN browser, WebSVN is the best I’ve seen. It’s built on PHP and works well.

   9. Subclipse is the best SVN client for use with Eclipse IDE

   10. TortoiseSVN is a good SVN windows explorer client

Moving repositories/ Creating backups in SVN

Recently, I needed to move some projects around in my SVN and it turned out to be very easy, although it took a bit of googling. This also gives you a way to take file backups of your repositories. Basically, what I needed to do was move one repository into a subfolder inside another repository. To do this, you just need two commands and you also need to do this on the SVN server. You can’t do it from an SVN client. Here are the two commands:

svnadmin dump /path/to/repository > filename.dmp
svnadmin load –-parent-dir path/to/subfolder repository-name < filename.dmp

So basically, the first command dumps your current repository including its previous revisions, branches and tags all to the standard output and you can redirect that into a file. You can use this command to take periodic backups of your repository which is really neat. I was actually searching for this feature as well and there might be other ways to take backups as well.

The second command then loads up the repository from the dump file into the subfolder. If you don’t specify the –-parent-dir, it will just load up the file into the new repository. The parent-dir path is relative to the root of your new repository.

Here are the links that helped me find this:

http://svnbook.red-bean.com/nightly/en/svn.ref.svnadmin.c.load.html

http://www.digitalmediaminute.com/article/2251/how-to-move-a-subversion-repository