My referencing process

I suspect everyone has a subtly different way of doing this, depending on the tools they prefer and the way they like to work. I thought I would document my process as I think it's pretty efficient and has some real advantages, I'm sure there is still plenty of room for improvement though.

Overall aim

As an academic I often need to refer to work done by others, this normally done by noting their published work as a list of references or bibliography at the end of my documents. The most basic way of achieving this would be to have a stack of published books and papers in my drawer that I can refer to and then type in the reference details at the end of the each document I want to write. I'm sure that plenty of people do work this way, but from my point of view I see it as pretty inefficient (lots of printing, lots of sorting, lots of typing, not very portable, an awful pain to change reference styles, etc).

I manage my references entirely in software using a few specific tools. I've mentioned most of them in other posts but I'll go into a bit more detail here on the actual process I use. 

I'm going to split referencing into two separate processes. Firstly, as I'm researching a topic, I tend to gather references to get an idea what I'm doing. Secondly, when I come to document my work I need to search and cite the references I've found.

Tools

Google Scholar is generally my primary resource for finding papers and documents. I also rely on standard Google searches a massive amount and I have a Google alert set up to email me when a few key phrases appear in new articles added to the web.

IEEExplore generally has most of the published work (in terms of papers, Journal articles, etc) that I need to refer to. The University has a subscription to this that allows me to download what I need (otherwise I'd have to pay!).

Pdf format is generally how almost all papers are delivered and the format that I keep them in. I use Foxit reader to read pdf documents. I keep all my reference documents in one big folder rather than worrying about any kind of complex filing system.

I use Bazaar to version control my folder of pdf documents. I've previously discussed how this works and how it allows me to work between different computers, even without installing any software on them.

I use a tool called cb2bib to maintain a list of references in bibtex format. I started off just using this to add references to my bibtex file, but I've found it to also be really good for browsing references and citing whilst writing a document. I changed some of the default setup to help it retrieve data from the net.

I use Latex to typeset most of my work. Within this I can simply point it at the bibtex file for all the details of the references. I have previously mentioned how to use a bibtex file that is not in the same place as the rest latex document.

Gathering references

1. Search for the document that I need to find using Google, Google Scholar or general web browsing.
2. Open and read the document to see if it looks relevant and useful. Assuming that it does...
3. Download the document to my big folder of "third party" references. I tend to use the full title of the work as the save name of the document - this can lead to quite long file names, but it makes it a lot easier to find things!
4. Add the saved file to my bazaar version control system. This only takes a couple of clicks through  tortoiseBzr menus in windows explorer.
5. Add the document to my bibtex file using cb2bib. This is a pretty straightforward process:
   1. Open cb2bib, I have a keyboard shortcut setup for this (it should also remember what bibtex file is being used)
   2. Click "import from pdf file"
   3. Click "select files"
   4. Select pdf files saved previously (hold ctrl to select a bunch at once)
   5. Click "process"
   6. The software will try to extract as much info as possible from the pdf file, this probably won't be enough so...
   7. Click Network query to retrieve all the info about the file from the web (this usually works fine, but it's worth checking the results, you may need to give it the right title to start it off)
   8. Click save to add the reference to the bibtex file
6. The changed bibtex file and added references will need to be "commited" to the version control repository.

UPDATE: I have added now made a batch file that effectively takes the place of step 4 and occurs after step 5 - it can be run after a whole set of references have been input through cd2bib and adds them all to my version control repository using "Alt"+"p".

Citing references

1. With a Latex document that has a pointer to the bibtex file within it.
2. Open cb2bib citer (I use a keyboard shortcut) and select the reference(s) required:
   1. [optional] Select the way I want my reference list displayed by pressing "a" (author), "j" (journal), "t" (title) or "y" (year). I find author is usually best.
   2. [optional] Filter the reference list by pressing "f" and then typing what you want to search for ("d" clears the search)
   3. Click on a chosen reference
   4. [optional] Press "o" to open the reference and read it
   5. Press "enter" to cite the reference, a small pen marker will appear next to it (in author view it will appear next to each author for the same paper). Multiple references may be cited by selecting them and pressing "enter". "delete" clears all the selected references.
   6. Press "c" once all the references for citing are selected, this will close the citer window and copy the latex text for the references to the clipboard.
3. Paste the text into the latex file to include the references at that point in the document.

Although those might seem like a lot of steps it's really pretty straightforward once you get used to it. The only real difficulty is remembering the keyboard commands for cb2bib citer. With this setup I can take all my references with me between machines (even between Linux and Windows) and use the same process everywhere.

Bazaar GUI issue on Windows 7 - adding files using the commandline GUI interface

Due to an unfortunate hard drive issue I recently had cause to swap to a new drive. Along with this came an upgrade from XP to Windows7 (64bit). This was not a real issue, just a bit of hassle moving everything over and re-installing all my applications.

As part of the process I had to re-install Bazaar which I'm using to version control all my work. This appeared to go fine, including the addition of tortoiseBzr to integrate the Bazaar commands into the windows explorer GUI. Unfortunately when I came to use it with my work it gave me an error, I think due to the "special" folders structure in windows7. Here is a copy of the message I posted to the Bazaar user group:

Hi everyone, I hope someone can help me out.

I'm using Bazaar to version control a whole set of different files between a few different machines. On a previous windows XP machine I was able to use the whole user area as my repository and only add the files that I wanted to control.

I have just upgraded to a windows 7 machine (64bit) and there seem to be some issues with the preconfigured folders not being accessible by Bazaar. I have been able to commit files fine, but when I try to open a window to add new files it crashes as it tries to display them, with the error:

bzr: ERROR: [Error 5] Access is denied: u'C:/Users/welf/AppData/Local/Application Data\\*.*'

(It also seems to have issues with "My Music" and similar)

Any thoughts on why it's happening or a way round the problem?

(I suspect that it would work if I had the repository at a lower level without any "windows" folders, but that is not really the way I'd like to work. I don't actually need to version control anything in these folders, so if they can be skipped in some way that would be fine.)

It seems suspiciously similar to an outstanding bug here:
https://bugs.launchpad.net/qbzr/+bug/1012907

The general upshot is that I can't add new files to version control using the GUI, although I seem to be able to do pretty much everything else. I thought I would detail the workaround I'm currently using. It isn't too bad, but wasn't totally obvious, so I thought I would document it in case it was of use to anyone else:

1. Right click in the folder containing the files and select the "Tortoise Bazaar" menu item, then select "Run command". A GUI window to allow you to run a command will open.

2. Select "Core" after "Category"
3. Select "add" after "Command"
4. Click "Insert filenames...", a file selecting window will open.
5. Select the files you wish to add (hold 'Ctrl' to select multiple files) and click "Open"

6. Click "OK" and the status window should tell you that the files have been added

You're now at the same stage as if you'd added the files using the standard GUI. Of course you will still need to perform a commit before the files are part of the repository.

Hope that's useful to someone.


UPDATE: Discussion on a Bazaar email list confirms that the problem I'm seeing is due to the error linked above. Unfortunately a fix does not look imminent therefore I'll have to continue using the command-line method described.

Bazaar revert only one file to previous version

So this was the situation I was recently in:

• I found that a simulation I was running in Simulink was giving me results that seemed to be wrong
• I was sure that previously it had given me the right results
• Therefore I wanted to "roll-back" to the previous version of that Simulink model

I thought this would be a simple and easy thing to do as I have all my files under version control. This sort of procedure is termed "reverting" in version control lingo. It was certainly possible, however it wasn't quite as easy as I expected, so I'll detail the process required.

As noted previously I'm using Bazaar for version control, through both the TortoiseBzr windows interface and also the Bazaar Explorer GUI. Bazaar supports several different methods of undoing mistakes. Through either method of accessing Bazaar you can get at a log of all the commits done on a certain file. From this window you have the option of either "Revert to this revision" or "Update to this revision". Either of these will give you the previous version (although the first one will not count the changes as committed); however they will give you whole of the repository at that revision. This might be useful for pulling out a whole set of software that works together, or establishing the state of an entire project at a particular point, but this is not what I needed.

It is possible to get at an individual file, but it involves a little bit more work. It may also be worth ensuring that the current version is committed in case you later decide that reverting was a mistake. Here is the process:

1. Look at the log of file to determine what revision number contains the version you want (this is where having written good commit messages will help!)
2. Make a note of the revision number that you require
3. Highlight the address of the folder you are in and copy it
4. Open a command line (on windows go to start - run... - "cmd")
5. Change to the directory you require by typing "cd " and then pasting the folder address you copied before
6. Type in the Bazaar revert command as follows:

bzr revert -r num file.ext
where:
num is the revision number that you require (as noted from the log)
file.ext is the filename and extension that you wish to revert to (this could be a list of files if you prefer)

This should give you the previous version of only that file. It will not be committed so you can check that it is the right version and make any changes you want before recommitting.

Matlab figures in Powerpoint

I've just been putting together a presentation in which I wanted to include a set of Matlab plots. Whilst there are plenty of ways to do this I thought I would detail the way I managed it...

I've mentioned before about maintaining quality by using vector graphics. Previously I've used .eps format for this, but I don't think it's well supported in Powerpoint (I did consider using alternative presentation software but I was pressed for time and this seemed like the easiest option). Instead I used .emf format which seems pretty well supported by Matlab. This is a Windows specific format so I can't say I'm particularly keen on it for general use, but it served a purpose here.

I saved my figures as separate images from Matlab and then imported into Powerpoint, but I think the same result can be achieved if you simply copy and paste between the two.

One of the benefits of having the plots in vector format is the ability to manipulate them within Powerpoint itself. For the unscrupulous this could be a good technique for "adjusting" a few points here and there to improve  your results - but obviously I didn't do that! What I did do was separate out several lines that were on the same plot, so that I could make them appear (using Powerpoint's animation features) gradually. This made the plots (which were a comparison of the results of several different simulations) much cleaner.

It was a little bit of a hassle to do this, but not awful, and I think worthwhile as it made the results much easier to present. When the .emf file is initially imported it is handled as an image; however, if you right click on it you will find that it can be "ungrouped". You might have to repeat this process a few times as the image seems to be built up of several sub-groups, but once you're done you'll find that individual lines are available to be manipulated. Unfortunately I then had to go through and "regroup" some of them manually to reform complete lines, which involved a bit of 'Ctrl' clicking - this was the tedious bit. I guess it depends how complicated the lines in your plots are as to how long this will take (and whether it is worthwhile or not).

Another benefit was being able to alter the font of my axes to match my presentation style. A trivial point perhaps, but it all goes to improve the overall appearance.

So to summarise:

1. Copy figure in Matlab
2. Paste in Powerpoint
3. Right click and "ungroup"
4. Repeat 3. until you get to a  level that you can manipulate the objects that you want
5. You may find there are a few large white squares that form the background of the plot - these can be deleted (it will make things easier when you come to regrouping things in step 7.)
6. 'ctrl'+'a' to select everything and then change the font to match your presentation
7. 'ctrl' and click on objects to collect all the ones that should be in a group (you may need to zoom in on your slides to do this accurately)
8. right click and "group" them
9. move/recolour/animate/resize the objects as you desire to suit your presentation!

Hope that is of help to someone!?

IET conference paper reference style

I've recently prepared a paper for a forthcoming IET conference in which I cited references.
Whilst the template for the paper provided an example of the format it would like references to journals to be included in, it gave no further advice.

Assuming that there would be a standard format for all IET conference papers I went a-googling - with very little success. My next option was to ask on the IET forums here. Unfortunately this didn't turn up any standard formats either. Therefore my conclusion is that a "standard" doesn't exit.

This isn't a major issue as I think you'd have to work quite hard to contrive an example where a reader wouldn't be able to decipher a reference due to the use of an obtuse format. However I think it would be good practice to document the format I've used here...

The conference template suggests (for a journal article):
[1]    A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110-120, (2000).

Surprisingly this differs from the IET advice for research journal citations and the two documents on IEEE reference formats.

Therefore the style I decided to use is:

Journal article

•  A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110 – 120, (year).

Conference paper

•  A. B. Author, C. D. Author. “The title of the conference paper”, The Conference, Location City, Location Country, pp. 1-7, (Month year).

Book, book chapter and manual

• A. B. Author, C. D. Author. “The title of the book chapter”, in D. Editor (Ed.), The title of the book (Publisher, year, 1st edn.), pp. 1 – 7.

Book

           A. B. Author, C. D. Author. The title of the book (Publisher, year, 2nd edn. 2006).

Patent

• A. B. Author.  “The title of the patent”, British Patent 123456, (Month year).
• A. B. Author, C. D. Author. British Patent Application 98765, (Month year).

Thesis

• A. B. Author.  “The title of the thesis”, PhD thesis, XYZ University, year.

Standard

• BS1234:  “The title of the standard”, year.

Website

• http://www.theiet.org, accessed Month year.

As I prepared the paper using LaTeX, all my referencing was done using BibTeX. I therefore needed to create a custom bibliography style file. I did this using the custom-bib package. This a fairly straightforward process once it's figured out, roughly consisting of:

1. run latex on "username"\Application Data\MikTeX\2.8\tex\latex\custom-bib\makebst.tex
2. answer questions appearing in the output window
3. get *.dbj file produced at the end
4. run latex on that file to produce *.bst file (I found that the full path to the file was required at this step, so it needed to be run from the command line rather than through an editor)

There is a fairly large set of questions that allow you to customise the style through this process, so I ended up with a style that was pretty close to what I wanted. Unfortunately there were a few niggling differences in the output. Most significantly custom-bib does not allow the option of changing the comma between the author list and title to a period. To rectify this I had to dive into the .bst file and start hacking!

Now I wouldn't recommend this if it can be avoided as the language used is pretty obscure, however I eventually got something that looked about right.

If you want to use my style file it's here. Obviously it comes with no warranty and no official backing from the IET. It looked ok for the paper I submitted but I'd recommend thorough checking of the output if you do use it.

Please let me know if this is of use to you, or if you have any comments on my chosen style, using custom-bib or any other part of my process.

Weather in iGoogle

I've been using iGoogle for a while now. For those not in the know, this personalises the Google homepage and allows you to add "gadgets". Since I've started using Chrome as my browser I tend to Google search from the address bar anyway, but I still find iGoogle pretty handy.

I use it as a way to amass all the stuff I tend to look at in one session. Currently I have three different tabs:

• News - Local and world affairs and specific technology news
• Comics - Webcomics that I like to read
• Weather - Local weather forecasts so I know if I need a coat

These are all areas that I tend to want to update myself on regularly - approximately once a day. So instead of trawling through various bookmarked pages I can accumulate all the info in one convenient page.

This works out really well and I've found there are specific gadgets for most of the pages the I want to visit. Some of the news related ones didn't have gadgets but I was able to use use their RSS feed to display them in an RSS reader gadget.

The only real pain was the some of the weather forecast sites that I wanted to look at didn't have gadgets - most notably Metcheck.com. I really like the way they display the forecast so I set about trying to get it onto an iGoogle page. The 'official' way to do this I guess would be to make a specific gadget; however this seemed like it would be a lot of effort and learning time. So I went with a different option.

I used a gadget that allows the inclusion of html. Then I wrote some HTML to include a portion of another website. This allowed me to trim the site down to just the specific window I wanted. For example:
<style type="text/css"> #container{ width: 0px; height: 0px;} #container iframe {width: 780px;   height: 590px;   margin: -180px 0px 0px -240px; }   </style> <div id="container"> <iframe scrolling="no" width="0" height="0" src="https://www.netweather.tv/index.cgi?action=uk7dayx7;page=1;ct=19121~Manchester;sess=#forecast></iframe></div>

This includes the portion of netweather.tv for my specific region. I was previously using metcheck but have recently switched as their results page seems to regularly give errors at the moment.

There might be an easier way of doing things, and for other websites the numbers in the HTML will need fiddling with so that the right portion of the page is in the window, but this technique seems to work pretty well for now.

Future documentation methods

Coming towards the end of the first year of my PhD and spending some time writing up my progress so far has led me to muse over the nature of report writing and ask the question: "surely there must be something better"...

Whilst the old adage "if it ain't broke don't fix it" might well apply here, I can't help but think that in the 21st century of immersive 3D virtual reality game playing, home 3D printing, and everyone carrying at least one state of the art electronic device about their person at all times, the concept of a paper report seems a little dated.

Here are a few examples to try to illustrate my point:

• If there is a book and a film of the same story (or even a webpage and a youtube video) I will inevitably look at the the film first as it will convey the information to me far faster, and with less effort than reading the book.
• If I have a choice between a photo or drawing of an object and a 3D model (either manipulable on screen or available to touch) I would get a better understanding of it from the 3D.
• If I want information on a specific subject then I turn to Google/Wikipedia before I head off to the library.

I don't think these are examples of me being weird, they are simply illustrations of modern life making information more readily accessible. I'm sure you could argue over semantics ("the book contains more detail than the film", "library books have a more systematic review process than google hits", etc) but I hope you can accept my general point.

It therefore seems strange to me that a piece of work, perhaps costing thousands of pounds and many hundreds of man hours, should be presented in such a one dimensional format as a printed report. Here is a summary of what I see as the limitations to a printed report:

1. One dimensionality - sure pictures might take this up to 2D, but all too often there aren't enough of these!
2. Lack of user interaction - I can't search for a keyword, interactively link to source, or request further detail on a topic.
3. Page constrained format - diagrams need to fit within a certain width, zooming in is limited by your eyes and printer resolution and page breaks artificially chop things up.
4. Visual sense only - My other senses are put on hold, and only serve as a distraction.

So what have people done to improve on this? Here're a few examples that I can think of:

• Video - a good recent example is this guys youtube CV
• Hyperlinking - within sections of a document or out to other documents or web sites
• Wiki formats - taking linking between sections to the extreme and making progress through the information less linear
• 3D graphics - starting to be seen more in web pages, an excellent example is Google body
• Powerpoint - a format often used in place of a standard document, it has many of the same issues, however users often seem to feel a little less constrained in terms of layout (perhaps this is only due to convention?)
• Computable document format - this is a really exciting new development that reflects a lot of what I'm describing here

This last concept may or may not take off but I can see what they are hoping it will achieve. Some of its functionality can already be achieved in a pdf (details of how to achieve a lot of them through LaTeX are here) and almost all of it could also be done through HTML and javascript. An interesting discussion on this is given here. In fact the recently developed HTML5, in combination with javascript programming, offers a whole mass of interesting possibilities for the presentation of information. A step towards using HTML5 for what I'm talking about here is Tangle. This is a javascript library that supports the production of "reactive documents", allowing a reader to play with the content of the document.

Another alternative format with a lot of capability is Flash animation, these animations are typically web-based and often allow user interaction. Some basic options for creating these are given here. Although it is a very widely used format it requires a good level of experience to be able to code it. It has also faced quite widespread criticism recently, the most high profile of which came from Apple, and therefore there is speculation about whether HTML5 will ultimately replace it.

An obvious downside to these types of advanced documentation method is the length of time it takes to actually produce a document. Even when the author has a good knowledge of the specific tool they're using I think it's safe to say that nothing I've mentioned above will be as quick to produce as a simple text document. In fact the more advanced the documentation method - the longer it's likely to take to produce.

I'd love to be able to round off with a recommendation of the ultimate tool or combination of tools that can be used to create the perfect document, but as far as I've seen it doesn't yet exist. Lots of things seem to offer at least part of the solution I'm looking for, but none pull it all together into one great package. So instead I'll do two things, firstly I'll make a few plain points in summary/prediction, then I'll put together a set of use cases that I'd like to see available to the end user of my "ultimate document".

Summary/Predictions

• The plain printed word document is currently in the process of being overtaken by more electronic forms of documentation, inherently bringing a lot more potential to the document itself (hyperlinking and embedded video being two major ones). I would expect this to be a continuing trend (that may eventually even reach formal engineering reporting or even academia!).
• There is the potential for this to go a lot further than the type of electronic documents seen today with the addition of 3D effects, audio tracks and similar.
• HTML5 currently seems to offer the most potential for supporting this type of advanced documentation (although the computable document format may also be a candidate if it manages to pickup much of a user base).
• Very little progress towards this end goal will be achieved until there are good tools for authoring the type of document I'm discussing here.
• It seems highly likely that viewing of any document of this type will be through a web browser or similar.

Use cases - scenarios that I, as an end 'reader', would like to see supported in this ultimate document format.

1. User managed detail level - I'd like to be able to look in more detail at sections I'm interested in or know little about, whilst invisibly skipping over the mundane or tedious stuff.
2. Unconstrained document flow - If I want to read summary, then the contents, then the conclusions, then methods, it should be easy for me to work through that way.
3. Recommended document flow - If I simply want to be guided through the document ensuring that I pick up all the important information then this should also be easy
4. User interaction - Where more information could be made available then I should be able to access it. For example I should be able to zoom in on a waveform or rotate a 3D model.
5. Multiple sense stimulation - practically this is likely to be limited to visual and audio currently (at least until we develop smell-o-vision and feel-o-vision...)
6. Portability - I want this document to be viewable in as many places as possible, consequently it must be compact and easily openable on a variety of devices (laptops, mobiles, touchpads, e-readers, etc). This might even extend to alternative language/disability support and (somewhat ironically) the ability to print onto plain old paper.

So what have I missed? I'd love to discuss this topic and related areas more so please leave me a comment.
I'd also love to have the time, skills and supervisor buy-in to trying to present my thesis in the manner I've outlined; however I suspect that that will remain a pipe-dream...