Adding bzr files through the cb2bib

I recently described my referencing process where I simultaneously hold reference pdf files in a version control system and keep track of their details in bibtex file. I have also described a problem I've been having with my version control system of choice, Bazaar.

Based on these I decided that there might be a better (more automated) way of adding my references. After a very useful email exchange with the author of cb2bib they confirmed that this should be possible. Now, after a day of messing around and learning about various command-line tools that I hadn't used before, I think I've got it working. Here's how...

First I created a new batch file that I called "bzrAddRef.bat". Here are its contents:


@echo OFF

rem --------------------------------------------------------------------------
rem cb2Bib Tools addition by J. Welford
rem --------------------------------------------------------------------------


echo   bzrAddRef:
echo   cb2Bib script for adding BibTeX files to a Bazaar repository
echo.
echo   Using sed and xargs utilities from:
echo   http://gnuwin32.sourceforge.net
echo.
echo   Path below may need changing to be within the repository
echo.


echo   Processing:
echo.
cd "C:\Users\welf\Documents\literatureReview"
sed -n -e "s|.*file.*=.*{\(.*\)}.*|\"\1\"|p" %1 > bzrRefs.tmp
xargs bzr add <bzrRefs.tmp
del bzrRefs.tmp

Only the last 4 lines are really important. The first one sets the working directory, I don't think it matters what you use as long as you have write access and it is within the repository you want to add to. The next line runs through the current bibtex file and extracts the location of all the references to a temporary file. The next line adds all these files to Bazaar repository. The final line deletes the temporary file. (maybe I could have used a pipe between commands so that the temporary file was not required?)


Some special commands are used that will need to be installed, what you need are: sed.exe (and its dependencies:  regex2.dll, libintl3.dll, libiconv2.dll) and xargs.exe.

Within the settings of cb2bib this batch file can now be pointed at under the "Configure BibTeX" - "External BibTeX Postprocessing" - "Command:" section. Once that is done simply hitting "Alt"+"p" in the cb2bib window should run the batch file and add all the references to version control.

I hope the helps someone! (I presume it could be altered for other version control systems or be called from other applications.)

My referencing process

I suspect everyone has a subtly different way of doing this, depending on the tools they prefer and the way they like to work. I thought I would document my process as I think it's pretty efficient and has some real advantages, I'm sure there is still plenty of room for improvement though.

Overall aim

As an academic I often need to refer to work done by others, this normally done by noting their published work as a list of references or bibliography at the end of my documents. The most basic way of achieving this would be to have a stack of published books and papers in my drawer that I can refer to and then type in the reference details at the end of the each document I want to write. I'm sure that plenty of people do work this way, but from my point of view I see it as pretty inefficient (lots of printing, lots of sorting, lots of typing, not very portable, an awful pain to change reference styles, etc).

I manage my references entirely in software using a few specific tools. I've mentioned most of them in other posts but I'll go into a bit more detail here on the actual process I use. 

I'm going to split referencing into two separate processes. Firstly, as I'm researching a topic, I tend to gather references to get an idea what I'm doing. Secondly, when I come to document my work I need to search and cite the references I've found.

Tools

Google Scholar is generally my primary resource for finding papers and documents. I also rely on standard Google searches a massive amount and I have a Google alert set up to email me when a few key phrases appear in new articles added to the web.

IEEExplore generally has most of the published work (in terms of papers, Journal articles, etc) that I need to refer to. The University has a subscription to this that allows me to download what I need (otherwise I'd have to pay!).

Pdf format is generally how almost all papers are delivered and the format that I keep them in. I use Foxit reader to read pdf documents. I keep all my reference documents in one big folder rather than worrying about any kind of complex filing system.

I use Bazaar to version control my folder of pdf documents. I've previously discussed how this works and how it allows me to work between different computers, even without installing any software on them.

I use a tool called cb2bib to maintain a list of references in bibtex format. I started off just using this to add references to my bibtex file, but I've found it to also be really good for browsing references and citing whilst writing a document. I changed some of the default setup to help it retrieve data from the net.

I use Latex to typeset most of my work. Within this I can simply point it at the bibtex file for all the details of the references. I have previously mentioned how to use a bibtex file that is not in the same place as the rest latex document.

Gathering references

1. Search for the document that I need to find using Google, Google Scholar or general web browsing.
2. Open and read the document to see if it looks relevant and useful. Assuming that it does...
3. Download the document to my big folder of "third party" references. I tend to use the full title of the work as the save name of the document - this can lead to quite long file names, but it makes it a lot easier to find things!
4. Add the saved file to my bazaar version control system. This only takes a couple of clicks through  tortoiseBzr menus in windows explorer.
5. Add the document to my bibtex file using cb2bib. This is a pretty straightforward process:
   1. Open cb2bib, I have a keyboard shortcut setup for this (it should also remember what bibtex file is being used)
   2. Click "import from pdf file"
   3. Click "select files"
   4. Select pdf files saved previously (hold ctrl to select a bunch at once)
   5. Click "process"
   6. The software will try to extract as much info as possible from the pdf file, this probably won't be enough so...
   7. Click Network query to retrieve all the info about the file from the web (this usually works fine, but it's worth checking the results, you may need to give it the right title to start it off)
   8. Click save to add the reference to the bibtex file
6. The changed bibtex file and added references will need to be "commited" to the version control repository.

UPDATE: I have added now made a batch file that effectively takes the place of step 4 and occurs after step 5 - it can be run after a whole set of references have been input through cd2bib and adds them all to my version control repository using "Alt"+"p".

Citing references

1. With a Latex document that has a pointer to the bibtex file within it.
2. Open cb2bib citer (I use a keyboard shortcut) and select the reference(s) required:
   1. [optional] Select the way I want my reference list displayed by pressing "a" (author), "j" (journal), "t" (title) or "y" (year). I find author is usually best.
   2. [optional] Filter the reference list by pressing "f" and then typing what you want to search for ("d" clears the search)
   3. Click on a chosen reference
   4. [optional] Press "o" to open the reference and read it
   5. Press "enter" to cite the reference, a small pen marker will appear next to it (in author view it will appear next to each author for the same paper). Multiple references may be cited by selecting them and pressing "enter". "delete" clears all the selected references.
   6. Press "c" once all the references for citing are selected, this will close the citer window and copy the latex text for the references to the clipboard.
3. Paste the text into the latex file to include the references at that point in the document.

Although those might seem like a lot of steps it's really pretty straightforward once you get used to it. The only real difficulty is remembering the keyboard commands for cb2bib citer. With this setup I can take all my references with me between machines (even between Linux and Windows) and use the same process everywhere.

IET conference paper reference style

I've recently prepared a paper for a forthcoming IET conference in which I cited references.
Whilst the template for the paper provided an example of the format it would like references to journals to be included in, it gave no further advice.

Assuming that there would be a standard format for all IET conference papers I went a-googling - with very little success. My next option was to ask on the IET forums here. Unfortunately this didn't turn up any standard formats either. Therefore my conclusion is that a "standard" doesn't exit.

This isn't a major issue as I think you'd have to work quite hard to contrive an example where a reader wouldn't be able to decipher a reference due to the use of an obtuse format. However I think it would be good practice to document the format I've used here...

The conference template suggests (for a journal article):
[1]    A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110-120, (2000).

Surprisingly this differs from the IET advice for research journal citations and the two documents on IEEE reference formats.

Therefore the style I decided to use is:

Journal article

•  A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110 – 120, (year).

Conference paper

•  A. B. Author, C. D. Author. “The title of the conference paper”, The Conference, Location City, Location Country, pp. 1-7, (Month year).

Book, book chapter and manual

• A. B. Author, C. D. Author. “The title of the book chapter”, in D. Editor (Ed.), The title of the book (Publisher, year, 1st edn.), pp. 1 – 7.

Book

           A. B. Author, C. D. Author. The title of the book (Publisher, year, 2nd edn. 2006).

Patent

• A. B. Author.  “The title of the patent”, British Patent 123456, (Month year).
• A. B. Author, C. D. Author. British Patent Application 98765, (Month year).

Thesis

• A. B. Author.  “The title of the thesis”, PhD thesis, XYZ University, year.

Standard

• BS1234:  “The title of the standard”, year.

Website

• http://www.theiet.org, accessed Month year.

As I prepared the paper using LaTeX, all my referencing was done using BibTeX. I therefore needed to create a custom bibliography style file. I did this using the custom-bib package. This a fairly straightforward process once it's figured out, roughly consisting of:

1. run latex on "username"\Application Data\MikTeX\2.8\tex\latex\custom-bib\makebst.tex
2. answer questions appearing in the output window
3. get *.dbj file produced at the end
4. run latex on that file to produce *.bst file (I found that the full path to the file was required at this step, so it needed to be run from the command line rather than through an editor)

There is a fairly large set of questions that allow you to customise the style through this process, so I ended up with a style that was pretty close to what I wanted. Unfortunately there were a few niggling differences in the output. Most significantly custom-bib does not allow the option of changing the comma between the author list and title to a period. To rectify this I had to dive into the .bst file and start hacking!

Now I wouldn't recommend this if it can be avoided as the language used is pretty obscure, however I eventually got something that looked about right.

If you want to use my style file it's here. Obviously it comes with no warranty and no official backing from the IET. It looked ok for the paper I submitted but I'd recommend thorough checking of the output if you do use it.

Please let me know if this is of use to you, or if you have any comments on my chosen style, using custom-bib or any other part of my process.

Using a remote Bibtex file

As mentioned previously:

1. I am using version control software to backup my work
2. I'm using LaTeX
3. I don't like duplicating files around my system
4. I'm working between both Windows and Linux

So I've found myself today writing an end of year report (housed within its own folder in my filesystem) and wanting to pull in some references from my bibtex file (housed within the literature review folder in my filesystem). I could use the copying hack that I previously devised for figures, but that wasn't very elegant.

There are plenty of questions about this out there in internetland, mostly with the following suggestions:

• Move your bibtex file to within the tex installation root - I can't really do this because of 1. above.
• Use symbolic links to the original in the local folder - I can't do this becuase of 4. (windows doesn't support them)
• Give a relative filename reference e.g. '\bibiography{../literatureReview/references}'
• Adding the location as an environment variable as described here.

Both of these later two methods seemed to work for me. I went with the relative referencing for portability between systems. Odd that such a straightforward solution works for references but not for figures!?