24/05/2015

Reflections

It's now been 9 months since I successfully submitted my thesis and passed my viva. I'm glad to say that although I was given minor corrections, they were very minor, requiring only a a few modifications to my thesis.

Although at this stage I cannot make the thesis itself publicly available, I think its worth closing out this blog with some reflections on what I think worked well about the tools and techniques that I used, and also noting some areas that I might perhaps do differently a second time around.

What worked

Version control

This was really key for me, and helped me keep everything in good shape throughout the work. I wrote about the technique I used and a few tips and tricks for the software (Bazaar) that I used. I know that plenty of people get through by making manual backups and having clever naming conventions or similar, but I wouldn't want to do it any other way.

LaTeX

I'd used LaTeX before and knew it was pretty powerful, but I really enjoyed using it to write my thesis (and a few papers along the way). It is really versatile and, although it has plenty of quirks and idiosyncrasies, it can be moulded to do almost anything that you could want. There is also a wealth of information out there on the web to help with getting it to do what you want. Although it might seem like a steep learning curve at first, it's worth it in the end.

Referencing

I'm aware that there are more sophisticated web-based solutions around for reference management (Mendeley for example), but I found that my large folder of pdf files all referenced from a bibtex file and managed using cb2bib was a pretty good approach.

Figures

I developed an approach that allowed me to recompile everything in my thesis from the raw data. This was certainly a hassle at times, as writing a script to lay out a plot definitely takes more time than laying it out by hand, but overall I think it saved me time. Making changes late-on in the process I was able to do things like change colour schemes and a grid to all my figures relatively easily, without having to manually re-edit everything. Having said this, I might consider changing the software that I used for the job in future - see below.

What I might do differently

Not use Matlab for figures

Don't get me wrong, I'm a long time Matlab user and think it's great, but in trying to produce figures that looked as good as possible I feel like I really pushed it to its limits. What started out as some relatively basic plotting scripts, using mostly built-in functionality, evolved over the course of the work to be really quite complicated beasts. In some instances I had multiple sub-plots per figure, all using modified locations, with stacks of up to 4 axes sets in each plot (in order to achieve both axis breaks and different colour grids). These turned out fine in the end, but were quite a frustration to get right.

Since completing the work I have been increasingly producing my visualisations in javascript. This retains the scripted nature of the source code and the ability to regenerate plots as the data changes; it produces/exports vector graphics in a standard format; and it is increasingly common and well supported (both in terms of software and community) on the web, with libraries such as D3 enabling some really awesome stuff. But additionally I think it has the following advantages over Matlab plots:
  • It is an open language that is supported in all major web browsers and is therefore about a portable as you can get
  • It is less constrained than Matlab, as you have access to the raw line drawing commands.
  • It enables a much greater level of user interaction in the results, with tooltips, data selection, animation and linked plots all being achievable.
Although I would likely still use Matlab/Simulink for simulations, calculations and data analysis, I would write my results out in a format such as .csv and then plot the results using D3 in javascript. It should then be relatively straightforward to include the resulting .svg files in a LaTeX document.

Make my thesis more 'sectioned'

I struggled to get a consistent story that I wanted my thesis to tell until very late on in the process. Consequently the sections I wrote got juggled around and rewritten a few times. The techniques I used for writing it certainly helped in allowing me to play with the structure, but I feel like I could have done better in this regard. It's always easy to think like this in hindsight, but I would like to have had the content broken up in a more reusable manner. 

Unfortunately having reusable sections of content is almost the opposite of having the consistent and coherent narrative that a good thesis (allegedly) requires. I don't have a solution to this at present and I'm not sure I ever will, but it's certainly something I would think more about (if there were going to be) a second time around (which there won't).

Make my thesis more engaging

It upsets me that I put a great deal of effort into something that perhaps only 4 other people will actually ever read through in it's entirety. It also doesn't seem to me to be the best use of the funding that I received. Nearly 4 years ago I proselytised about future documentation methods, and I think we've since seen a lot of progress in this area, particularly on the web. I would love to have been able to present my thesis in a novel and engaging format, but unfortunately I would still be working on it now if I had tried to go down that route (and also still arguing with supervisors and examination boards about accepting it).

I think it needs someone who is well on top of their work to forge ahead in this area, and produce a truly dazzling thesis that sets a standard for others to copy. With the right approach I believe it should be possible to produce something with the technical depth and quality to merit the award of PhD, whilst still being accessible to the lay reader. Perhaps through staged levels of detail, or interactive facilities or similar. Once someone has proved that it can be done and it's "gone viral" or whatever, I feel like the overall approach will be replicable. Unfortunately changing the overall PhD thesis paradigm, whilst simultaneously studying for and obtaining a PhD, is probably a bit much for most students!

13/03/2014

LaTeX wordcount

I found out that I need one of these in my thesis recently and thought I would probably be a major PITA. However I then found TeXcount, and then found that it was included in my install by default. So typing
"texcount thesis.tex" at a command line gave me a load of interesting output.

A bit more playing and Googling showed me that I could run it during compile to automatically update in the document. I was using "\write18" anyway to pull images from other folders so this was no problem. In order to include the wordcount in the document I made a new command to do this. Mine is slightly simpler than in that link as I only need a total, so here it is:

% command to count words in document
\newcommand\wordcount{
\immediate\write18{texcount -inc -sum -0 -template={SUM} thesis.tex > count.txt}

(\input{count.txt}words)}

with that in my preamble I can then call \wordcount where I want it inserted in my document - easy.

Some more detail on the texcount options I'm using:
-inc includes all the "\include" files and "\input" files that I call in my thesis
-sum calculates the sum of all the words (including headers, captions, body text, etc)
-0 only prints the output, not all the details (although I'm not sure if it's necessary with the -template option)
-template={SUM} prints the output without any details of errors in the calculates

The output gets piped to a text file and then read back in again.

I should really find time to investigate the errors it's giving me and decide if there are any parts of the text that I don't want to count, but that can wait for now...

11/10/2013

Writing format / Style guide

As I write my thesis I keep coming up with questions about the style, layout and formatting that I should use. In many cases I doubt there is a right or wrong answer to the questions I'm asking, however I need to make a decision either way, and be consistent throughout my document. Therefore I've decided to start noting down the questions I'm coming up with and the direction I'm taking, ideally along with reasoned arguments as to why. Hopefully this should serve as a reference for me down the line.

Double spacing after full-stops?

I vividly remember being taught at school to double space after a full stop and single space after a comma, colon or semi-colon. This is a throwback to the days of using typewriters and it is widely regarded as no longer needed. Although it sounds like a slightly larger space can help with readability, modern proportional fonts and processing tools allow for this anyway. The Wikipedia page gives a lot of details on this.

I believe LaTeX can (and does) adjust spacing after a period anyway.

Title case?

What words should be capitalised in a title (e.g. a chapter/section/subsection heading)? "Title case" refers to the practice of capitalising all words in the title except for some subset of short or unimportant words. However what the subset is and whether title case should be used at all does not seem to be clearly defined. Once again the Wikipedia page kicks off discussion on this. It seems that several major publications use sentence case for titles. It seems that its main motivation is as form of emphasis, however it is argued that with the use of modern digital fonts emphasis is more easily added in other ways and that sentence case is easier to read.

Some contributors suggest using title case only for main titles, such as the title of the book and possibly chapters, but using sentence case for lower level headings. I will use title case in this manner, for the title of my PhD and for the chapter titles, but not elsewhere.

Capitalising references?

Throughout my document I make references to other sections and chapters, as well as to figures and tables. Of course I link these properly in my LaTeX source so that hyperlinks are formed, but should I capitalise the word section/chapter/figure/equation? It seems that most, but not all, scholarly articles choose to capitalise in these cases; the argument being, I think, over whether it behaves as a proper noun or not. This raises the question of whether "Page" should be capitalised in cross-references, this would appear very strange and this style does not appear to be in use anywhere. A possible explanation for this is that figures, chapters, etc are 'intentionally' named, whereas pages are not, although this seems somewhat unsatisfactory.

I will capitalise all references except for pages (which I do not expect to use) to fit with the most common useage.

Company, brand and product names?

There does not seem to be a common style in use for formatting these. Most commonly they are presented in standard body text with whatever capitalisation and punctuation the brand commonly uses. At most there may be some specific rules relating to capitalisation for certain publications. However in reading heavily technical work I find that confusion can often arise between technical terminology and product details, therefore it would make sense to be able to show the difference in some way. For example consider the sentences:
" The PID controller is simulated using a Matlab model. The ABB controller is simulated using a parametric model"
Here both "Matlab" and "ABB" are product and company names respectively, whereas "PID" is a technical acronym and "parametric" is a technical term explaining the simulation type. I doubt this distinction would be obvious to anyone unfamiliar with the terminology.

In order to assist with this I would like to apply some formatting convention to highlight company, brand or product names in the text. I have seen one dissertation that used small caps to denote products. This seems to work well as I am not using them for anything else in my document, although it is slightly unfortunate that acronym names do not stand out in this way. 

I am not fully decided on the best way to proceed with this at present.

Indenting paragraphs?

Separating paragraphs in some way to prompt a slight pause whilst reading seems like a sensible writing style. The most common technique for this in printed material is to indent the start of paragraphs, although alternatively an increase in line spacing may be used. As a pause is not required at the start of a new section (or rather, that pause has already been prompted) an indent is typically omitted.

This standard in online material seems to be to use "block text", that is no indents but an increased gap between paragraphs. This makes some sense as there is an infinite space available and there is no confusion over paragraphs breaking across pages

I will be indenting printed paragraphs other than the first and using block text in online materials.

Serif or sans-serif?

Although the jury seems to be out on whether serif fonts are actually any better for print reading, I went with convention and chose a serif font for my thesis.

Table format

As far as possible I followed the excellent recommendations in the booktabs usage guide.

Unit format

I'll be trying to stick with SI units wherever possible in the thesis, although there may be some deviations where other conventions are the industry standard. Where appropriate I'll be using prefixes to avoid long and ugly numbers. LaTeX provides the excellent siunitx package for typesetting units which I will be using heavily. When used in tables and figures to apply to an axis, row or column I will be using a slash to denote the units. This is recommended by the BIPM as the correct method of expressing values for multiple quantities. It effectively divides the title by the units, making the values dimensionless.
However, going against the BIPM advice, I will be including a space before and after the slash, as I feel that this separates it from the axis title - which may be multiple words, and it reduces confusion between the slash and the units themselves. With the slash present all the denominator parts of the units will be referred to as a power of -1.

Scientific notation

Due to the mathematics of my work covering a large number of physical domains, including thermal, mechanical and electrical, I ended up with a relatively large notation section. I also began to run out of letters in the latin and greek alphabets. I therefore ended up using a cursive script to denote thermal parameters, allowing me to reuse some letters.

Figure design

As far as possible I followed the guidance of Tufte in presenting data in my thesis.

Punctuation

http://tex.stackexchange.com/questions/55246/how-to-properly-typeset-all-forms-of-punctuation-used-in-english-language-docume/55911#55911

Acronyms


Line spacing


Margins

This seems to be the subject of as much debate as anything else in typography and formatting! Things I think are worth considering: binding, screen reading, ease of reading, general prettiness. By default LaTeX gives you a larger margin on the outside when you request twosided document. This is to maintain the same distance between text in the centre as it does at the edge. It does not make any allowance for binding.

The "classicthesis" LaTeX style gives really quite a narrow column of text. This style emulates Bringhursts The Elements of Typographic Style, which seems to be regarded by many as the definitive guide to the subject. The classicthesis documentation has this to say:
"The size of the text body is intentionally shaped like it is. It supports both legibility and allows a reasonable amount of information to be on a page. And, no: the lines are not too short."

None of this addressed binding, which seems to be considered separately, however must be allowed for in thesis printing. University guidelines have this to say: "To allow for binding the margin at the binding edge of any page must be not less than 40mm; other margins must be not less than 15mm.".

A further consideration is the fact that the thesis will frequently be viewed in digital format. In this case margins are not necessarily required, and variation in margins between odd and even pages is likely to be distracting. This suggests that specific formatting for two sided printing should be avoided.

As a compromise I will be using equal margins either side of the text. When bound this will produce a narrower inside margin. I will also be placing page numbering at the centre of the page and titles always on the left, this should maintain uniform readability for screen reading. I will use 40mm for both margins, as this is the minimum allowed for the binding margin by the guidelines, this also achieves a 0.62 proportion, not far from the 2/3 often advocated.

The top and bottom margins are allowed to go as low as 15mm, and at this limit the document scrolls very nicely during screen reading. However, it looks unwieldy in printed form. Therefore I've opted for a 25mm margin at the top and a 35mm margin at the bottom, with page numbering placed in the centre of the bottom margin.

Punctuating equations

Punctuate as if they were part of the sentence.
or don't as it may confuse the reader?
Personally I agree with this answer, therefore I shall punctuate equations that are displayed inline, or where the sentence continues below the equation, but I won't use periods to end a sentence within a numbered equation; in this instance I shall try to use a colon prior to the equation.

Related: should notation be introduced in the text flow or in brackets? I introduce them without brackets.

Font

I ended up using LaTeX's default Times font rather than anything too fancy. This was easy and familiar to me and most readers.


Justification

Most technical articles tend to use fully justified text and there is some evidence that this is of benefit to dyslexics. One downside is the increased tendency for justified text to create distracting "rivers" of white space. One improvement is the use of microtypography techniques. These are pretty easy to use in LaTeX and there doesn't seem to be any downsides, so I'll be using them.

Tables and Figures

Tables and figures will appear throughout as soon a practically possible after they are referenced in the text. They will all be centred with a centred caption, Tables will have a caption above, Figures below. Captions will not be punctuated with a final period if they are only a sentence fragment, however they will be if they are a complete sentence or there are multiple sentences. Figures will not have a title above them, the caption below providing all the required information.


This is obviously a work in progress post... but if anyone has comments then I'd love to hear them!

01/08/2013

LaTeX equations split over two columns

Drafting a paper I've just got stuck trying to fit some lengthy equations into a two column document. For some I've gotten away with splitting lines using \begin{split} or \begin{align} environments, but for some it starts to look ridiculous. I also started to run into issues where my \left( and \right) bracket commands didn't like to be split over lines. So I decided that there was no option except spreading the equation over the full page width.

This is actually not particularly straightforward in LaTeX. A reasonable forum discussion on options gives a few suggestions. It seems that it needs to be floated within a figure environment. This is also suggested in a very comprehensive document (p35) on typesetting equations. Unfortunately both of these require manual fiddling with equation numbers and quite a lot of extra code in the document. I've therefore modified their code to define my own environment for a floated equation and used counters to automate the equation numbering.

Preamble:
% for floated 2 column equations
\newcounter{tempEquationCounter}
\newcounter{thisEquationNumber}
\newenvironment{floatEq}
{\setcounter{thisEquationNumber}{\value{equation}}\addtocounter{equation}{1}% record equation as happened and remember number
\begin{figure*}[!t]% float following equation across columns
\normalsize\setcounter{tempEquationCounter}{\value{equation}}% record current equation number in floated location
\setcounter{equation}{\value{thisEquationNumber}}% use previous equation number
}
{\setcounter{equation}{\value{tempEquationCounter}}% set back to equation number in floated location
\hrulefill\vspace*{4pt}% add a horizontal rule separator
\end{figure*}% end float environment

}

In the text simply wrap the standard equation environment in the new environment:
\begin{floatEq}
\begin{equation}
a = b + c
\label{equ:floatedEquation}
\end{equation}
\end{floatEq}

This should float the equation across the whole page and allow LaTeX to fit it in the right place whilst still numbering it as if it appeared where it is defined in the text. Or at least that's what it's achieving for me at the moment!

30/05/2013

More thoughts on thesis documentation...

It's over a year since I put together some thoughts about how documents (PhD theses in particular) get written up. Having just started on the third year of my PhD I need to start compiling all the little bits I've written in the last two years together into something resembling a final thesis format. I can then build this up into a body of work that I can present at the end. This raises the question - how to best go about this process?
First a few conflicting thoughts about the final document I need to produce:
  1. I'd like to write in short concise sections that will either stand on their own, or can be built up into a larger document. I'm concerned that this will not produce a coherent final document though.
  2. I'd like to setup the individual sections in a hierarchy that reflects the hierarchy of the systems they are describing. Unfortunately this is likely to produce far too many levels in the document (when I drew up a draft layout I got to 12 levels!).
  3. I'd like to describe the work down to a level that anyone (even a schoolchild) could understand, however this would probably end up being a massive document that would be too basic for most readers.
I think the problem I have, that lies behind all of this, is that the thesis is really designed to be a document for your examiners to read, cover to cover, in a format that they are familiar and happy with. It will appear printed on paper, be long and probably dull, and, as has been pointed out to me by other people and stated a number of times on the internet, it should "tell a story". A story that took 3 years to write and that a maximum of 3 people will ever actually read.

What I would prefer to write is a document that I could upload to the web and could be referenced and useful to anyone. I've spent some time thinking about how I might be able to cleverly combine the two, but I don't think I'll be able to write just one document to satisfy both aims. So my plan now is to write the thesis in a fairly conventional manner in order to satisfy the requirements for a  PhD, then to later revise this into a format that I prefer. Obviously this second part will be easier if I know what I'm planning during the first part, so here is my initial outline...

Thesis mark I

I'll be using LaTeX for the writing, in whatever document template I'm given (or have to make to fulfill my departments guidelines). I'll try to keep it split into relatively small chunks, but without going to too many levels of subheading (4 max). I plan to keep each chapter in its own subdirectory and include it using the \include command. Within chapters I will break things down again using \input commands to allow me to keep sections and subsections in their own individual tex files.
I'll try to use hyperlinks where possible using \usepackage{hyperref}, using the colorlinks option to avoid the default ugly boxes. By using a subtle colour for the links they shouldn't be too jarring in a printed report (where they will be no use) but obvious enough when read on a computer.
As previously discussed in this blog, I'll try to include figures from matlab and simulink as vector images, stored wherever they are generated. I'll also use a single bibtex reference file for all my references.
As with other work, ongoing revision of the thesis will be controlled using Bazaar. I can include the version used to produce output files within them using vc.tex, which is pretty neat. That way when I have multiple versions printed out and sent to people for comments I hopefully shouldn't lose track of which version they're commenting on!
Latex to Kindle?

Thesis mark II

Latex to html?
html to chd?

I never really finished writing this post, but it gives a flavour of what I was thinking a little while ago when I started writing in earnest!

20/08/2012

TeXworks shortcuts

I use TeXworks as my LaTeX editor currently and I recently discovered a few shortcuts that I'm finding really handy in the preview window:

Shift + click = Zoom in
Alt + click = Zoom out
Ctrl + click = Find paragraph in source code

Hope those few are helpful to someone. Let me know if there are any others that I might find useful!

02/08/2012

Adding bzr files through the cb2bib

I recently described my referencing process where I simultaneously hold reference pdf files in a version control system and keep track of their details in bibtex file. I have also described a problem I've been having with my version control system of choice, Bazaar.

Based on these I decided that there might be a better (more automated) way of adding my references. After a very useful email exchange with the author of cb2bib they confirmed that this should be possible. Now, after a day of messing around and learning about various command-line tools that I hadn't used before, I think I've got it working. Here's how...

First I created a new batch file that I called "bzrAddRef.bat". Here are its contents:


@echo OFF

rem --------------------------------------------------------------------------
rem cb2Bib Tools addition by J. Welford
rem --------------------------------------------------------------------------


echo   bzrAddRef:
echo   cb2Bib script for adding BibTeX files to a Bazaar repository
echo.
echo   Using sed and xargs utilities from:
echo   http://gnuwin32.sourceforge.net
echo.
echo   Path below may need changing to be within the repository
echo.


echo   Processing:
echo.
cd "C:\Users\welf\Documents\literatureReview"
sed -n -e "s|.*file.*=.*{\(.*\)}.*|\"\1\"|p" %1 > bzrRefs.tmp
xargs bzr add <bzrRefs.tmp
del bzrRefs.tmp

Only the last 4 lines are really important. The first one sets the working directory, I don't think it matters what you use as long as you have write access and it is within the repository you want to add to. The next line runs through the current bibtex file and extracts the location of all the references to a temporary file. The next line adds all these files to Bazaar repository. The final line deletes the temporary file. (maybe I could have used a pipe between commands so that the temporary file was not required?)


Some special commands are used that will need to be installed, what you need are: sed.exe (and its dependencies:  regex2.dll, libintl3.dll, libiconv2.dll) and xargs.exe.

Within the settings of cb2bib this batch file can now be pointed at under the "Configure BibTeX" - "External BibTeX Postprocessing" - "Command:" section. Once that is done simply hitting "Alt"+"p" in the cb2bib window should run the batch file and add all the references to version control.

I hope the helps someone! (I presume it could be altered for other version control systems or be called from other applications.)

31/07/2012

My referencing process

I suspect everyone has a subtly different way of doing this, depending on the tools they prefer and the way they like to work. I thought I would document my process as I think it's pretty efficient and has some real advantages, I'm sure there is still plenty of room for improvement though.

Overall aim

As an academic I often need to refer to work done by others, this normally done by noting their published work as a list of references or bibliography at the end of my documents. The most basic way of achieving this would be to have a stack of published books and papers in my drawer that I can refer to and then type in the reference details at the end of the each document I want to write. I'm sure that plenty of people do work this way, but from my point of view I see it as pretty inefficient (lots of printing, lots of sorting, lots of typing, not very portable, an awful pain to change reference styles, etc).

I manage my references entirely in software using a few specific tools. I've mentioned most of them in other posts but I'll go into a bit more detail here on the actual process I use. 

I'm going to split referencing into two separate processes. Firstly, as I'm researching a topic, I tend to gather references to get an idea what I'm doing. Secondly, when I come to document my work I need to search and cite the references I've found.

Tools

Google Scholar is generally my primary resource for finding papers and documents. I also rely on standard Google searches a massive amount and I have a Google alert set up to email me when a few key phrases appear in new articles added to the web.
IEEExplore generally has most of the published work (in terms of papers, Journal articles, etc) that I need to refer to. The University has a subscription to this that allows me to download what I need (otherwise I'd have to pay!).
Pdf format is generally how almost all papers are delivered and the format that I keep them in. I use Foxit reader to read pdf documents. I keep all my reference documents in one big folder rather than worrying about any kind of complex filing system.
I use Bazaar to version control my folder of pdf documents. I've previously discussed how this works and how it allows me to work between different computers, even without installing any software on them.
I use a tool called cb2bib to maintain a list of references in bibtex format. I started off just using this to add references to my bibtex file, but I've found it to also be really good for browsing references and citing whilst writing a document. I changed some of the default setup to help it retrieve data from the net.
I use Latex to typeset most of my work. Within this I can simply point it at the bibtex file for all the details of the references. I have previously mentioned how to use a bibtex file that is not in the same place as the rest latex document.

Gathering references

  1. Search for the document that I need to find using Google, Google Scholar or general web browsing.
  2. Open and read the document to see if it looks relevant and useful. Assuming that it does...
  3. Download the document to my big folder of "third party" references. I tend to use the full title of the work as the save name of the document - this can lead to quite long file names, but it makes it a lot easier to find things!
  4. Add the saved file to my bazaar version control system. This only takes a couple of clicks through  tortoiseBzr menus in windows explorer.
  5. Add the document to my bibtex file using cb2bib. This is a pretty straightforward process:
    1. Open cb2bib, I have a keyboard shortcut setup for this (it should also remember what bibtex file is being used)
    2. Click "import from pdf file"
    3. Click "select files"
    4. Select pdf files saved previously (hold ctrl to select a bunch at once)
    5. Click "process"
    6. The software will try to extract as much info as possible from the pdf file, this probably won't be enough so...
    7. Click Network query to retrieve all the info about the file from the web (this usually works fine, but it's worth checking the results, you may need to give it the right title to start it off)
    8. Click save to add the reference to the bibtex file
  6. The changed bibtex file and added references will need to be "commited" to the version control repository.
UPDATE: I have added now made a batch file that effectively takes the place of step 4 and occurs after step 5 - it can be run after a whole set of references have been input through cd2bib and adds them all to my version control repository using "Alt"+"p".

Citing references

  1. With a Latex document that has a pointer to the bibtex file within it.
  2. Open cb2bib citer (I use a keyboard shortcut) and select the reference(s) required:
    1. [optional] Select the way I want my reference list displayed by pressing "a" (author), "j" (journal), "t" (title) or "y" (year). I find author is usually best.
    2. [optional] Filter the reference list by pressing "f" and then typing what you want to search for ("d" clears the search)
    3. Click on a chosen reference
    4. [optional] Press "o" to open the reference and read it
    5. Press "enter" to cite the reference, a small pen marker will appear next to it (in author view it will appear next to each author for the same paper). Multiple references may be cited by selecting them and pressing "enter". "delete" clears all the selected references.
    6. Press "c" once all the references for citing are selected, this will close the citer window and copy the latex text for the references to the clipboard.
  3. Paste the text into the latex file to include the references at that point in the document.
Although those might seem like a lot of steps it's really pretty straightforward once you get used to it. The only real difficulty is remembering the keyboard commands for cb2bib citer. With this setup I can take all my references with me between machines (even between Linux and Windows) and use the same process everywhere.

Bazaar GUI issue on Windows 7 - adding files using the commandline GUI interface

Due to an unfortunate hard drive issue I recently had cause to swap to a new drive. Along with this came an upgrade from XP to Windows7 (64bit). This was not a real issue, just a bit of hassle moving everything over and re-installing all my applications.

As part of the process I had to re-install Bazaar which I'm using to version control all my work. This appeared to go fine, including the addition of tortoiseBzr to integrate the Bazaar commands into the windows explorer GUI. Unfortunately when I came to use it with my work it gave me an error, I think due to the "special" folders structure in windows7. Here is a copy of the message I posted to the Bazaar user group:

Hi everyone, I hope someone can help me out.

I'm using Bazaar to version control a whole set of different files between a few different machines. On a previous windows XP machine I was able to use the whole user area as my repository and only add the files that I wanted to control.

I have just upgraded to a windows 7 machine (64bit) and there seem to be some issues with the preconfigured folders not being accessible by Bazaar. I have been able to commit files fine, but when I try to open a window to add new files it crashes as it tries to display them, with the error:

bzr: ERROR: [Error 5] Access is denied: u'C:/Users/welf/AppData/Local/Application Data\\*.*'
(It also seems to have issues with "My Music" and similar)

Any thoughts on why it's happening or a way round the problem?

(I suspect that it would work if I had the repository at a lower level without any "windows" folders, but that is not really the way I'd like to work. I don't actually need to version control anything in these folders, so if they can be skipped in some way that would be fine.)

It seems suspiciously similar to an outstanding bug here:
https://bugs.launchpad.net/qbzr/+bug/1012907

The general upshot is that I can't add new files to version control using the GUI, although I seem to be able to do pretty much everything else. I thought I would detail the workaround I'm currently using. It isn't too bad, but wasn't totally obvious, so I thought I would document it in case it was of use to anyone else:
  1. Right click in the folder containing the files and select the "Tortoise Bazaar" menu item, then select "Run command". A GUI window to allow you to run a command will open.
  1. Select "Core" after "Category"
  2. Select "add" after "Command"
  3. Click "Insert filenames...", a file selecting window will open.
  4. Select the files you wish to add (hold 'Ctrl' to select multiple files) and click "Open"
  1. Click "OK" and the status window should tell you that the files have been added
You're now at the same stage as if you'd added the files using the standard GUI. Of course you will still need to perform a commit before the files are part of the repository.

Hope that's useful to someone.


UPDATE: Discussion on a Bazaar email list confirms that the problem I'm seeing is due to the error linked above. Unfortunately a fix does not look imminent therefore I'll have to continue using the command-line method described.

23/03/2012

Bazaar revert only one file to previous version

So this was the situation I was recently in:

  • I found that a simulation I was running in Simulink was giving me results that seemed to be wrong
  • I was sure that previously it had given me the right results
  • Therefore I wanted to "roll-back" to the previous version of that Simulink model

I thought this would be a simple and easy thing to do as I have all my files under version control. This sort of procedure is termed "reverting" in version control lingo. It was certainly possible, however it wasn't quite as easy as I expected, so I'll detail the process required.

As noted previously I'm using Bazaar for version control, through both the TortoiseBzr windows interface and also the Bazaar Explorer GUI. Bazaar supports several different methods of undoing mistakes. Through either method of accessing Bazaar you can get at a log of all the commits done on a certain file. From this window you have the option of either "Revert to this revision" or "Update to this revision". Either of these will give you the previous version (although the first one will not count the changes as committed); however they will give you whole of the repository at that revision. This might be useful for pulling out a whole set of software that works together, or establishing the state of an entire project at a particular point, but this is not what I needed.

It is possible to get at an individual file, but it involves a little bit more work. It may also be worth ensuring that the current version is committed in case you later decide that reverting was a mistake. Here is the process:
  1. Look at the log of file to determine what revision number contains the version you want (this is where having written good commit messages will help!)
  2. Make a note of the revision number that you require
  3. Highlight the address of the folder you are in and copy it
  4. Open a command line (on windows go to start - run... - "cmd")
  5. Change to the directory you require by typing "cd " and then pasting the folder address you copied before
  6. Type in the Bazaar revert command as follows:
bzr revert -r num file.ext
where:
num is the revision number that you require (as noted from the log)
file.ext is the filename and extension that you wish to revert to (this could be a list of files if you prefer)

This should give you the previous version of only that file. It will not be committed so you can check that it is the right version and make any changes you want before recommitting.



Matlab figures in Powerpoint

I've just been putting together a presentation in which I wanted to include a set of Matlab plots. Whilst there are plenty of ways to do this I thought I would detail the way I managed it...

I've mentioned before about maintaining quality by using vector graphics. Previously I've used .eps format for this, but I don't think it's well supported in Powerpoint (I did consider using alternative presentation software but I was pressed for time and this seemed like the easiest option). Instead I used .emf format which seems pretty well supported by Matlab. This is a Windows specific format so I can't say I'm particularly keen on it for general use, but it served a purpose here.

I saved my figures as separate images from Matlab and then imported into Powerpoint, but I think the same result can be achieved if you simply copy and paste between the two.

One of the benefits of having the plots in vector format is the ability to manipulate them within Powerpoint itself. For the unscrupulous this could be a good technique for "adjusting" a few points here and there to improve  your results - but obviously I didn't do that! What I did do was separate out several lines that were on the same plot, so that I could make them appear (using Powerpoint's animation features) gradually. This made the plots (which were a comparison of the results of several different simulations) much cleaner.

It was a little bit of a hassle to do this, but not awful, and I think worthwhile as it made the results much easier to present. When the .emf file is initially imported it is handled as an image; however, if you right click on it you will find that it can be "ungrouped". You might have to repeat this process a few times as the image seems to be built up of several sub-groups, but once you're done you'll find that individual lines are available to be manipulated. Unfortunately I then had to go through and "regroup" some of them manually to reform complete lines, which involved a bit of 'Ctrl' clicking - this was the tedious bit. I guess it depends how complicated the lines in your plots are as to how long this will take (and whether it is worthwhile or not).

Another benefit was being able to alter the font of my axes to match my presentation style. A trivial point perhaps, but it all goes to improve the overall appearance.

So to summarise:

  1. Copy figure in Matlab
  2. Paste in Powerpoint
  3. Right click and "ungroup"
  4. Repeat 3. until you get to a  level that you can manipulate the objects that you want
  5. You may find there are a few large white squares that form the background of the plot - these can be deleted (it will make things easier when you come to regrouping things in step 7.)
  6. 'ctrl'+'a' to select everything and then change the font to match your presentation
  7. 'ctrl' and click on objects to collect all the ones that should be in a group (you may need to zoom in on your slides to do this accurately)
  8. right click and "group" them
  9. move/recolour/animate/resize the objects as you desire to suit your presentation!

Hope that is of help to someone!?

13/02/2012

IET conference paper reference style

I've recently prepared a paper for a forthcoming IET conference in which I cited references.
Whilst the template for the paper provided an example of the format it would like references to journals to be included in, it gave no further advice.

Assuming that there would be a standard format for all IET conference papers I went a-googling - with very little success. My next option was to ask on the IET forums here. Unfortunately this didn't turn up any standard formats either. Therefore my conclusion is that a "standard" doesn't exit.

This isn't a major issue as I think you'd have to work quite hard to contrive an example where a reader wouldn't be able to decipher a reference due to the use of an obtuse format. However I think it would be good practice to document the format I've used here...

The conference template suggests (for a journal article):
[1]    A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110-120, (2000).

Surprisingly this differs from the IET advice for research journal citations and the two documents on IEEE reference formats.

Therefore the style I decided to use is:

Journal article
  •  A. B. Author, C. D. Author. “Title of the article”, The Journal, volume, pp. 110 – 120, (year).
Conference paper
  •  A. B. Author, C. D. Author. “The title of the conference paper”, The Conference, Location City, Location Country, pp. 1-7, (Month year).
Book, book chapter and manual
  • A. B. Author, C. D. Author. “The title of the book chapter”, in D. Editor (Ed.), The title of the book (Publisher, year, 1st edn.), pp. 1 – 7.
Book
           A. B. Author, C. D. Author. The title of the book (Publisher, year, 2nd edn. 2006).
Patent
  • A. B. Author.  “The title of the patent”, British Patent 123456, (Month year).
  • A. B. Author, C. D. Author. British Patent Application 98765, (Month year).
Thesis
  • A. B. Author.  “The title of the thesis”, PhD thesis, XYZ University, year.
Standard
  • BS1234:  “The title of the standard”, year.
Website
  • http://www.theiet.org, accessed Month year.

As I prepared the paper using LaTeX, all my referencing was done using BibTeX. I therefore needed to create a custom bibliography style file. I did this using the custom-bib package. This a fairly straightforward process once it's figured out, roughly consisting of:
  1. run latex on "username"\Application Data\MikTeX\2.8\tex\latex\custom-bib\makebst.tex
  2. answer questions appearing in the output window
  3. get *.dbj file produced at the end
  4. run latex on that file to produce *.bst file (I found that the full path to the file was required at this step, so it needed to be run from the command line rather than through an editor)
There is a fairly large set of questions that allow you to customise the style through this process, so I ended up with a style that was pretty close to what I wanted. Unfortunately there were a few niggling differences in the output. Most significantly custom-bib does not allow the option of changing the comma between the author list and title to a period. To rectify this I had to dive into the .bst file and start hacking!

Now I wouldn't recommend this if it can be avoided as the language used is pretty obscure, however I eventually got something that looked about right.

If you want to use my style file it's here. Obviously it comes with no warranty and no official backing from the IET. It looked ok for the paper I submitted but I'd recommend thorough checking of the output if you do use it.

Please let me know if this is of use to you, or if you have any comments on my chosen style, using custom-bib or any other part of my process.

26/10/2011

Weather in iGoogle

I've been using iGoogle for a while now. For those not in the know, this personalises the Google homepage and allows you to add "gadgets". Since I've started using Chrome as my browser I tend to Google search from the address bar anyway, but I still find iGoogle pretty handy.

I use it as a way to amass all the stuff I tend to look at in one session. Currently I have three different tabs:

  • News - Local and world affairs and specific technology news
  • Comics - Webcomics that I like to read
  • Weather - Local weather forecasts so I know if I need a coat
These are all areas that I tend to want to update myself on regularly - approximately once a day. So instead of trawling through various bookmarked pages I can accumulate all the info in one convenient page.

This works out really well and I've found there are specific gadgets for most of the pages the I want to visit. Some of the news related ones didn't have gadgets but I was able to use use their RSS feed to display them in an RSS reader gadget.

The only real pain was the some of the weather forecast sites that I wanted to look at didn't have gadgets - most notably Metcheck.com. I really like the way they display the forecast so I set about trying to get it onto an iGoogle page. The 'official' way to do this I guess would be to make a specific gadget; however this seemed like it would be a lot of effort and learning time. So I went with a different option.

I used a gadget that allows the inclusion of html. Then I wrote some HTML to include a portion of another website. This allowed me to trim the site down to just the specific window I wanted. For example:
<style type="text/css"> #container{ width: 0px; height: 0px;} #container iframe {width: 780px;   height: 590px;   margin: -180px 0px 0px -240px; }   </style> <div id="container"> <iframe scrolling="no" width="0" height="0" src="https://www.netweather.tv/index.cgi?action=uk7dayx7;page=1;ct=19121~Manchester;sess=#forecast></iframe></div>

This includes the portion of netweather.tv for my specific region. I was previously using metcheck but have recently switched as their results page seems to regularly give errors at the moment.

There might be an easier way of doing things, and for other websites the numbers in the HTML will need fiddling with so that the right portion of the page is in the window, but this technique seems to work pretty well for now.

28/09/2011

Future documentation methods

Coming towards the end of the first year of my PhD and spending some time writing up my progress so far has led me to muse over the nature of report writing and ask the question: "surely there must be something better"...

Whilst the old adage "if it ain't broke don't fix it" might well apply here, I can't help but think that in the 21st century of immersive 3D virtual reality game playing, home 3D printing, and everyone carrying at least one state of the art electronic device about their person at all times, the concept of a paper report seems a little dated.

Here are a few examples to try to illustrate my point:
  • If there is a book and a film of the same story (or even a webpage and a youtube video) I will inevitably look at the the film first as it will convey the information to me far faster, and with less effort than reading the book.
  • If I have a choice between a photo or drawing of an object and a 3D model (either manipulable on screen or available to touch) I would get a better understanding of it from the 3D.
  • If I want information on a specific subject then I turn to Google/Wikipedia before I head off to the library.
I don't think these are examples of me being weird, they are simply illustrations of modern life making information more readily accessible. I'm sure you could argue over semantics ("the book contains more detail than the film", "library books have a more systematic review process than google hits", etc) but I hope you can accept my general point.

It therefore seems strange to me that a piece of work, perhaps costing thousands of pounds and many hundreds of man hours, should be presented in such a one dimensional format as a printed report. Here is a summary of what I see as the limitations to a printed report:
  1. One dimensionality - sure pictures might take this up to 2D, but all too often there aren't enough of these!
  2. Lack of user interaction - I can't search for a keyword, interactively link to source, or request further detail on a topic.
  3. Page constrained format - diagrams need to fit within a certain width, zooming in is limited by your eyes and printer resolution and page breaks artificially chop things up.
  4. Visual sense only - My other senses are put on hold, and only serve as a distraction.
So what have people done to improve on this? Here're a few examples that I can think of:
  • Video - a good recent example is this guys youtube CV
  • Hyperlinking - within sections of a document or out to other documents or web sites
  • Wiki formats - taking linking between sections to the extreme and making progress through the information less linear
  • 3D graphics - starting to be seen more in web pages, an excellent example is Google body
  • Powerpoint - a format often used in place of a standard document, it has many of the same issues, however users often seem to feel a little less constrained in terms of layout (perhaps this is only due to convention?)
  • Computable document format - this is a really exciting new development that reflects a lot of what I'm describing here
This last concept may or may not take off but I can see what they are hoping it will achieve. Some of its functionality can already be achieved in a pdf (details of how to achieve a lot of them through LaTeX are here) and almost all of it could also be done through HTML and javascript. An interesting discussion on this is given here. In fact the recently developed HTML5, in combination with javascript programming, offers a whole mass of interesting possibilities for the presentation of information. A step towards using HTML5 for what I'm talking about here is Tangle. This is a javascript library that supports the production of "reactive documents", allowing a reader to play with the content of the document.

Another alternative format with a lot of capability is Flash animation, these animations are typically web-based and often allow user interaction. Some basic options for creating these are given here. Although it is a very widely used format it requires a good level of experience to be able to code it. It has also faced quite widespread criticism recently, the most high profile of which came from Apple, and therefore there is speculation about whether HTML5 will ultimately replace it.

An obvious downside to these types of advanced documentation method is the length of time it takes to actually produce a document. Even when the author has a good knowledge of the specific tool they're using I think it's safe to say that nothing I've mentioned above will be as quick to produce as a simple text document. In fact the more advanced the documentation method - the longer it's likely to take to produce.

I'd love to be able to round off with a recommendation of the ultimate tool or combination of tools that can be used to create the perfect document, but as far as I've seen it doesn't yet exist. Lots of things seem to offer at least part of the solution I'm looking for, but none pull it all together into one great package. So instead I'll do two things, firstly I'll make a few plain points in summary/prediction, then I'll put together a set of use cases that I'd like to see available to the end user of my "ultimate document".

Summary/Predictions

  • The plain printed word document is currently in the process of being overtaken by more electronic forms of documentation, inherently bringing a lot more potential to the document itself (hyperlinking and embedded video being two major ones). I would expect this to be a continuing trend (that may eventually even reach formal engineering reporting or even academia!).
  • There is the potential for this to go a lot further than the type of electronic documents seen today with the addition of 3D effects, audio tracks and similar.
  • HTML5 currently seems to offer the most potential for supporting this type of advanced documentation (although the computable document format may also be a candidate if it manages to pickup much of a user base).
  • Very little progress towards this end goal will be achieved until there are good tools for authoring the type of document I'm discussing here.
  • It seems highly likely that viewing of any document of this type will be through a web browser or similar.


Use cases - scenarios that I, as an end 'reader', would like to see supported in this ultimate document format.

  1. User managed detail level - I'd like to be able to look in more detail at sections I'm interested in or know little about, whilst invisibly skipping over the mundane or tedious stuff.
  2. Unconstrained document flow - If I want to read summary, then the contents, then the conclusions, then methods, it should be easy for me to work through that way.
  3. Recommended document flow - If I simply want to be guided through the document ensuring that I pick up all the important information then this should also be easy
  4. User interaction - Where more information could be made available then I should be able to access it. For example I should be able to zoom in on a waveform or rotate a 3D model.
  5. Multiple sense stimulation - practically this is likely to be limited to visual and audio currently (at least until we develop smell-o-vision and feel-o-vision...)
  6. Portability - I want this document to be viewable in as many places as possible, consequently it must be compact and easily openable on a variety of devices (laptops, mobiles, touchpads, e-readers, etc). This might even extend to alternative language/disability support and (somewhat ironically) the ability to print onto plain old paper.


So what have I missed? I'd love to discuss this topic and related areas more so please leave me a comment.
I'd also love to have the time, skills and supervisor buy-in to trying to present my thesis in the manner I've outlined; however I suspect that that will remain a pipe-dream...

18/08/2011

Creating Gantt charts

I've had this issue before and it annoyed me. It's still annoying me now!
Gantt charts are one of the types of chart that I have come to accept as a part of life that's not going away, and actually they're not all that bad. They are however difficult to draw.

I (like plenty of other people I've seen) have tried kludging together a Gantt chart in an Excel spreadsheet and it always seems to come out looking pretty horrific. I've also seen them drawn freehand in paint or Powerpoint or similar. Obviously these aren't really a proper solution, so I had a look round to try to find something that would let me manage a simple Gantt chart to plan and track my PhD progress.

Two options looked promising:
Gnome Planner - part of the gnome desktop but also usable under Windows
GanttProject - another open source cross platform piece of software

I started out with the latter and then switched to the former, but to be honest I don't find either that useful. I think the primary issue is that I'm not looking for a tool to perform PERT or CPM or do resource levelling for me. All I want is something that will accept all of my task details in a well contained format and produce a vector graphic of the chart that I can include in a LaTeX document. It's this last part where both of these tools fall down.

The output of these tools is much more of a formal report than it is a nice image. They also seem to really struggle with the multi-year duration of my work. With planner I've had to resort to taking screenshots of the window and including those in reports!

There are packages for LaTeX that support drawing of Gantt charts, but they're a bit too fiddly for the kind of quick changes that I'd like to be able to make. As Planner saves in XML format there must be a method of auto-converting to the LaTeX code...

Any solutions or alternatives out there that I'm missing?

UPDATE: In the end I went with drawing it out using one of the Latex packages. This used all the same data that the planner XML file had in it but in a different format - I was sorely tempted to put together a script to do the conversion for me but in the end I couldn't justify the time required and did it manually. I also had to meddle around a bit to get the gantt to fit on an A4 page - in the end I made it landscape and adjusted the page margins. I think the output from Latex looked a lot more professional and it would be really useful to have a conversion tool. If there is any interest in how I put the latex code together then leave a comment and I'll write a bit more about it...

16/08/2011

Using a remote Bibtex file

As mentioned previously:
  1. I am using version control software to backup my work
  2. I'm using LaTeX
  3. I don't like duplicating files around my system
  4. I'm working between both Windows and Linux
So I've found myself today writing an end of year report (housed within its own folder in my filesystem) and wanting to pull in some references from my bibtex file (housed within the literature review folder in my filesystem). I could use the copying hack that I previously devised for figures, but that wasn't very elegant.

There are plenty of questions about this out there in internetland, mostly with the following suggestions:
  • Move your bibtex file to within the tex installation root - I can't really do this because of 1. above.
  • Use symbolic links to the original in the local folder - I can't do this becuase of 4. (windows doesn't support them)
  • Give a relative filename reference e.g. '\bibiography{../literatureReview/references}'
  • Adding the location as an environment variable as described here.
Both of these later two methods seemed to work for me. I went with the relative referencing for portability between systems. Odd that such a straightforward solution works for references but not for figures!?

27/07/2011

Making a bibtex file from a folder of pdf files

The issue
As I'm going to be writing some big documents with lots of references I'd be a fool to try to manage these manually, I therefore needed to pick a reference management piece of software. After some browsing I settled on JabRef because: it's free, it's open source, it's lightweight, it's cross-platform and it handles bibtex format natively (which is what I need for it to integrate with latex). It should also link nicely into the Sciplore mind mapping software which I'm using (more about that some other time).

JabRef is basically a database management tool for references that stores its database in bibtex format. It  looks like it will work rather well, but unfortunately my first stumbling block is that I already have a folder full of my references in pdf format (~200). This means that I'm immediately faced with the big task of going through and adding the details of each pdf individually. There must be a better way...

Someone else asked the same question here. The answer seemed to be that there was no easy way in JabRef, but it could be done in some other reference management software - such as Mendeley. So I could install that as well and export from there to use JabRef, that seemed like a pain though, especially as you need log in details and all sorts for Mendeley.

The solution
Somewhere else cb2Bib was suggested. This looks like an awesome piece of software, almost to the point that I could use it instead of JabRef, although I don't think it does quite the same job. It's designed as a bibtex database manager, however it is more tailored towards reference entry than editing or final use (e.g. citations) - although it can do this. Its method of adding a new reference is based on what's currently in the clipboard - thats whatever you most recently 'cut' or 'copied' in your operating system. This can either be a piece of text or a pdf file.

Files from the system can also be queued up to be added to the clipboard for addition to the bibtex database - in this manner a folders worth of pdf files can be added. Once the file is in the clipboard the software interrogates it to try to extract the right details for the bibtex reference entry. It is also able to do some other clever things like search the web and find a web reference for it that matches only one of the pieces of data it has extracted. There is also the option to manually edit the fields or to set off a whole run of files to add automatically.

My implementation
In practice the software took a little while to get used to; the buttons aren't in quite the locations I'd expect, there seem to be about 3 different windows that are independent but interrelated and the method of specifying a bibtex file and then successively saving additions to it felt a little odd (rather than running through to create a file and then saving it all at once). But once I was used to it at that level it all worked.

When I came to actually try to add all of my pre-saved pdfs however, I hit problems. Whilst automatic extraction usually managed to pull out a few nuggets of useful data, it rarely found enough for a complete entry. Hitting the button to search the web didn't seem to give much assistance. So it was time to dig a little deeper.

Probing through the website there is quite a lot of useful information on how to configure the software to do what you want. What I needed to do was look into where was being searched on the web for my articles. This is all setup in a configuration file located at:
C:\Program Files\cb2bib\data\netqinf.txt (windows)
or
/usr/share/cb2bib/data/netqinf.txt (linux) (you'll need permissions or to be root to edit)

Wading into there you can find out where is being searched and in what order. What would have been ideal for me would have been a search of the IEEE Xplore site, as that would have turned up most of my papers. Unfortunately it was not in there. Second best was google scholar, sitting at the bottom of the list of options. The documentation in the file wasn't brilliant, but with a bit of trial and error I was able to work out what was going on.

The major change I made to the file was to add this at the top of the queries list:

# QUERY INFO FOR Google Scholar
journal=
query=http://scholar.google.com/scholar?hl=en&lr=&ie=UTF-8&q=<<title>>&btnG=Search
capture_from_query=info:(.+):scholar
referenceurl_prefix=http://scholar.google.com/scholar.bib?hl=en&lr=&ie=UTF-8&q=info:
referenceurl_sufix=:scholar.google.com/&output=citation&oe=ASCII&oi=citation
pdfurl_prefix=
pdfurl_sufix=
action=


journal=
query=http://scholar.google.com/scholar?hl=en&lr=&ie=UTF-8&q=<<excerpt>>&btnG=Search
capture_from_query=info:(.+):scholar
referenceurl_prefix=http://scholar.google.com/scholar.bib?hl=en&lr=&ie=UTF-8&q=info:
referenceurl_sufix=:scholar.google.com/&output=citation&oe=ASCII&oi=citation
pdfurl_prefix=
pdfurl_sufix=
action=

The important changes here are the <<title>> and <<excerpt>> search strings, and the change from capture_from_query=info:(\w+):scholar in the existing scholar searches to capture_from_query=info:(.+):scholar in my search. I'm not too sure what the latter change did, but its effect was that it found the details - where previously it was often missing them!

The other change I made was to untick the option "Set 'title' in double braces" box in the configuration window. After I'd made these changes it worked a lot more consistently.

Some of the time it still pulled out the wrong details if it mis-extracted the article title, however I'd named all my pdfs with the title of the paper, therefore it was simply a case of copying and pasting the filename into the title field and rerunning. It would have been really nice to be able to use the title of my pdf as part of the search but unfortunately I couldn't find a way of doing that.

The only other issue I'm having is that although cb2bib adds in the link to the pdf file, JabRef wont understand it as it uses a very slightly different bibtex format for it. The cb2bib format seems to be:
file = {location}
whereas the JabRef format seems to be:
file = {description:location:type}
There is a comment here by a Mendeley admin that suggests that there is no prescribed format for this aspect of a bibtex file, so I guess it's to be expected. I should be able to work around it with a bit of clever find/replace, but it's an annoyance.
ACTUALLY - this seems to be working under windows! It looks like a different version of JabRef has gotten around this issue.

UPDATE: After a couple of months of getting used to cb2bib and using it to produce a document I'm not really finding the need to use JabRef at all! The 'citer' facility of cb2bib is actually really good.

UPDATE: I hadn't previously gotten round to extracting from IEEE Xplore, as almost everything is on Google Scholar. However I've just tried to set it up and found that the IEEE pages use javascript buttons to produce the citation. This makes it difficult to fully automate.


If you add the following to netqinf.txt then it should search IEEE Xplore for the title, you can then manually click the "download citation" button, select BibTeX format and then copy the BibTeX citation into cb2bib:

# QUERY INFO FOR IEEEXplore
journal=
query=http://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=true&queryText=<<title>>&x=35&y=7
capture_from_query=arnumber=(\d+)&contentType
referenceurl_prefix=http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=
referenceurl_sufix=
pdfurl_prefix=
pdfurl_sufix=
action=browse_referenceurl