Louis-Philippe Véronneau - accountinghttps://veronneau.org/2022-02-05T00:00:00-05:00Migrating from ledger to hledger2022-02-05T00:00:00-05:002022-02-05T00:00:00-05:00Louis-Philippe Véronneautag:veronneau.org,2022-02-05:/migrating-from-ledger-to-hledger.html<p>I first started using ledger — the original <a href="https://plaintextaccounting.org/">plain-text accounting
software</a> — in 2017. Since then, I had been pretty happy with my
accounting routine, but grew a little annoyed by the repetitive manual work I
had to do to assign recurring transactions to the right account.</p>
<p>To make things easier, I …</p><p>I first started using ledger — the original <a href="https://plaintextaccounting.org/">plain-text accounting
software</a> — in 2017. Since then, I had been pretty happy with my
accounting routine, but grew a little annoyed by the repetitive manual work I
had to do to assign recurring transactions to the right account.</p>
<p>To make things easier, I had a collection of bash scripts to parse and convert
the CSV files from my bank's website<sup id="fnref:csv"><a class="footnote-ref" href="#fn:csv">1</a></sup> into ledger entries. They were of
course ugly, unreadable piles of sed-grep-regex and did not let met achieve the
automation complexity I longed for.</p>
<p>Dissatisfied with ledger's features, I decided to migrate to hledger. Contrary
to ledger, hledger comes with a <a href="https://hledger.org/hledger.html#csv-format">CSV parser</a> one can use to import
and classify recurring transactions automagically.</p>
<p>Having a proper DSL for this makes all the difference: I can easily add new
rules and understand the old ones. In the end, I get a very consistent result,
something my old bash scripts weren't great at.</p>
<p>Here is what my <code>debit.csv.rules</code> file looks like. It is used to tell hledger
how CSV data from my debit card should be parsed:</p>
<pre>
# skip the headings line:
skip 1
# assign fields
fields , , , date, , description, , amount2-in, amount2-out
# assign account where the money comes from
# 99 to get it at the bottom of the transaction
account99 assets:checking
include matchers.rules
</pre>
<p>As you can see, the rules matching transactions to specific accounts are
imported from another file, <code>matchers.rules</code>. I'm doing this since I want to be
able to use the same set of rules for debit and credit and use the cards
interchangeably.</p>
<p>Here's a snippet of my <code>matchers.rules</code> file:</p>
<pre>
# house
if Loyer/bail
account2 expenses:rent
# leisure
if PAYPAL .*STEAM GAMES
account2 expenses:leisure:videogame
if PAYPAL .*BANDCAMP
account2 expenses:leisure:music
</pre>
<p>Using this ruleset, a transaction looking like this:</p>
<pre>
"SOME ACCOUNT DESC","111111","EOP","2022/01/03",00002,"Loyer/bail","",521.00,"","","","",""
</pre>
<p>Would be transformed into:</p>
<pre>
2022-01-03 Loyer/bail
expenses:rent 521.00
assets:checking
</pre>
<p>Sadly, hledger's CSV rules won't let you do arithmetics. This can be useful
when you know a certain transaction needs to be split between accounts.</p>
<p>This is where <a href="https://hledger.org/hledger.html#auto-postings">auto postings</a> come in. They are a way to specify arbitrary
rules when an account is encountered.</p>
<p>Going back to my previous rent example, I split it 50/50 with my SO using this
rule:</p>
<pre>
= expenses:rent
assets:receivable:rent *0.5
assets:checking
</pre>
<p>After it is applied, the final transaction looks like this:</p>
<pre>
2022-01-03 Loyer/bail ; modified:
expenses:rent 521.00
assets:receivable:rent 260.50 ; generated-posting: = expenses:rent
assets:checking
</pre>
<p>Neat eh? Here is the little bash script I've written to automate all these
steps:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1">#</span>
<span class="c1"># Convert CSV to ledger using hledger</span>
<span class="nb">declare</span><span class="w"> </span>-a<span class="w"> </span><span class="nv">assets</span><span class="o">=(</span><span class="s2">"credit"</span><span class="w"> </span><span class="s2">"debit"</span><span class="o">)</span>
<span class="c1"># Read the array values with space</span>
<span class="k">for</span><span class="w"> </span>i<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="s2">"</span><span class="si">${</span><span class="nv">assets</span><span class="p">[@]</span><span class="si">}</span><span class="s2">"</span>
<span class="k">do</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span>-f<span class="w"> </span><span class="s2">"</span><span class="nv">$i</span><span class="s2">.csv"</span>
<span class="w"> </span><span class="k">then</span>
<span class="w"> </span><span class="c1"># convert to ledger</span>
<span class="w"> </span><span class="nv">LEDGER</span><span class="o">=</span><span class="k">$(</span>hledger<span class="w"> </span>-f<span class="w"> </span><span class="s2">"</span><span class="nv">$i</span><span class="s2">.csv"</span><span class="w"> </span>--rules-file<span class="w"> </span>rules/<span class="s2">"</span><span class="nv">$i</span><span class="s2">.csv"</span>.rules<span class="w"> </span>print<span class="k">)</span>
<span class="w"> </span><span class="c1"># add auto_postings</span>
<span class="w"> </span><span class="nv">LEDGER</span><span class="o">=</span><span class="k">$(</span><span class="nb">printf</span><span class="w"> </span><span class="s2">"include rules/auto_postings.ledger\n\n</span><span class="nv">$LEDGER</span><span class="s2">\n"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>hledger<span class="w"> </span>print<span class="w"> </span>-f-<span class="w"> </span>--auto<span class="k">)</span>
<span class="w"> </span><span class="c1"># remove superfluous assets:checking lines</span>
<span class="w"> </span><span class="nv">LEDGER</span><span class="o">=</span><span class="k">$(</span><span class="nb">printf</span><span class="w"> </span><span class="s2">"</span><span class="nv">$LEDGER</span><span class="s2">"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sed<span class="w"> </span><span class="s1">'/assets:checking \+; generated.\+/d'</span><span class="k">)</span>
<span class="w"> </span><span class="nb">printf</span><span class="w"> </span><span class="s2">"</span><span class="nv">$LEDGER</span><span class="s2">"</span><span class="w"> </span>><span class="w"> </span><span class="s2">"</span><span class="nv">$i</span><span class="s2">.ledger"</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="nb">printf</span><span class="w"> </span><span class="s2">"File </span><span class="nv">$i</span><span class="s2">.csv does not exist\n"</span>
<span class="w"> </span><span class="k">fi</span>
<span class="k">done</span>
</code></pre></div>
<p>Migrating to hledger, I've cut down the time I spend on accounting from 1 hour
per month to about 10 minutes, all while making my workflow much cleaner.</p>
<p>Many thanks to the kind folks on <code>#hledger @irc.libera.chat</code> for the help!</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:csv">
<p>One for my checking+savings account and another one for my credit card. <a class="footnote-backref" href="#fnref:csv" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
</ol>
</div>