Louis-Philippe Véronneau - codehttps://veronneau.org/2024-03-09T00:00:00-05:00Acts of active procrastination: example of a silly Python script for Moodle2024-03-09T00:00:00-05:002024-03-09T00:00:00-05:00Louis-Philippe Véronneautag:veronneau.org,2024-03-09:/acts-of-active-procrastination-example-of-a-silly-python-script-for-moodle.html<p>My brain is currently suffering from an overload caused by grading student
assignments.</p>
<p>In search of a somewhat productive way to procrastinate, I thought I
would share a small script I wrote sometime in 2023 to facilitate my grading
work.</p>
<p>I use Moodle for all the classes I teach and …</p><p>My brain is currently suffering from an overload caused by grading student
assignments.</p>
<p>In search of a somewhat productive way to procrastinate, I thought I
would share a small script I wrote sometime in 2023 to facilitate my grading
work.</p>
<p>I use Moodle for all the classes I teach and students use it to hand me out
their papers. When I'm ready to grade them, I download the ZIP archive Moodle
provides containing all their PDF files and comment them <a href="https://veronneau.org/grading-using-the-wacom-intuos-s.html">using xournalpp and
my Wacom tablet</a>.</p>
<p>Once this is done, I have a directory structure that looks like this:</p>
<pre>
Assignment FooBar/
├── Student A_21100_assignsubmission_file
│ ├── graded paper.pdf
│ ├── Student A's perfectly named assignment.pdf
│ └── Student A's perfectly named assignment.xopp
├── Student B_21094_assignsubmission_file
│ ├── graded paper.pdf
│ ├── Student B's perfectly named assignment.pdf
│ └── Student B's perfectly named assignment.xopp
├── Student C_21093_assignsubmission_file
│ ├── graded paper.pdf
│ ├── Student C's perfectly named assignment.pdf
│ └── Student C's perfectly named assignment.xopp
⋮
</pre>
<p>Before I can upload files back to Moodle, this directory needs to be copied (I
have to keep the original files), cleaned of everything but the <code>graded
paper.pdf</code> files and compressed in a ZIP.</p>
<p>You can see how this can quickly get tedious to do by hand. Not being a
<em>complete</em> tool, I often resorted to crafting a few spurious shell one-liners
each time I had to do this<sup id="fnref:oneliner"><a class="footnote-ref" href="#fn:oneliner">1</a></sup>. Eventually I got tired of <code>ctrl-R</code>-ing my
shell history and wrote something reusable.</p>
<p>Behold this script! When I began writing this post, I was certain I had cheaped
out on my 2021 New Year's resolution and written it in Shell, but glory!, it
seems I used a proper scripting language instead.</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/python3</span>
<span class="c1"># Copyright (C) 2023, Louis-Philippe Véronneau <pollo@debian.org></span>
<span class="c1">#</span>
<span class="c1"># This program is free software: you can redistribute it and/or modify</span>
<span class="c1"># it under the terms of the GNU General Public License as published by</span>
<span class="c1"># the Free Software Foundation, either version 3 of the License, or</span>
<span class="c1"># (at your option) any later version.</span>
<span class="c1">#</span>
<span class="c1"># This program is distributed in the hope that it will be useful,</span>
<span class="c1"># but WITHOUT ANY WARRANTY; without even the implied warranty of</span>
<span class="c1"># MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span>
<span class="c1"># GNU General Public License for more details.</span>
<span class="c1">#</span>
<span class="c1"># You should have received a copy of the GNU General Public License</span>
<span class="c1"># along with this program. If not, see <http://www.gnu.org/licenses/>.</span>
<span class="sd">"""</span>
<span class="sd">This script aims to take a directory containing PDF files exported via the</span>
<span class="sd">Moodle mass download function, remove everything but the final files to submit</span>
<span class="sd">back to the students and zip it back.</span>
<span class="sd">usage: ./moodle-zip.py <target_dir></span>
<span class="sd">"""</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">shutil</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">tempfile</span>
<span class="kn">from</span> <span class="nn">fnmatch</span> <span class="kn">import</span> <span class="n">fnmatch</span>
<span class="k">def</span> <span class="nf">sanity</span><span class="p">(</span><span class="n">directory</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""Run sanity checks before doing anything else"""</span>
<span class="n">base_directory</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">basename</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">normpath</span><span class="p">(</span><span class="n">directory</span><span class="p">))</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">directory</span><span class="p">):</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Target directory </span><span class="si">{</span><span class="n">directory</span><span class="si">}</span><span class="s2"> is not a valid directory"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="sa">f</span><span class="s2">"/tmp/</span><span class="si">{</span><span class="n">base_directory</span><span class="si">}</span><span class="s2">.zip"</span><span class="p">):</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Final ZIP file path '/tmp/</span><span class="si">{</span><span class="n">base_directory</span><span class="si">}</span><span class="s2">.zip' already exists"</span><span class="p">)</span>
<span class="k">for</span> <span class="n">root</span><span class="p">,</span> <span class="n">dirnames</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">directory</span><span class="p">):</span>
<span class="k">for</span> <span class="n">dirname</span> <span class="ow">in</span> <span class="n">dirnames</span><span class="p">:</span>
<span class="n">corrige_present</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">listdir</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">dirname</span><span class="p">)):</span>
<span class="k">if</span> <span class="n">fnmatch</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="s1">'graded paper.pdf'</span><span class="p">):</span>
<span class="n">corrige_present</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">if</span> <span class="n">corrige_present</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Directory </span><span class="si">{</span><span class="n">dirname</span><span class="si">}</span><span class="s2"> does not contain a 'graded paper.pdf' file"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">clean</span><span class="p">(</span><span class="n">directory</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""Remove superfluous files, to keep only the graded PDF"""</span>
<span class="k">with</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">TemporaryDirectory</span><span class="p">()</span> <span class="k">as</span> <span class="n">tmp_dir</span><span class="p">:</span>
<span class="n">shutil</span><span class="o">.</span><span class="n">copytree</span><span class="p">(</span><span class="n">directory</span><span class="p">,</span> <span class="n">tmp_dir</span><span class="p">,</span> <span class="n">dirs_exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="k">for</span> <span class="n">root</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">filenames</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">tmp_dir</span><span class="p">):</span>
<span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">filenames</span><span class="p">:</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">fnmatch</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="s1">'graded paper.pdf'</span><span class="p">):</span>
<span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">file</span><span class="p">))</span>
<span class="n">compress</span><span class="p">(</span><span class="n">tmp_dir</span><span class="p">,</span> <span class="n">directory</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">compress</span><span class="p">(</span><span class="n">directory</span><span class="p">,</span> <span class="n">target_dir</span><span class="p">):</span>
<span class="w"> </span><span class="sd">"""Compress directory into a ZIP file and save it to the target dir"""</span>
<span class="n">target_dir</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">basename</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">normpath</span><span class="p">(</span><span class="n">target_dir</span><span class="p">))</span>
<span class="n">shutil</span><span class="o">.</span><span class="n">make_archive</span><span class="p">(</span><span class="sa">f</span><span class="s2">"/tmp/</span><span class="si">{</span><span class="n">target_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="s1">'zip'</span><span class="p">,</span> <span class="n">directory</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Final ZIP file has been saved to '/tmp/</span><span class="si">{</span><span class="n">target_dir</span><span class="si">}</span><span class="s2">.zip'"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="w"> </span><span class="sd">"""Main function"""</span>
<span class="n">target_dir</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">sanity</span><span class="p">(</span><span class="n">target_dir</span><span class="p">)</span>
<span class="n">clean</span><span class="p">(</span><span class="n">target_dir</span><span class="p">)</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">main</span><span class="p">()</span>
</code></pre></div>
<p>If for some reason you happen to have a similar workflow as I and end up using
this script, hit me up?</p>
<p>Now, back to grading...</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:oneliner">
<p>If I recall correctly, the lazy way I used to do it involved
copying the directory, renaming the extension of the <code>graded paper.pdf</code>
files, deleting all <code>.pdf</code> and <code>.xopp</code> files using <code>find</code> and changing
<code>graded paper.foobar</code> back to a PDF. Some clever regex or learning <code>awk</code>
from the ground up could've probably done the job as well, but you know,
that would have required using my brain and <a href="https://debconf17.debconf.org/talks/92/">spending spoons</a>... <a class="footnote-backref" href="#fnref:oneliner" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
</ol>
</div>membernator -- validate membership cards2019-06-20T00:00:00-04:002019-06-20T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2019-06-20:/membernator-validate-membership-cards.html<p>I currently work part-time for student unions in Montreal and they often have
large general assemblies (more than 2000 people). As you can likely figure out
by yourself, running through paper lists to validate people's identity is a
real PITA and takes quite a long time.</p>
<p>For example, even if …</p><p>I currently work part-time for student unions in Montreal and they often have
large general assemblies (more than 2000 people). As you can likely figure out
by yourself, running through paper lists to validate people's identity is a
real PITA and takes quite a long time.</p>
<p>For example, even if you have 4 people checking names, if validating someone's
identity takes 5 seconds on average (that's pretty fast), it takes around 40
minutes to go through 2000 people.</p>
<p>Introducing <a href="https://gitlab.com/baldurmen/membernator/">membernator</a>, a python program written using <a href="https://www.pygame.org/news">pygame</a> that
validates membership cards against a CSV database! The idea is to use barcode
scanners to scan people's school ID cards and see if they are in our digital
lists. Hopefull, it will make our GA process easier for everyone.</p>
<p>I want to thank Jonathan Carter who provided the inspiration (and a codebase)
for this project. membernator is a heavily-modified fork of <a href="https://salsa.debian.org/debconf-video-team/toetally">ToeTally</a>, a
program currently in developpement for the DebConf Video Team.</p>
<p>membernator will eventually be packaged in Debian (I've started
<a href="https://salsa.debian.org/debian/firmware-tomu">packaging</a> <a href="https://salsa.debian.org/python-team/applications/rename-flac">stuff</a>!), but for now you can either
install it manually or <a href="https://pypi.org/project/membernator/">get it from PyPi</a>.</p>
<p>Here's a quick video of what running membernator looks like. I'm typing the IDs
by hand since I left my barcode scanner at work. Excuse the weird screen
glitches, it seems I'm somewhat bad a screen recording.</p>
<video src="https://veronneau.org/media/blog/2019-06-20/membernator_demo.webm" title="Demo of membernator" alt="Demo of membernator" preload></video>IMAP Spam Begone (ISBG) version 2.1.0 is out!2018-06-14T00:00:00-04:002018-06-14T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2018-06-14:/imap-spam-begone-isbg-version-210-is-out.html<p>When I first started at the non-profit where I work, one of the problems people
had was rampant spam on their email boxes. The email addresses we use are pretty
old (+10 years) and over time they have been added to all the possible spam
lists there are.</p>
<p>That would …</p><p>When I first started at the non-profit where I work, one of the problems people
had was rampant spam on their email boxes. The email addresses we use are pretty
old (+10 years) and over time they have been added to all the possible spam
lists there are.</p>
<p>That would not be a real problem if our email hosting company did not have very
bad spam filters. They are a worker's coop and charge us next to nothing for
hosting our emails, but sadly they lack the resources to run a real
bayesian-based spam filtering solution like SpamAssassin. "Luckily" for us, it
seems that a lot of ISPs and email hosting enterprises also tend to have pretty
bad spam filtering on the email boxes they provide and there were a few programs
out there to fix this.</p>
<p>One of the solutions I found to alleviate this problem was to use <a href="https://github.com/isbg/isbg">IMAP Spam
Begone (ISBG)</a>, a script that makes it easy to scan an IMAP inbox for spam
using your own SpamAssassin server and get your spam moved around via IMAP.
Since then, I've been maintaining the upstream project.</p>
<p>At the time, ISBG was somewhat abandoned and was mostly a script made of old
python2 code. No classes, no functions, just a long script that ran from top to
bottom.</p>
<p>Well, I'm happy to say that ISBG now has a new major release! Version 2.1.0 is
out and replaces the last main release, 1.0.0. From a script, ISBG has now
evolved into a full-fledged python module using classes and functions. Although
the code still works with python2, everything is now python3 compliant as well.
We even started using CI tests recently!</p>
<p>That, and you know, tons of bugs were fixed. I'd like to thank all the folks who
submitted patches, as very few of the actual code was written by me.</p>
<p>If you want to give ISBG a try, you can find the documentation <a href="https://isbg.readthedocs.io/en/v2.1.0/">here</a>.
Here's also a nice terminal capture I made of ISBG working in verbose mode:</p>
<video src="https://veronneau.org/media/blog/2018-06-14/isbg_demo.webm" title="Demo of a verbose ISBG" alt="Demo of a verbose ISBG" preload></video>