{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "28creation",
  "home_page_url": "https://28creation.com/",
  "feed_url": "https://28creation.com/feed.json",
  "description": "Practical guides to automating office work, from email and spreadsheets to AI agents, plus same-day news on the software people work in.",
  "icon": "https://28creation.com/icon-512.png",
  "language": "en",
  "authors": [
    {
      "name": "28creation",
      "url": "https://28creation.com/about/"
    }
  ],
  "items": [
    {
      "id": "https://28creation.com/articles/excel-csv-dates-leading-zeros/",
      "url": "https://28creation.com/articles/excel-csv-dates-leading-zeros/",
      "title": "Excel CSV imports that mangle zeros and dates: 12 traps tested",
      "content_html": "<h2>The short read</h2><p><strong>In our test, one LibreOffice import read 1/2 as January 2 under US settings and February 1 under UK settings.</strong></p><ul><li>On a sample CSV with 12 trap columns, guessing importers changed up to 18 of 24 cells; importing as text changed none.</li><li>Excel 365 and Excel 2024 can now switch off four of these conversions; Power Query can load every column as text.</li></ul><h3>What you can do with this</h3><blockquote><p>Before you open a CSV with IDs, ZIP codes or codes like 3-4, import it with every column set to Text, then convert only the columns you know are numbers or dates.</p></blockquote><hr><p>Open a CSV in Excel and it decides what each field is. ZIP code <code>02134</code> becomes the number 2134, an 18-digit order ID loses its last three digits, and a fraction like <code>1/2</code> can turn into a date. <a href=\"https://support.microsoft.com/en-us/office/keep-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7\">Microsoft’s own support page</a> says Excel removes leading zeros and rounds anything past the 15th digit. Once it has done that, formatting cannot bring the data back.</p>\n<p>This guide shows why it happens, what each fix does, and when each one fails. We built a test CSV with 12 trap columns and ran it through LibreOffice’s importer, pandas and a short Python script that writes an Excel file with nothing guessed. We did not have Excel in our test environment, so the Excel steps come from Microsoft’s documentation and are marked as such. Sources were read on October 21, 2026.</p>\n<h2 id=\"a-csv-carries-no-types-so-every-program-guesses\">A CSV carries no types, so every program guesses</h2>\n<p>The CSV format’s reference document, <a href=\"https://www.rfc-editor.org/rfc/rfc4180.txt\">RFC 4180</a>, says there is “no formal specification” and describes only records, commas and quotes. Nothing in the file can say “this column is text”. So the program that opens it has to guess from what each value looks like.</p>\n<p>Excel’s guess runs at the moment data comes in. Microsoft’s <a href=\"https://support.microsoft.com/en-us/office/set-automatic-data-conversions-33d1d872-9926-4ef3-9364-492444d0c7ea\">automatic data conversion page</a> lists when: opening a .csv or .txt file, typing, and pasting from outside sources. It names four conversions:</p>\n<ul>\n<li><code>00123</code> becomes the number 123 (leading zeros removed).</li>\n<li><code>12345678901234567890</code> becomes <code>12345678901234500000</code> or <code>1.23457E+19</code>, because Excel keeps <a href=\"https://support.microsoft.com/en-us/office/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3\">15 digits of precision</a>.</li>\n<li><code>123E5</code> becomes <code>1.23E+07</code>, read as scientific notation.</li>\n<li><code>JAN1</code> becomes the date January 1.</li>\n</ul>\n<p>The damage happens on the way in. <a href=\"https://support.microsoft.com/en-us/office/keep-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7\">Microsoft says</a> a custom number format such as <code>00000</code> “will not restore leading zeros that were removed prior to formatting”. The same holds for digits past the 15th: they are gone from the cell, not hidden.</p>\n<h2 id=\"the-guessing-has-corrupted-published-science\">The guessing has corrupted published science</h2>\n<p>This is not a rare edge case. In 2016, Mark Ziemann, Yotam Eren and Assam El-Osta <a href=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC4994289/\">screened supplementary files from 18 genomics journals</a> published from 2005 to 2015. They confirmed gene name errors in 987 files from 704 papers: 19.6% of the papers with Excel gene lists. Gene symbols such as <code>SEPT2</code> and <code>MARCH1</code> had become <code>2-Sep</code> and <code>1-Mar</code>, and a RIKEN identifier, <code>2310009E13</code>, had become <code>2.31E+13</code>.</p>\n<p>The fix that stuck was renaming the genes. The HUGO Gene Nomenclature Committee’s <a href=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC7494048/\">2020 guidelines in Nature Genetics</a> report that “all symbols that auto-converted to dates in Microsoft Excel have been changed”. <code>SEPT1</code> is now <code>SEPTIN1</code>, and <code>MARCH1</code> is now <code>MARCHF1</code>. Your part numbers and customer IDs will not get renamed, so the fix has to happen at import.</p>\n<h2 id=\"our-test-12-trap-columns-seven-ways-to-open-them\">Our test: 12 trap columns, seven ways to open them</h2>\n<p>We wrote a two-row CSV with 12 columns, each holding a value type that spreadsheets are known to change. Every value in it is made up.</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>zip,employee_id,order_id,gene,gene_2,promo_code,portion,part_no,accession,product_code,phone,ship_date</span></span>\n<span class=\"line\"><span>02134,00123,123456789012345678,SEPT2,MARCH1,JAN1,1/2,3-4,2310009E13,1E5,+46701234567,03/04/2026</span></span>\n<span class=\"line\"><span>07030,00456,987654321098765432,SEPT9,MARCH5,DEC25,3/4,10-12,4930430E12,2E10,+14155550100,12/05/2026</span></span></code></pre>\n<p>We opened it seven ways on Linux on October 21, 2026, and counted how many of the 24 data cells came back different from the file. The LibreOffice runs used version 24.2.7.2, converting to .xlsx from the command line, with each result read back with openpyxl 3.1.5. Here is what changed.</p>\n<table>\n<thead>\n<tr>\n<th>Route</th>\n<th>Cells changed (of 24)</th>\n<th>What went wrong</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>LibreOffice, default import</td>\n<td>12</td>\n<td>Zeros dropped, order IDs cut to 15 digits, <code>2310009E13</code> and <code>1E5</code> read as numbers, <code>+</code> dropped from phones</td>\n</tr>\n<tr>\n<td>LibreOffice, “detect special numbers” on, English (US)</td>\n<td>16</td>\n<td>All of the above, plus <code>1/2</code> read as January 2 and <code>03/04/2026</code> as March 4</td>\n</tr>\n<tr>\n<td>LibreOffice, “detect special numbers” on, English (UK)</td>\n<td>18</td>\n<td>All of the above, but <code>1/2</code> read as February 1, <code>03/04/2026</code> as April 3, and <code>3-4</code> as April 3</td>\n</tr>\n<tr>\n<td>LibreOffice, every column set to Text</td>\n<td>0</td>\n<td>Nothing</td>\n</tr>\n<tr>\n<td>pandas 3.0.5, <code>read_csv</code> defaults</td>\n<td>10</td>\n<td>Zeros dropped, <code>+</code> dropped, accession numbers and <code>1E5</code> read as floats</td>\n</tr>\n<tr>\n<td>pandas, <code>dtype=str, keep_default_na=False</code></td>\n<td>0</td>\n<td>Nothing</td>\n</tr>\n<tr>\n<td>Our script (below)</td>\n<td>0</td>\n<td>Nothing, and a LibreOffice re-export matched the source byte for byte</td>\n</tr>\n</tbody>\n</table>\n<p>Guess before you read on: which value was the most dangerous? Not the dropped zeros, which are visible. The fraction column was worse.</p>\n<p>The same file, imported with the same program, gave <code>1/2</code> as January 2 under US settings and February 1 under UK settings. Both looked like valid dates, and both took the current year (2026), which was never in the file.</p>\n<p>Two smaller findings. First, LibreOffice stored the order ID as <code>1.23456789012346E+017</code> in the saved file, so the digits after the 15th were gone from the workbook itself, not just from the display.</p>\n<p>Second, LibreOffice left <code>SEPT2</code>, <code>MARCH1</code> and <code>JAN1</code> alone in every setting we tried, while Microsoft’s page says Excel turns <code>JAN1</code> into a date. Results from one program do not predict another. LibreOffice’s <a href=\"https://help.libreoffice.org/latest/en-US/text/shared/00/00000208.html\">import documentation</a> confirms the locale effect: with “Detect special numbers” on, “the selected language influences how such special numbers are detected”.</p>\n<h2 id=\"fix-1-set-every-column-to-text-before-the-data-comes-in\">Fix 1: set every column to Text before the data comes in</h2>\n<p>The fix that worked in every route we tried is the same idea in each program: tell the importer the type of each column before it reads a single value. Text is the safe default, because text can always become a number later, and a number cannot become the original text again.</p>\n<p><strong>Excel, Power Query</strong> (from Microsoft’s documentation; we did not run Excel). Microsoft’s <a href=\"https://support.microsoft.com/en-us/office/keep-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7\">steps</a> are:</p>\n<ol>\n<li>Choose <strong>Data</strong>, then <strong>From Text/CSV</strong>, and pick the file.</li>\n<li>In the preview, open the data type detection list. Microsoft’s <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/text-csv\">Power Query documentation</a> says it can be based on the first 200 rows or the entire data set, or turned off so every column defaults to Text. Turn it off.</li>\n<li>Choose <strong>Transform Data</strong>. Select the columns that are real numbers or dates and change their type on <strong>Home</strong>, <strong>Transform</strong>, <strong>Data Type</strong>. For text columns Microsoft’s path is the same menu with <strong>Text</strong>, then <strong>Replace Current</strong> in the Change Column Type dialog.</li>\n<li>Choose <strong>Close &amp; Load</strong>.</li>\n</ol>\n<p>Two settings on the same preview screen matter for this too. <strong>File Origin</strong> sets the character set, and the documentation says UTF-8 is only recognized automatically when the file starts with a UTF-8 byte-order mark. <strong>Delimiter</strong> offers comma, semicolon, tab and others, for files that do not use commas.</p>\n<p>The trade-off: a query is a saved recipe, so refreshing it next month applies the same types again. That is good for a monthly export with a fixed layout and annoying for a one-off file. And if a new column appears later, check its type by hand; the recipe was built for the old layout.</p>\n<p><strong>Excel, the old wizard</strong> (from Microsoft’s documentation, untested). Microsoft’s <a href=\"https://support.microsoft.com/en-us/office/import-or-export-text-txt-or-csv-files-5250ac4c-663c-47ce-937b-339e391393ba\">import page</a> says that opening a CSV directly uses “the current default data format settings”. Renaming the file from .csv to .txt before opening it starts the Import Text Wizard instead.</p>\n<p>The <a href=\"https://support.microsoft.com/en-us/office/text-import-wizard-c5b02af6-fda1-4440-899f-f78bafe41857\">wizard’s page</a> also gives a menu route. Turn on <strong>File</strong>, <strong>Options</strong>, <strong>Data</strong>, <strong>Show legacy data import wizards</strong>, <strong>From Text (Legacy)</strong>, then choose <strong>Data</strong>, <strong>Get Data</strong>, <strong>Legacy Wizards</strong>, <strong>From Text (Legacy)</strong>. In step 3, select each column in the preview and set <strong>Column data format</strong> to <strong>Text</strong>. It is quicker than Power Query for a single file and saves nothing for next time.</p>\n<h2 id=\"libreoffice-and-pandas-take-the-same-text-setting\">LibreOffice and pandas take the same Text setting</h2>\n<p><strong>LibreOffice Calc.</strong> In the <a href=\"https://help.libreoffice.org/latest/en-US/text/shared/00/00000208.html\">Text Import dialog</a>, select the columns in the preview and set <strong>Column type</strong> to <strong>Text</strong>. On the command line, the <a href=\"https://help.libreoffice.org/latest/en-US/text/shared/guide/csv_params.html\">CSV filter options</a> take column/format pairs in the fifth token, where format code 2 means Text. This is the string we ran for our 12 columns:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"sh\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">soffice</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> --headless</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> --convert-to</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> xlsx</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> \\</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">  --infilter=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"CSV:44,34,76,1,1/2/2/2/3/2/4/2/5/2/6/2/7/2/8/2/9/2/10/2/11/2/12/2,1033,false,false,,,,,,,false\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> \\</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">  traps.csv</span></span></code></pre>\n<p>The first four tokens are comma (44), double quote (34), UTF-8 (76) and start at line 1. The token after the pairs, 1033, is the language: English (US).</p>\n<p>The empty tokens that follow leave formula evaluation off, and the final <code>false</code> turns off scientific-notation detection. Our locale runs left the column types at Standard and set token 8, “Detect special numbers”, to <code>true</code>. Then switching 1033 to 2057 (English, UK) moved our dates from the second table row to the third.</p>\n<p><strong>pandas.</strong> <code>pd.read_csv(\"file.csv\", dtype=str, keep_default_na=False)</code> read all 24 cells exactly in our test. <code>keep_default_na=False</code> matters as well: by default pandas <a href=\"https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html\">reads “N/A”, “NULL” and other common markers as missing</a>. In a separate check, <code>dtype=str</code> alone turned the country code <code>NA</code> (Namibia) and the text <code>null</code> into empty values; with <code>keep_default_na=False</code> both survived.</p>\n<h2 id=\"fix-2-switch-off-excels-four-conversions-excel-365-and-2024-only\">Fix 2: switch off Excel’s four conversions (Excel 365 and 2024 only)</h2>\n<p>Microsoft added settings for each of the four conversions. The <a href=\"https://support.microsoft.com/en-us/office/set-automatic-data-conversions-33d1d872-9926-4ef3-9364-492444d0c7ea\">page</a> lists Excel for Microsoft 365 and Excel 2024, on Windows and Mac:</p>\n<ul>\n<li>Windows: <strong>File</strong>, <strong>Options</strong>, the <strong>Data</strong> tab, <strong>Automatic Data Conversion</strong>.</li>\n<li>Mac: <strong>Excel</strong>, <strong>Preferences</strong>, the <strong>Edit</strong> tab, <strong>Automatic Data Conversion</strong>.</li>\n</ul>\n<p>Under it are four options. Clear the ones you do not want:</p>\n<ul>\n<li>“Remove leading zeros and convert to number”</li>\n<li>“Keep the first 15 digits of long numbers and display in scientific notation if required”</li>\n<li>“Convert digits surrounding the letter “E” into a number in scientific notation”</li>\n<li>“Convert “date-like” combinations of continuous letters and numbers into a date”</li>\n</ul>\n<p>Know the limits before you rely on it. The date option covers continuous letters and numbers like <code>JAN1</code>. Microsoft notes that <code>JAN 1</code> or <code>JAN-1</code> may still be read as dates, and the page does not say whether values like <code>1/2</code> or <code>3-4</code> are covered. We could not test it without Excel, so assume they are not.</p>\n<p>The settings live in each person’s copy of Excel, so a colleague who opens the same file sees the old behavior. Microsoft also says the options do not apply to Power Query, which has its own types. Older versions, such as Excel 2021 or 2019, are not on the page’s list.</p>\n<p>Our view: switch them off on your own machine, and still use Fix 1 for any file that matters. A setting that differs per person is not a process.</p>\n<h2 id=\"fix-3-for-one-value-at-a-time-type-it-as-text\">Fix 3: for one value at a time, type it as text</h2>\n<p>When you type into a cell rather than import, Microsoft’s page on <a href=\"https://support.microsoft.com/en-us/office/stop-automatically-changing-numbers-to-dates-452bd2db-cc96-47d1-81e4-72cec11c4ed8\">stopping numbers changing to dates</a> gives four ways:</p>\n<ul>\n<li>Format the cells as Text before you type.</li>\n<li>Type an apostrophe first, as in <code>'1/47</code>. The apostrophe is not shown.</li>\n<li>Type a space first. The space stays in the cell.</li>\n<li>Type <code>0 1/2</code> for the fraction one half.</li>\n</ul>\n<p>These fit a handful of entries, not a 5,000-row export. The leading space in particular is a trap of its own: the cell now holds “ 1/2”, which will not match “1/2” in a lookup.</p>\n<h2 id=\"fix-4-write-the-excel-file-yourself-with-types-you-chose\">Fix 4: write the Excel file yourself, with types you chose</h2>\n<p>If a CSV arrives every week from the same system, the most reliable route is to skip the import step entirely. A script reads the file as plain text and writes an .xlsx in which every column is text unless you named it as a number or a date. The receiver opens a workbook, and no program has anything left to guess.</p>\n<p>The script below does that with Python and <a href=\"https://openpyxl.readthedocs.io/en/stable/\">openpyxl</a>. It also formats text columns as Text (<code>@</code>), so values someone types into them later stay text too. Then it reads the saved file back and stops if any text cell differs from its CSV field. It writes to a temporary file first, so a failed run leaves no half-finished workbook.</p>\n<p>One trap we found while building it: openpyxl stores any string that starts with <code>=</code> as a formula. In our edge-case file, a field <code>=1+2</code> saved by a plain <code>append</code> came back from LibreOffice as <code>3</code>. The script forces those cells back to text, which also stops a CSV field from running as a formula in the receiver’s spreadsheet.</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"Turn a CSV into an .xlsx where nothing is guessed.</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Every column is stored as text unless you name it:</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">  --number amount,qty        stored as numbers (a value that is not a plain number stops the run)</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">  --date ship_date=%m/%d/%Y  stored as real dates, read with the format you give</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Usage: python csv_to_xlsx.py input.csv output.xlsx [--number ...] [--date ...]</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> argparse, csv, os, sys, tempfile</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> openpyxl </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Workbook, load_workbook</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> argparse.ArgumentParser()</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p.add_argument(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"src\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p.add_argument(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"dst\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p.add_argument(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"--number\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">default</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">help</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"comma-separated column names\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p.add_argument(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"--date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">action</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"append\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">default</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">[], </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">help</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"column=strptime format\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">p.add_argument(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"--delimiter\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">default</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">a </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> p.parse_args()</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># utf-8-sig drops a byte-order mark at the start of the file, if there is one</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">with</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> open</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(a.src, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">newline</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">encoding</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"utf-8-sig\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> f:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> list</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(csv.reader(f, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">delimiter</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">a.delimiter))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">a.src</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> is empty, nothing written\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">header </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">numbers </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> a.number.split(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c}</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dates </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> dict</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(d.split(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"=\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> a.date)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">missing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (numbers </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dates.keys()) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> missing:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"no such column(s): </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">', '</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(missing))</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> convert</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(col, value, line):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        return</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#6A737D;--shiki-dark:#768390\">                      # an empty cell stays empty</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> numbers:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"_\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value:             </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># Python reads 1_000 as 1000; a CSV should not</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                raise</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ValueError</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            return</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> float</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(value) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> any</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(ch </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ch </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \".eE\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> int</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(value)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dates:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime.strptime(value, dates[col]).date()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"line </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">line</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, column </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">col</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">value</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!r</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> is not a </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'number'</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> numbers </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dates[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ' date'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value                         </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># everything else: text, exactly as in the file</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">wb </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Workbook()</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">ws </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> wb.active</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">ws.append(header)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line, row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> enumerate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rows[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:], </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">start</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"line </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">line</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> has </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> fields, the header has </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    ws.append([convert(c, v, line) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c, v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> zip</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header, row)])</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># Format text columns as Text (@) so values typed into them later stay text too.</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># openpyxl stores any string that starts with \"=\" as a formula; force those back to text.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i, col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> enumerate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">start</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    fmt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"yyyy-mm-dd\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dates </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"General\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> numbers </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"@\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (cell,) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ws.iter_rows(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">min_col</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">i, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">max_col</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">i, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">min_row</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        cell.number_format </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> fmt</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> fmt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"@\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> isinstance</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(cell.value, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            cell.data_type </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"s\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># Write to a temporary file and rename, so a failed run never leaves half a workbook</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">fd, tmp </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tempfile.mkstemp(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">suffix</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\".xlsx\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">dir</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">os.path.dirname(os.path.abspath(a.dst)))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">os.close(fd)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    wb.save(tmp)</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">    # Read it back: every text cell must equal its CSV field exactly</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    back </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> list</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(load_workbook(tmp).active.iter_rows(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">values_only</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line, (want, got) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> enumerate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">zip</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rows[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:], back[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:]), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">start</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col, w, g </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> zip</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header, want, got):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">not</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> numbers </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">not</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dates </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (g </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> w:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"read-back mismatch, line </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">line</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, column </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">col</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">w</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!r</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> became </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">g</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!r</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    os.replace(tmp, a.dst)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">finally</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> os.path.exists(tmp):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        os.remove(tmp)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">a.dst</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rows) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> rows, </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> columns, \"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(numbers) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(dates)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> as text; read-back check passed\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span></code></pre>\n<h2 id=\"run-it-on-your-file-and-watch-it-refuse-bad-input\">Run it on your file, and watch it refuse bad input</h2>\n<p>To use it:</p>\n<ol>\n<li>Install the library once: <code>pip install openpyxl</code>.</li>\n<li>Save the script as <code>csv_to_xlsx.py</code>.</li>\n<li>Run it with the columns you know are numbers or dates. For our test file, with the ship date as a real US-style date:</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>$ python csv_to_xlsx.py traps.csv typed.xlsx --date ship_date=%m/%d/%Y</span></span>\n<span class=\"line\"><span>typed.xlsx: 2 rows, 12 columns, 11 as text; read-back check passed</span></span></code></pre>\n<p>With no options, all 12 columns stayed text, all 24 cells matched the file, and LibreOffice’s own CSV export of the workbook was identical to the source. With <code>--date</code>, the two ship dates became the real dates March 4 and December 5, 2026, because we said which format the file uses instead of letting a locale decide.</p>\n<p>We also ran it on the failures you will meet. These were an empty file, a short row, a column name that does not exist, <code>12 USD</code> in a number column and a date in the wrong format. Each stopped the run with the line and column named, and left no output file behind. A file with a byte-order mark, a quoted field with a comma in it and an empty amount came through correctly.</p>\n<p>When not to use it: the script reads the whole file into memory. We ran it only on small files, so try it on a copy before a large export. It does not handle thousands separators (<code>1,250.00</code>) in number columns; it stops on them, so clean those first or leave the column as text. And a worksheet holds at most 1,048,576 rows, per <a href=\"https://support.microsoft.com/en-us/office/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3\">Microsoft’s limits page</a>, so a larger CSV needs a different plan.</p>\n<h2 id=\"fix-5-when-you-produce-the-csv-make-the-receivers-job-easier\">Fix 5: when you produce the CSV, make the receiver’s job easier</h2>\n<p>If your own system or script writes the CSV, two habits reduce the damage on the other side. Write dates as <code>2026-03-04</code> (year, month, day). In our LibreOffice runs it read as March 4 under both US and UK settings. And when the receiver opens files in Excel, send an .xlsx with typed columns instead, made with the script above.</p>\n<p>Do your CSVs come out of email or a PDF in the first place? Our guides to <a href=\"https://28creation.com/articles/email-data-into-excel/\">getting email data into Excel</a> and <a href=\"https://28creation.com/articles/bank-statement-pdf-to-excel/\">bank statement PDFs to Excel</a> cover the step before this one. More on the <a href=\"https://28creation.com/topics/automation/\">automation topic page</a>.</p>\n<h2 id=\"which-fix-to-use\">Which fix to use</h2>\n<ul>\n<li><strong>A one-off file you open yourself:</strong> Fix 1 in your spreadsheet (Power Query with detection off, the wizard, or LibreOffice’s Text column type). Nothing to maintain.</li>\n<li><strong>The same export every week or month, opened in Excel:</strong> a saved Power Query with types set once, or the script in Fix 4 if the file passes through a script anyway. Both repeat the same types every time.</li>\n<li><strong>Data a script reads:</strong> <code>dtype=str, keep_default_na=False</code> in pandas, or Python’s <code>csv</code> module, where <a href=\"https://docs.python.org/3/library/csv.html\">“no automatic data type conversion is performed”</a> unless you ask for it.</li>\n<li><strong>Your own Excel 365 or 2024 for everyday typing and pasting:</strong> Fix 2 as well, knowing it covers four named conversions and may not cover slash or dash dates.</li>\n</ul>\n<p>Whichever route you pick, test it once on a real export before you trust it. Find a row with a leading zero, a long ID and a date where the day is 12 or lower, and compare them with the CSV opened in a text editor. If <code>02134</code> is still <code>02134</code>, the import did its job.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>On October 21, 2026, we wrote a two-row CSV with 12 trap columns of made-up values. We opened it on Linux with LibreOffice 24.2.7.2 under four import settings, pandas 3.0.5 in two ways and the script in this guide (Python 3.13, openpyxl 3.1.5), and read every result back cell by cell; the script also ran on small edge-case files. We read Microsoft’s support and Power Query pages, LibreOffice’s CSV filter documentation, the pandas reference, RFC 4180 and the two gene-name papers that day. We did not have Excel, so every Excel step here comes from Microsoft’s documentation and is untested, and two rows of test data cannot show every value a real export holds.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/keep-leading-zeros-and-large-numbers-1bf7b935-36e1-4985-842f-5dfa51f85fe7\">Keeping leading zeros and large numbers</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/set-automatic-data-conversions-33d1d872-9926-4ef3-9364-492444d0c7ea\">Set automatic data conversions</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/stop-automatically-changing-numbers-to-dates-452bd2db-cc96-47d1-81e4-72cec11c4ed8\">Stop automatically changing numbers to dates</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3\">Excel specifications and limits</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/import-or-export-text-txt-or-csv-files-5250ac4c-663c-47ce-937b-339e391393ba\">Import or export text (.txt or .csv) files</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/text-import-wizard-c5b02af6-fda1-4440-899f-f78bafe41857\">Text Import Wizard</a>, read October 21, 2026</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/text-csv\">Power Query Text/CSV connector</a>, read October 21, 2026</li>\n<li>LibreOffice Help, <a href=\"https://help.libreoffice.org/latest/en-US/text/shared/00/00000208.html\">Text Import</a>, read October 21, 2026</li>\n<li>LibreOffice Help, <a href=\"https://help.libreoffice.org/latest/en-US/text/shared/guide/csv_params.html\">CSV Filter Options</a>, read October 21, 2026</li>\n<li>Python, <a href=\"https://docs.python.org/3/library/csv.html\">csv module documentation</a>, read October 21, 2026</li>\n<li>pandas, <a href=\"https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html\">read_csv reference</a>, read October 21, 2026</li>\n<li>openpyxl, <a href=\"https://openpyxl.readthedocs.io/en/stable/\">documentation</a>, version 3.1.5</li>\n<li>IETF, <a href=\"https://www.rfc-editor.org/rfc/rfc4180.txt\">RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files</a></li>\n<li>Ziemann, Eren and El-Osta, <a href=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC4994289/\">Gene name errors are widespread in the scientific literature</a>, Genome Biology, 2016</li>\n<li>Bruford et al., <a href=\"https://pmc.ncbi.nlm.nih.gov/articles/PMC7494048/\">Guidelines for human gene nomenclature</a>, Nature Genetics, 2020</li>\n<li>28creation test, October 21, 2026: one generated CSV, 12 columns, 24 cells, scored by our own script</li>\n</ul>\n",
      "summary": "A CSV has no column types, so spreadsheets guess. Our test file lost half its cells to guessing; typed text columns kept all 24.",
      "image": "https://28creation.com/og/excel-csv-dates-leading-zeros.jpg",
      "date_published": "2026-10-25T07:00:00.000Z",
      "date_modified": "2026-10-25T07:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Work software",
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/articles/power-automate-vs-zapier-microsoft-365/",
      "url": "https://28creation.com/articles/power-automate-vs-zapier-microsoft-365/",
      "title": "Power Automate vs Zapier for Microsoft 365: 35 apps checked",
      "content_html": "<h2>The short read</h2><p><strong>An Outlook to SharePoint and Teams flow costs $0 extra on Microsoft 365, and from $239.88 a year on Zapier.</strong></p><ul><li>Microsoft 365 plans include Power Automate with standard connectors only. Of 35 common apps we checked, 14 are premium there and QuickBooks Online has none.</li><li>Zapier bills each action and keeps SharePoint, Salesforce and QuickBooks for paid plans, but its Excel app can start a workflow on a new row.</li></ul><h3>What you can do with this</h3><blockquote><p>List the apps your flow touches and check each one&apos;s class on Microsoft&apos;s connector page. All Standard: build it in Power Automate for $0. One Premium: weigh a $180 yearly owner license against Zapier.</p></blockquote><hr><p>An office on Microsoft 365 already owns an automation tool. Microsoft’s plans include Power Automate, and flows between Outlook, Excel, SharePoint and Teams cost nothing extra. Zapier, its best-known rival, charges for every action from the first one.</p>\n<p>So the question for a Microsoft 365 office is rarely “which is better”. It is which apps the flow touches, and whether either vendor puts one of them behind a paywall.</p>\n<p>We checked 35 apps that small offices commonly connect, on both tools, from Microsoft’s connector reference pages and Zapier’s app pages on October 21, 2026. Then we priced three ordinary flows with a short script, from both vendors’ price lists read the same day. The table below is the part you will not find on either vendor’s site.</p>\n<h2 id=\"microsoft-365-pays-for-standard-connectors-and-only-those\">Microsoft 365 pays for standard connectors, and only those</h2>\n<p>Microsoft says select Office 365 licenses include <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">“limited Power Automate use rights”</a> meant “for personal productivity scenarios”. In practice that means cloud flows built only from standard connectors. Users on those plans “can only access standard connectors”, the same FAQ says; the exception is premium connectors inside Dataverse for Teams environments. Premium and custom connectors are not included, and neither is the on-premises data gateway.</p>\n<p>The step up is <a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">Power Automate Premium at $15.00 per user a month, paid yearly</a>, or $180 a year. The rule that decides how many you need is short. For an automated or scheduled flow that uses a premium connector, “only the owner needs to have a Premium license”. For an instant flow (a button, or one started from Power Apps), “every user who runs the flow needs a Power Automate Premium license”.</p>\n<p>There is a trap in that first rule. Microsoft calls it multiplexing: pooling access so fewer people need licenses.</p>\n<p>Its own example is a premium flow that starts when someone adds an item to a SharePoint list. If the flow emails only its owner, one license is enough. If it emails the person who added the item, “both the owner and the user need a Premium license”.</p>\n<p>A single Premium license on a shared service account that many people use is, in Microsoft’s words, “multiplexing, and the flow isn’t compliant”. Microsoft adds that this “isn’t hard enforcement”; admins are responsible.</p>\n<h2 id=\"zapier-charges-per-action-and-gates-some-apps\">Zapier charges per action and gates some apps</h2>\n<p><a href=\"https://zapier.com/pricing\">Zapier’s pricing page</a> counts tasks. Each successful action uses one, though Zapier says “some apps or actions may use more than one task”. Triggers, filters and Formatter steps use none, and neither does polling for new data.</p>\n<p>The Free plan gives 100 tasks a month and two-step Zaps only, meaning one trigger and one action. Professional starts at $19.99 a month billed yearly for 750 tasks, and adds multi-step Zaps and premium apps, plus webhooks. The Team plan starts at $69 a month billed yearly and adds 25 users with shared app connections.</p>\n<p>Zapier also differs on speed. It checks polling triggers every 15 minutes on Free, every 2 minutes on Professional and every minute on Team. Power Automate’s shortest schedule, its <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">minimum recurrence interval</a>, is 60 seconds on every license.</p>\n<h2 id=\"35-apps-checked-on-both-tools-october-21-2026\">35 apps checked on both tools, October 21, 2026</h2>\n<p>Microsoft classes each connector Standard or Premium on its reference page; Zapier marks some apps Premium, which means paid plans only. We read each page for the apps below. In the Zapier column, “Free plan” means the app is not Premium; a Zap with more than one action still needs a paid plan, whatever the app.</p>\n<table>\n<thead>\n<tr>\n<th>App</th>\n<th>Power Automate connector</th>\n<th>Zapier app</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Outlook</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Excel</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>SharePoint</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/sharepointonline/\">Standard</a></td>\n<td><a href=\"https://zapier.com/apps/sharepoint/integrations\">Premium</a></td>\n</tr>\n<tr>\n<td>Teams</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/teams/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>OneDrive for Business</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Gmail</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/gmail/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Google Sheets</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/googlesheet/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Google Drive</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Google Calendar</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Slack</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/slack/\">Standard</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Trello</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Asana</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Jira</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/jira/\">Premium</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Smartsheet</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Notion</td>\n<td>Premium, independent publisher</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Zoom</td>\n<td>Premium, independent publisher, preview</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Dropbox</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Box</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Docusign</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Mailchimp</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/mailchimp/\">Premium</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>HubSpot</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/hubspotcrm/\">Premium</a>, independent publisher, preview</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Salesforce</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/salesforce/\">Premium</a></td>\n<td><a href=\"https://zapier.com/apps/salesforce/integrations\">Premium</a></td>\n</tr>\n<tr>\n<td>Pipedrive</td>\n<td>Premium</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Zendesk</td>\n<td>Premium</td>\n<td><a href=\"https://zapier.com/apps/zendesk/integrations\">Premium</a></td>\n</tr>\n<tr>\n<td>Freshdesk</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Typeform</td>\n<td>Standard, preview</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Calendly</td>\n<td>Premium, preview</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>SurveyMonkey</td>\n<td>Premium</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Twilio</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>SendGrid</td>\n<td>Standard</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Stripe</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/stripe/\">Premium</a></td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>Shopify</td>\n<td>Premium, independent publisher, preview</td>\n<td><a href=\"https://zapier.com/apps/shopify/integrations\">Premium</a></td>\n</tr>\n<tr>\n<td>WooCommerce</td>\n<td>Premium, third-party publisher</td>\n<td>Free plan</td>\n</tr>\n<tr>\n<td>QuickBooks Online</td>\n<td>No connector found</td>\n<td><a href=\"https://zapier.com/apps/quickbooks/integrations\">Premium</a></td>\n</tr>\n<tr>\n<td>Xero</td>\n<td>Premium, third-party publisher, preview</td>\n<td><a href=\"https://zapier.com/apps/xero/integrations\">Premium</a></td>\n</tr>\n</tbody>\n</table>\n<p>Unlinked rows come from the same sets of pages: Microsoft’s at learn.microsoft.com/en-us/connectors/ and Zapier’s at zapier.com/apps/. “Independent publisher” and “third-party publisher” mean the connector page names someone other than Microsoft or the app’s own maker as publisher. “Preview” is Microsoft’s label for a connector not yet generally available. For QuickBooks Online we searched <a href=\"https://learn.microsoft.com/en-us/connectors/connector-reference/connector-reference-powerautomate-connectors\">Microsoft’s list of Power Automate connectors</a> and found only a QuickBooks Time connector.</p>\n<h2 id=\"power-automate-gates-14-of-the-35-apps-zapier-6\">Power Automate gates 14 of the 35 apps, Zapier 6</h2>\n<p>Guess how many of the 35 sit behind a paywall on each side. On Power Automate, 14 are Premium and one has no connector at all. On Zapier, 6 are Premium. The five Microsoft 365 apps in the table are all Standard on Power Automate, and SharePoint is the one Zapier keeps for paid plans.</p>\n<p>The pattern is plain. Microsoft and Google apps are Standard on Power Automate, and so are Slack and Trello, while Jira is Premium. Every CRM and online-store app in the table is Premium, as are Stripe and SurveyMonkey. For HubSpot, Shopify, Notion and Zoom, every connector we found in Microsoft’s list comes from an independent publisher, not Microsoft or the app’s maker.</p>\n<h2 id=\"excel-is-where-zapier-does-something-power-automate-does-not\">Excel is where Zapier does something Power Automate does not</h2>\n<p>The Excel Online (Business) connector’s <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">reference page</a> lists actions only, from adding and updating rows to running Office Scripts. It lists no trigger. So a Power Automate flow cannot start on the Excel connector because a row appeared. The usual route is a schedule that reads the table, and that read returns 256 rows by default unless you turn on pagination.</p>\n<p>The connector also caps files at 25 MB and warns that a file “may be locked for an update or delete up to 6 minutes” after the connector last used it.</p>\n<p><a href=\"https://zapier.com/apps/excel/integrations\">Zapier’s Excel app</a> has four triggers, among them New Row and Updated Row, polling every 15 minutes on Free. Its catch: “Only files on OneDrive for Business (available with Office 365 and Sharepoint server) are supported.” A workbook on a personal OneDrive won’t work.</p>\n<p>So if the job starts when someone types a row into a shared workbook, Zapier fits it directly. Power Automate needs a scheduled flow that compares rows, or a different starting point such as a SharePoint list or Microsoft Forms.</p>\n<h2 id=\"three-flows-priced-at-four-volumes\">Three flows priced at four volumes</h2>\n<p>We wrote a 39-line Node.js script with Zapier’s Professional tiers and Microsoft’s Premium price, both read on October 21, 2026. It priced three hypothetical flows, each a common office job.</p>\n<p>Zapier figures are the cheapest yearly-billed Professional tier that holds the tasks, without pay-per-task overage. Power Automate figures are the extra cost on top of Microsoft 365.</p>\n<p><strong>Flow 1: an invoice email arrives in Outlook; save the attachment to SharePoint and post in Teams.</strong> Two actions, so two Zapier tasks a run, and SharePoint makes it a paid plan anyway.</p>\n<table>\n<thead>\n<tr>\n<th>Runs a month</th>\n<th>Zapier tier</th>\n<th>Zapier a year</th>\n<th>Power Automate a year</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>50</td>\n<td>Pro, 750 tasks</td>\n<td>$239.88</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>300</td>\n<td>Pro, 750 tasks</td>\n<td>$239.88</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>1,000</td>\n<td>Pro, 2,000 tasks</td>\n<td>$588.00</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>5,000</td>\n<td>Pro, 10,000 tasks</td>\n<td>$1,548.00</td>\n<td>$0</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Flow 2: a new Trello card posts a message in Slack.</strong> One action, one task a run.</p>\n<table>\n<thead>\n<tr>\n<th>Runs a month</th>\n<th>Zapier tier</th>\n<th>Zapier a year</th>\n<th>Power Automate a year</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>50</td>\n<td>Free</td>\n<td>$0</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>300</td>\n<td>Pro, 750 tasks</td>\n<td>$239.88</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>1,000</td>\n<td>Pro, 1,500 tasks</td>\n<td>$468.00</td>\n<td>$0</td>\n</tr>\n<tr>\n<td>5,000</td>\n<td>Pro, 5,000 tasks</td>\n<td>$1,068.00</td>\n<td>$0</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Flow 3: a new Salesforce lead becomes a row in an Excel table and a Teams message.</strong> Salesforce is Premium on both tools. On Power Automate this is an automated flow, so one Premium license for the owner covers it.</p>\n<table>\n<thead>\n<tr>\n<th>Runs a month</th>\n<th>Zapier tier</th>\n<th>Zapier a year</th>\n<th>Power Automate a year</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>50</td>\n<td>Pro, 750 tasks</td>\n<td>$239.88</td>\n<td>$180.00</td>\n</tr>\n<tr>\n<td>300</td>\n<td>Pro, 750 tasks</td>\n<td>$239.88</td>\n<td>$180.00</td>\n</tr>\n<tr>\n<td>1,000</td>\n<td>Pro, 2,000 tasks</td>\n<td>$588.00</td>\n<td>$180.00</td>\n</tr>\n<tr>\n<td>5,000</td>\n<td>Pro, 10,000 tasks</td>\n<td>$1,548.00</td>\n<td>$180.00</td>\n</tr>\n</tbody>\n</table>\n<p>Power Automate’s limit is actions per day, not money. Microsoft counts “every trigger and action execution” as one, and the <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">licensing FAQ</a> sets 6,000 a day for Microsoft 365 users and 40,000 for Premium. At 5,000 runs a month, flows 1 and 3 use about 500 a day, so neither comes close.</p>\n<p>One license covers many flows. The $180 for flow 3 also pays for every other automated flow the same owner builds with premium connectors. A second Zapier flow adds tasks to the same bill.</p>\n<h2 id=\"where-power-automate-costs-more-than-its-price\">Where Power Automate costs more than its price</h2>\n<p>Power Automate on Microsoft 365 rights runs on what Microsoft calls the <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Low performance profile</a>. A failed action gets “up to two retries” there, against 12 on Premium. A flow without a trigger for 90 days “might be turned off”, unless its owner has a premium license.</p>\n<p>If the owner leaves the company, the flow “reverts to the Low performance profile”. A premium flow whose owner loses the Premium license is downgraded, and turned off after 14 days if nobody acts, the FAQ says.</p>\n<p>The connector gaps cost time too. The HubSpot connector names an independent publisher and carries Microsoft’s preview label, which marks a connector not yet generally available. If your CRM is on that list, you pay $180 a year and still depend on that connector.</p>\n<p>Zapier’s costs are easier to see. Every action is a task, so a long flow at high volume climbs tiers fast: flow 1 at 5,000 runs costs $1,548 a year. When tasks run out, Zapier either moves you to pay-per-task billing or pauses your Zaps until the next period. A Professional plan has one seat; sharing connections across a team means the Team plan, from $69 a month.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision starts with a list of the apps in the flow, not with the tools’ feature pages.</p>\n<ul>\n<li><strong>Every app is Standard on Power Automate:</strong> build it there. On Microsoft 365 it costs nothing extra at any of our volumes. Add a co-owner so the flow survives a departure.</li>\n<li><strong>One app is Premium on Power Automate:</strong> compare $180 a year for one owner license with Zapier’s tier for your task count. Zapier comes out cheaper only on its Free plan: one action, at most 100 tasks a month and no app Zapier marks Premium. Its cheapest paid tier, $239.88 a year, costs more than one license. Check Microsoft’s multiplexing example before you rely on one license: if the flow emails the people who start it, they need licenses too.</li>\n<li><strong>The flow must start on a new Excel row:</strong> Zapier does that directly, if the workbook is on OneDrive for Business or SharePoint. In Power Automate, move the input to a SharePoint list or Microsoft Forms.</li>\n<li><strong>Your app has no Microsoft connector, or only one from an independent publisher</strong> (QuickBooks Online, HubSpot): Zapier is the simpler route, on a paid plan for QuickBooks. Our <a href=\"https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/\">Zapier, Make and n8n price comparison</a> shows whether a third tool is cheaper.</li>\n<li><strong>The other side is Google, not a third-party app:</strong> read our <a href=\"https://28creation.com/articles/apps-script-vs-power-automate/\">Apps Script and Power Automate comparison</a> first.</li>\n</ul>\n<p>Before you build, run the flow through our <a href=\"https://28creation.com/articles/automation-payback-formula/\">payback formula</a>. A free tool still costs the hours to fix it. For a worked Power Automate flow on the free license, see <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments automatically</a>. More is under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Write the flow’s apps in one column and their class in the next. Microsoft’s own column decides most of these choices before the Zapier price matters.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read Microsoft’s connector reference pages for 35 apps, its licensing FAQ, limits page and pricing page, and Zapier’s pricing page and app pages, all on October 21, 2026. The connector classes and Zapier prices were taken from those pages with a script, and the three flows were priced by a second script we ran on those figures. The flows are hypothetical: we did not build or run them on either tool, because we have no Microsoft 365 tenant or paid Zapier account for testing.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">Power Automate licensing FAQ</a> (updated September 22, 2026; read October 21, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Limits of automated, scheduled, and instant flows</a> (updated July 17, 2026; read October 21, 2026)</li>\n<li>Microsoft, <a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">Power Automate pricing</a> (read October 21, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/connector-reference/connector-reference-powerautomate-connectors\">List of Power Automate connectors</a> and the reference page of each connector in the table, among them <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Excel Online (Business)</a> (read October 21, 2026)</li>\n<li>Zapier, <a href=\"https://zapier.com/pricing\">Plans and pricing</a> (read October 21, 2026)</li>\n<li>Zapier, app pages for each app in the table, among them <a href=\"https://zapier.com/apps/excel/integrations\">Microsoft Excel</a> and <a href=\"https://zapier.com/apps/sharepoint/integrations\">Microsoft SharePoint</a> (read October 21, 2026)</li>\n</ul>\n",
      "summary": "Microsoft 365 already pays for flows between Outlook, Excel, SharePoint and Teams. We checked 35 apps on both tools and priced three flows.",
      "image": "https://28creation.com/og/power-automate-vs-zapier-microsoft-365.jpg",
      "date_published": "2026-10-24T15:00:00.000Z",
      "date_modified": "2026-10-24T15:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/nextcloud-collabora-built-in-flaw-cisa-kev/",
      "url": "https://28creation.com/articles/nextcloud-collabora-built-in-flaw-cisa-kev/",
      "title": "Nextcloud Collabora flaw joins CISA list; 29 and older get no fix",
      "content_html": "<h2>The short read</h2><p><strong>On Nextcloud 29, the newest built-in Collabora app on offer is June&apos;s 26.4.104, below the 26.4.303 fix.</strong></p><ul><li>CISA added CVE-2026-110447, a login-free command injection in Nextcloud&apos;s built-in Collabora server app, to its exploited list on October 23.</li></ul><h3>What you can do with this</h3><blockquote><p>Run occ app:list on your Nextcloud server. If richdocumentscode is there, update it to 26.4.303 or later, or remove it; disabling it does not close the hole.</p></blockquote><hr><p>CISA added CVE-2026-110447 to its <a href=\"https://www.cisa.gov/known-exploited-vulnerabilities-catalog\">known exploited vulnerabilities catalog</a> on October 23, with an October 26 deadline for US federal agencies. It lets anyone, without logging in, run commands on a Nextcloud server that has the built-in Collabora office app, <code>richdocumentscode</code>. On Nextcloud 29 or older, no patched build exists.</p>\n<h2 id=\"a-restart-opens-the-door\">A restart opens the door</h2>\n<p><a href=\"https://github.com/CollaboraOnline/online/security/advisories/GHSA-g3vx-3m76-fghh\">Collabora’s advisory</a>, published October 14, rates the bug critical, 9.2 on CVSS v4. A crafted request to the app’s <code>proxy.php</code> makes the web server run an operating system command as the web server user. It works only while the built-in server is not running, for example after a container restart, an upgrade or a crash. Every version below 26.4.303 is affected.</p>\n<p>Two details matter more than the score. Disabling the app does not stop the attack; you have to upgrade or remove it. And standalone Collabora Online servers are not affected. <a href=\"https://www.bleepingcomputer.com/news/security/hackers-exploit-collabora-rce-flaw-to-backdoor-nextcloud-servers/\">BleepingComputer</a>, citing Collabora, adds that Docker, RPM and DEB installs and Nextcloud All-in-One are safe too.</p>\n<h2 id=\"attacks-ran-for-a-month-before-the-advisory\">Attacks ran for a month before the advisory</h2>\n<p>The fix came first and the warning came late. BleepingComputer reports that the fix was committed on September 9, version 26.4.303 shipped on September 11, and administrators reported attacks from around September 10. On October 3, a user posted on the <a href=\"https://help.nextcloud.com/t/to-collabora-built-in-user-update-asap/250516\">Nextcloud forum</a> that the flaw “is already being exploited” and that some admins had found web shells. The CVE followed on October 14.</p>\n<p>Admins told BleepingComputer the attackers planted PHP web shells and searched for password databases, password manager files and crypto wallets. One forum user wrote on October 16 that Nextcloud’s integrity scan flagged an extra file, <code>Nx_33143.php</code>, in the app’s folder, which they called a simple backdoor. Nextcloud told BleepingComputer it had not seen widespread exploitation. CISA calls its catalog the authoritative list of flaws “exploited in the wild,” so this one now counts.</p>\n<h2 id=\"nextcloud-29-is-stuck-on-an-unpatched-build\">Nextcloud 29 is stuck on an unpatched build</h2>\n<p>We checked the Nextcloud app store’s <a href=\"https://apps.nextcloud.com/apps/richdocumentscode/releases\">release list</a> and its API on October 24. Version 26.4.303 and every later build require Nextcloud 30 or newer. A server on Nextcloud 29 is offered 26.4.104, released June 7, which is below the fix. The advisory says Nextcloud 29 and earlier are end of life and tells those users to upgrade Nextcloud or remove the app.</p>\n<p>So an app update will not save an older server. That is the trap: the store has nothing newer to offer it, and the server stays exposed.</p>\n<p>The trade-off is real. Removing the app ends in-browser editing until you set up Collabora some other way, such as the Docker image. Upgrading Nextcloud across major versions takes longer than an app update.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Check for the app.</strong> Run <code>sudo -E -u www-data php occ app:list</code>, per <a href=\"https://docs.nextcloud.com/server/latest/admin_manual/occ_apps.html\">Nextcloud’s occ docs</a>, and look for <code>richdocumentscode</code>. If only <code>richdocuments</code> is there, the forum post says you are not affected.</li>\n<li><strong>On Nextcloud 30 or newer,</strong> run <code>occ app:update richdocumentscode</code> and confirm 26.4.303 or later. The store’s newest build was 26.4.502 on October 23.</li>\n<li><strong>On Nextcloud 29 or older,</strong> run <code>occ app:remove richdocumentscode</code> today, then plan the Nextcloud upgrade.</li>\n<li><strong>Treat a patched server as possibly compromised.</strong> If an old version ran since early September, look for PHP files you did not put in <code>apps/richdocumentscode</code> and change any passwords the web server user could read. CISA’s <a href=\"https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk\">BOD 26-04</a> asks agencies for this same step, a “forensic triage of the asset to assess whether the system is compromised.”</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>CISA, Known Exploited Vulnerabilities Catalog, CVE-2026-110447 (added October 23, 2026; read October 24): <a href=\"https://www.cisa.gov/known-exploited-vulnerabilities-catalog\">https://www.cisa.gov/known-exploited-vulnerabilities-catalog</a></li>\n<li>Collabora, GHSA-g3vx-3m76-fghh, “Unauthenticated OS command injection in the richdocumentscode proxy,” October 14, 2026: <a href=\"https://github.com/CollaboraOnline/online/security/advisories/GHSA-g3vx-3m76-fghh\">https://github.com/CollaboraOnline/online/security/advisories/GHSA-g3vx-3m76-fghh</a></li>\n<li>BleepingComputer (Bill Toulas), “Hackers exploit Collabora RCE flaw to backdoor Nextcloud servers,” October 15, 2026: <a href=\"https://www.bleepingcomputer.com/news/security/hackers-exploit-collabora-rce-flaw-to-backdoor-nextcloud-servers/\">https://www.bleepingcomputer.com/news/security/hackers-exploit-collabora-rce-flaw-to-backdoor-nextcloud-servers/</a></li>\n<li>Nextcloud community forum, “To Collabora Built-in User: Update ASAP,” October 3, 2026: <a href=\"https://help.nextcloud.com/t/to-collabora-built-in-user-update-asap/250516\">https://help.nextcloud.com/t/to-collabora-built-in-user-update-asap/250516</a></li>\n<li>Nextcloud App Store, richdocumentscode releases (read October 24, 2026): <a href=\"https://apps.nextcloud.com/apps/richdocumentscode/releases\">https://apps.nextcloud.com/apps/richdocumentscode/releases</a></li>\n<li>Nextcloud admin manual, occ app commands: <a href=\"https://docs.nextcloud.com/server/latest/admin_manual/occ_apps.html\">https://docs.nextcloud.com/server/latest/admin_manual/occ_apps.html</a></li>\n<li>CISA, BOD 26-04, Prioritizing Security Updates Based on Risk: <a href=\"https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk\">https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk</a></li>\n</ul>\n",
      "summary": "Attackers use a flaw in Nextcloud's built-in Collabora server to plant web shells. Nextcloud 29 and older have no patched build: upgrade or remove the app.",
      "image": "https://28creation.com/og/nextcloud-collabora-built-in-flaw-cisa-kev.jpg",
      "date_published": "2026-10-24T14:42:04.000Z",
      "date_modified": "2026-10-24T14:42:04.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Work software",
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/bank-statement-pdf-to-excel/",
      "url": "https://28creation.com/articles/bank-statement-pdf-to-excel/",
      "title": "Bank statement PDFs to Excel: 3 free tools tested on 5 layouts",
      "content_html": "<h2>The short read</h2><p><strong>On five sample bank statements we made, one free tool, Camelot, read all 154 transactions exactly.</strong></p><ul><li>When a statement prints a running balance, a script can check each extracted row against it and flag lost rows and wrong amounts.</li></ul><h3>What you can do with this</h3><blockquote><p>Download CSV from your bank first. For PDF-only months, run the script in this guide and fix every row the balance check flags. It will not catch a wrong date or description.</p></blockquote><hr><p>If you copy bank statement PDFs into Excel by hand, a free Python tool can do the copying, and the statement itself can check the result. We made five sample statements in five common layouts and ran three free table readers on them. Camelot read all 154 transactions exactly. The other two each lost whole statements under their default settings.</p>\n<p>The check is the useful part. Where a statement prints a balance after each line, the previous balance plus the amount must equal the next balance.</p>\n<p>A script that tests that on every row catches a dropped line, a misread amount or a debit booked as a credit. This guide shows where each tool broke and why, gives you the script, and says what the check cannot catch. Documentation and prices were read on October 21, 2026.</p>\n<h2 id=\"our-test-five-statements-three-free-tools\">Our test: five statements, three free tools</h2>\n<p>We wrote a Python script that drew five statements as PDFs from transactions we generated, so every extracted row could be scored against the truth. Every name and amount in them is made up. Each one tests a different layout problem:</p>\n<ul>\n<li><strong>S1, a ruled grid</strong>, US dates, separate Withdrawals and Deposits columns, two pages with the header repeated, 48 transactions.</li>\n<li><strong>S2, borderless columns</strong>, one signed Amount column (<code>-687.18</code>), and the date printed only on the first transaction of each day, 26 transactions.</li>\n<li><strong>S3, a UK-style layout</strong> with horizontal rules only, dates without a year (<code>02 Sep</code>), Paid out and Paid in columns, “balance brought forward” and “carried forward” rows, and long descriptions that wrap onto a second line, 34 transactions.</li>\n<li><strong>S4, European formats</strong>, borderless: <code>1.234,56</code>, dates as <code>14.09.2026</code>, and debits written with a trailing minus (<code>421,12-</code>), 22 transactions.</li>\n<li><strong>S5, a ruled grid</strong> with one Amount column where credits carry <code>CR</code> and debits carry nothing, and a balance that goes overdrawn and prints as <code>412.37 DR</code>, 24 transactions.</li>\n</ul>\n<p>Each tool’s tables went through the same parser, which we wrote from the five layouts before running any tool. It finds the header row, maps the columns, turns the amounts into signed numbers and joins wrapped lines. A transaction counted as right only when its date, description, amount and balance all matched.</p>\n<table>\n<thead>\n<tr>\n<th>Tool and settings</th>\n<th>Right, of 154</th>\n<th>Where it failed</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Camelot 2.0.0, lattice, then stream when lattice finds no table</td>\n<td>154</td>\n<td>nowhere</td>\n</tr>\n<tr>\n<td>tabula-py 2.10.0, default settings</td>\n<td>120</td>\n<td>found no table at all in S3</td>\n</tr>\n<tr>\n<td>tabula-py, <code>guess=False, stream=True</code></td>\n<td>0</td>\n<td>read all 154 amounts and balances, but put the date and description in one column</td>\n</tr>\n<tr>\n<td>pdfplumber 0.11.10, default settings</td>\n<td>72</td>\n<td>found tables only in the ruled grids (S1 and S5)</td>\n</tr>\n<tr>\n<td>pdfplumber, <code>text</code> strategy</td>\n<td>1</td>\n<td>cut words in two; split amounts across cells in S5</td>\n</tr>\n</tbody>\n</table>\n<p>Our parser is tuned to these five layouts, so the scores flatter all three tools. Real statements add logos and footers, and some arrive as scans. Read the table as a ranking of how each tool copes with layout, not as an accuracy rate you can expect.</p>\n<h2 id=\"each-tool-fails-at-the-same-thing-finding-the-columns\">Each tool fails at the same thing: finding the columns</h2>\n<p>All three tools read the text a PDF already contains. Their real job is deciding which words belong in which column, and each failure above traces to how a tool makes that guess.</p>\n<p><strong>Lines or whitespace.</strong> pdfplumber’s table finder <a href=\"https://github.com/jsvine/pdfplumber\">defaults to <code>lines</code></a> for both directions, using the page’s drawn lines as cell borders. Statements S2 and S4 have no lines, so it found nothing. S3 has rules between rows but none between columns, and that was not enough either.</p>\n<p>Its <code>text</code> strategy draws imaginary column borders from where words line up. On our files that went worse: it split “Granite Ridge Bank” into “Granite Rid” and “ge Ban”, and split <code>2,955.21 CR</code> across two cells.</p>\n<p><strong>Two modes, and a fallback between them.</strong> Camelot’s <a href=\"https://github.com/camelot-dev/camelot\">lattice</a> mode “detects the grid from the ruled lines”, and its stream mode uses “text-alignment / whitespace heuristics” for borderless tables. Lattice found the grids in S1 and S5 and nothing in S2, S3 and S4. Falling back to stream for those three is why Camelot read everything.</p>\n<p><strong>The guessed table area.</strong> tabula-py’s <code>guess</code> option, on by default, makes it <a href=\"https://tabula-py.readthedocs.io/en/latest/tabula.html\">“guess the portion of the page to analyze”</a>. On S3 the guess found no table. With <code>guess=False</code> it read every amount and balance in all five statements, but it merged the date and description columns.</p>\n<p>Giving it the column borders by hand fixed that. With <code>columns=[95, 330, 405, 480]</code>, the x positions in points of the borders between S3’s columns, it read all 34 S3 transactions. The <code>columns</code> option takes <a href=\"https://tabula-py.readthedocs.io/en/latest/tabula.html\">“X coordinates of column boundaries”</a>, so you need one list per bank layout.</p>\n<p>One more trap showed up in our own checks. On a version of S2 printed newest first, Camelot’s stream mode returned the same table twice, overlapping, which doubled 26 transactions to 52. The balance check flagged it, and the script below now drops a table whose rows were already read on the same page.</p>\n<h2 id=\"the-running-balance-is-a-free-answer-key\">The running balance is a free answer key</h2>\n<p>A statement with a running balance checks itself in a way an invoice does not. Each row’s balance must equal the previous balance plus that row’s amount. The first row checks against the opening balance and the last against the closing balance, when the statement prints them.</p>\n<p>To see what that check catches, we took Camelot’s correct output for S1 and planted one fault at a time, then ran the check.</p>\n<table>\n<thead>\n<tr>\n<th>Fault planted in S1’s 48 rows</th>\n<th>Flagged by the balance check</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>One row in the middle dropped</td>\n<td>yes, at the next row</td>\n</tr>\n<tr>\n<td>The last row dropped</td>\n<td>yes, against the closing balance</td>\n</tr>\n<tr>\n<td>One amount off by $9.00</td>\n<td>yes</td>\n</tr>\n<tr>\n<td>A withdrawal read as a deposit</td>\n<td>yes</td>\n</tr>\n<tr>\n<td>Two rows swapped</td>\n<td>yes, at three rows</td>\n</tr>\n<tr>\n<td>One row read twice</td>\n<td>yes</td>\n</tr>\n<tr>\n<td>One date wrong</td>\n<td>no</td>\n</tr>\n<tr>\n<td>One description cut short</td>\n<td>no</td>\n</tr>\n</tbody>\n</table>\n<p>The same check worked on the tools’ real failures. pdfplumber’s <code>text</code> strategy lost 10 of S3’s 34 rows in 7 gaps, and the check flagged the row after each gap. It also flagged all 8 rows where that strategy got an amount wrong in S5.</p>\n<p>What the check cannot see matters just as much. It tests money, not words or dates. pdfplumber’s text strategy read all 48 amounts and balances in S1 correctly while cutting 47 of the 48 descriptions, and the check passed it.</p>\n<p>So the check tells you the money is complete. Read a sample of descriptions and dates yourself before you trust a statement.</p>\n<h2 id=\"1-ask-your-bank-for-a-csv-first\">1. Ask your bank for a CSV first</h2>\n<p>The fastest PDF converter is the one you do not need. If your online banking offers a download of your activity, start there. QuickBooks’ own help page tells you to try a <a href=\"https://quickbooks.intuit.com/learn-support/en-global/help-article/bank-transactions/format-csv-files-excel-bank-transactions-quickbooks/L4BjLWckq_ROW_en\">QFX, QBO or OFX file</a> if your bank offers one of those but not CSV.</p>\n<p>Use the PDF route for what downloads do not cover: older months, a closed account, or statements a client sends you. Check your bank’s download page for how far back it goes. We did not test any bank’s export, because that needs a real account.</p>\n<h2 id=\"2-check-that-each-pdf-has-a-text-layer\">2. Check that each PDF has a text layer</h2>\n<p>Open a statement and try to select a word. Camelot’s documentation gives the same test: <a href=\"https://github.com/camelot-dev/camelot\">“If you can click and drag to select text in your table in a PDF viewer, then your PDF is text-based.”</a> Its built-in parsers need that text, and pdfplumber says it works best on <a href=\"https://github.com/jsvine/pdfplumber\">machine-generated rather than scanned</a> PDFs.</p>\n<p>For a scan, add a text layer first with <code>ocrmypdf scan.pdf scan.pdf</code>. OCRmyPDF uses the open-source Tesseract engine, and its own page warns that <a href=\"https://ocrmypdf.readthedocs.io/en/latest/introduction.html\">“the OCR accuracy may not match that of commercial OCR solutions”</a>. Our <a href=\"https://28creation.com/articles/pdf-invoices-to-excel/\">invoice test</a> found two stray characters on a clean, computer-made scan, so run the balance check on every OCR’d statement. The script below reports a PDF with no text and skips it rather than guessing.</p>\n<h2 id=\"3-run-the-script-on-a-folder-of-statements\">3. Run the script on a folder of statements</h2>\n<p>The script reads every PDF in a folder and writes <code>statements.xlsx</code> with two sheets. Transactions has one row per transaction with a check column. Files has one row per statement with how many rows need a look. It uses Camelot for the tables and pdfminer.six to read the year and the opening and closing balances from the page text.</p>\n<p>We ran it on Linux with Python 3.11. Camelot’s default backend needs <a href=\"https://github.com/camelot-dev/camelot\">no system dependencies</a>, and our test machine had no Ghostscript. Set it up with:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"bash\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">python3</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> -m</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> venv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> statements-env</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">statements-env/bin/pip</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> install</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> camelot-py</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> pdfminer.six</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> pandas</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> openpyxl</span></span></code></pre>\n<p>Save this as <code>statement_to_excel.py</code>:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"Turn PDF bank statements into one Excel sheet of transactions, and check</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">every row against the running balance the bank printed.</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Usage: python statement_to_excel.py &lt;folder&gt;\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re, sys</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pathlib </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Path</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> camelot, pandas </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdfminer.high_level </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> extract_text</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">ROLES</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"debit\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"withdraw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">paid out</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">debit</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">money out\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">         (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"credit\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"deposit</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">paid in</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">credit</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">money in\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">         (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">^(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">date</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">booking</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">posted</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"desc\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"desc</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">detail</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">transaction</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">text</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">payee</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">memo\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)]</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> money</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.sub(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">$€£</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">USD</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">EUR</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">GBP\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s)).upper()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    sign </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.endswith((</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"DR\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"OD\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)): sign, s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s[:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.endswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"CR\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s[:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.endswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): sign, s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s[:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.startswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): sign, s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.startswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"(\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.endswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\")\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): sign, s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.fullmatch(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{1,3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">,</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s): s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\".\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\".\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.fullmatch(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{1,3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">,</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}|</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s): s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sign </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> float</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> day</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s, year, day_first):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.sub(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s).strip()</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    fmts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/%m/%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%m/</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> day_first </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%m/</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/%m/%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> f </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> fmts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.%m.%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%Y-%m-</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%b</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> %d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, %Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> %b</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> %Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime.strptime(s, f).date()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">pass</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> f </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%b</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> %d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> %b</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%m/</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># no year printed: take it from the statement</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime.strptime(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">s</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> {</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">year</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, f </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" %Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).date()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">pass</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> tables</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf, flavor):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    out, seen </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> camelot.read_pdf(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">pages</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"all\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">flavor</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">flavor):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">tuple</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t.df.values.tolist() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> any</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(c.strip() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r)]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> all</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((t.page, r) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> seen </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows): </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#6A737D;--shiki-dark:#768390\">  # stream mode can find a table twice</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        seen.update((t.page, r) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows); out.append([</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">list</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows])</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> roles</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    found </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {}</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i, cell </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> enumerate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> role, pat </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ROLES</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> role </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">not</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> found </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(pat, cell.strip(), re.I):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                found[role] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">break</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> found </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">} </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> found.keys() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> parse</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(tables, year):</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"\"\"tables: lists of rows of cell strings, in page order. Returns the transactions</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    and the first \"brought forward\" balance, if a table has one.\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    tables </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [[[re.sub(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, c).strip() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tables]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tables </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    head </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> next</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(roles, rows) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r), {})</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    column </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> lambda</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> k: [r[head[k]] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> k </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> head </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> head[k]]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    day_first </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> any</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">int</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(m.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 12</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> column(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># 25/09/2026 is day first</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (m </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.match(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(\\d\\d)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d\\d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{4}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, c)))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    has_cr </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> any</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">CR</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, c.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), re.I) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> column(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    cols, out, last_date, brought </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, [], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tables </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> roles(row)):  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># a new table or a header row</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            cols, joinable </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> cols, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> cols </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> max</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(cols.values()): </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        get </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> lambda</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> k: row[cols[k]] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> k </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> cols </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        desc, bal </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"desc\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), money(get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"amount\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> cols:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> money(get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> has_cr:  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># CR marks credits, so an unmarked amount is a debit</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> abs</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(amt) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"CR</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), re.I) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">abs</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(amt)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            out_, in_ </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> money(get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"debit\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)), money(get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"credit\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out_ </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> in_ </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (in_ </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (out_ </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> day(get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), year, day_first) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> get(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> last_date </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (last_date </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d).days </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 180</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># a December to January statement</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d.replace(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">year</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">d.year </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> bal </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> brought </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"brought forward\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, desc, re.I):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                brought </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> bal</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> joinable </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> bal </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> desc </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d:  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># a wrapped description line</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                out[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"description\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> desc; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            joinable </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#6A737D;--shiki-dark:#768390\">  # balance brought or carried forward, page totals, other text</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        out.append({</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> last_date, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"description\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: desc, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(amt, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: bal})</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        last_date, joinable </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> d </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> last_date, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out, brought</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> balances</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(text):</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"\"\"Opening and closing balance as printed on the statement, if found.\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    num </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">EUR</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">USD</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">GBP</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">$€£</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[\\d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">.,</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d(?:\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">CR</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">DR</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">))</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    o </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">opening</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">beginning</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">old</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">previous</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> balance\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> num </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"|balance brought forward\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> num, text, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">closing</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">ending</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">new</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> balance\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> num, text, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (money(o.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> o.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> o </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), (money(c.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> check</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(txns, opening</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, closing</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"\"\"Flag every row where the previous balance plus the amount is not the printed balance.\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    prev </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> opening</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"no balance read\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> prev </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"first row, no opening balance to check against\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        elif</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(prev </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]: t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"ok\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"break: expected </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">prev </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'amount'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:.2f</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        prev </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> closing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"balance\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> closing:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        note </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"last balance differs from closing balance </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">closing</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:.2f</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> note </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"ok\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"; \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> note</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> __name__</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"__main__\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    folder </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Path(sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]); pdfs </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(folder.glob(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"*.pdf\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdfs: sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"No PDF files in </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">folder</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rows_out, summary </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [], []</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdf </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdfs:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> extract_text(pdf)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.strip():</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                summary.append({</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"rows\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"problems\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">                                \"result\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"no text layer: run ocrmypdf on it first\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            year </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (re.findall(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\b</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">20</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d\\d\\b</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, text) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [datetime.now().year])[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            opening, closing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> balances(text)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> flavor </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"lattice\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"stream\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># ruled tables first, then whitespace columns</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                txns, brought </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parse(tables(pdf, flavor), year)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">break</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">][</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                txns.reverse()  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># newest first: check the balances oldest first</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            txns </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> check(txns, opening </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> opening </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> brought, closing)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Exception</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> e:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            summary.append({</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"rows\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"problems\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"result\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"error: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">e</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        problems </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sum</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(t[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"ok\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        summary.append({</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"rows\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(txns), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"problems\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: problems,</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">                        \"result\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"no rows found\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"ok\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> problems </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"check by hand\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">})</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rows_out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [{</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name} </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> txns]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> folder </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"statements.xlsx\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    with</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd.ExcelWriter(out) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> xl:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        pd.DataFrame(rows_out).to_excel(xl, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">sheet_name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Transactions\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        pd.DataFrame(summary).to_excel(xl, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">sheet_name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Files\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pd.DataFrame(summary).to_string(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rows_out)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> transactions written to </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">out</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span></code></pre>\n<p>Run it with <code>statements-env/bin/python statement_to_excel.py statements/</code>. On a folder holding our five statements and a scanned copy of S2’s first page, it printed:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span>       file  rows problems                                  result</span></span>\n<span class=\"line\"><span>     s1.pdf    48        0                                      ok</span></span>\n<span class=\"line\"><span>s2-scan.pdf     0          no text layer: run ocrmypdf on it first</span></span>\n<span class=\"line\"><span>     s2.pdf    26        0                                      ok</span></span>\n<span class=\"line\"><span>     s3.pdf    34        0                                      ok</span></span>\n<span class=\"line\"><span>     s4.pdf    22        0                                      ok</span></span>\n<span class=\"line\"><span>     s5.pdf    24        0                                      ok</span></span>\n<span class=\"line\"><span></span></span>\n<span class=\"line\"><span>154 transactions written to statements/statements.xlsx</span></span></code></pre>\n<p>We compared the Transactions sheet with our generated data: all 154 rows matched on every field. The script rewrites <code>statements.xlsx</code> on every run, so close the workbook in Excel first. A PDF that crashes Camelot gets an <code>error:</code> row instead of stopping the run, and an empty folder stops with a message.</p>\n<h2 id=\"4-fix-what-the-check-column-flags\">4. Fix what the check column flags</h2>\n<p>Any value other than <code>ok</code> in the check column is a row to look at. These are the causes we know of, and the setting that fixes each:</p>\n<ul>\n<li><strong><code>first row, no opening balance to check against</code>.</strong> The script looks for “opening”, “beginning”, “old” or “previous balance” followed by an amount, or a “balance brought forward” row. If your bank writes “Balance on 1 September”, add that wording to the pattern in <code>balances()</code>.</li>\n<li><strong>A break at every row.</strong> The signs are probably wrong. The script reads a leading or trailing minus, brackets or <code>DR</code> as a debit. When any amount in an Amount column ends in <code>CR</code>, it treats every unmarked amount as a debit. If your bank marks debits some other way, add that marker to <code>money()</code>.</li>\n<li><strong>No rows found.</strong> The header was not recognized. The script needs a header cell matching “Date” (or “Booking” or “Posted”) and one matching “Balance”. Add your bank’s words to <code>ROLES</code>.</li>\n<li><strong>Wrong day and month.</strong> The script reads <code>03/09/2026</code> as March 9 unless some date in the column has a first number above 12, which proves the day comes first. A statement from early in a month can have no such date. If your bank always writes the day first, add the line <code>day_first = True</code> right after the <code>has_cr = ...</code> line in <code>parse()</code>.</li>\n<li><strong>No balance column at all.</strong> Then every row says <code>no balance read</code>. Compare the sum of the amounts with the statement’s own totals instead.</li>\n</ul>\n<p>Dates without a printed year take the first year (2000 to 2099) found in the statement’s text. December-to-January statements are handled by moving a date forward a year when it falls more than 180 days before the row above it.</p>\n<h2 id=\"5-load-the-result-into-your-accounting-software\">5. Load the result into your accounting software</h2>\n<p>QuickBooks Online accepts a CSV in <a href=\"https://quickbooks.intuit.com/learn-support/en-global/help-article/bank-transactions/format-csv-files-excel-bank-transactions-quickbooks/L4BjLWckq_ROW_en\">two shapes</a>: three columns (Date, Description, Amount) or four (Date, Description, Credit, Debit). The Transactions sheet already holds the three-column version. Delete the file, balance and check columns, then save the sheet as CSV.</p>\n<p>The same page lists what makes an upload fail. Keep every date in one format. Remove any zeros and leave those cells blank. Fix descriptions that are only numbers.</p>\n<p>On a Mac, save as a Windows CSV.</p>\n<h2 id=\"when-to-pay-for-extraction-instead\">When to pay for extraction instead</h2>\n<p>The free route costs setup time, and some upkeep per bank layout. A new bank means a new header word in <code>ROLES</code> or a new date format, and each change needs a test run on a statement you know. Our <a href=\"https://28creation.com/articles/automation-payback-formula/\">payback formula</a> counts that upkeep against the hours of typing it saves.</p>\n<p>Paid table extraction removes the layout work but not the check. Amazon’s AnalyzeDocument with the Tables feature costs <a href=\"https://aws.amazon.com/textract/pricing/\">$0.015 a page</a> for the first million pages a month in US West (Oregon). New customers get 100 pages a month free for three months.</p>\n<p>A hypothetical year of 12 three-page statements would cost 54 cents at that price. We did not test it, because it needs a paid account. Its output still needs the same mapping from columns to amounts and the same balance check.</p>\n<p>Two other routes have limits worth knowing before you choose. Excel’s own PDF import, through Power Query, is listed for <a href=\"https://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16\">Excel on Windows</a>, with .NET Framework 4.5 or later, and not for Excel for Mac. Our <a href=\"https://28creation.com/articles/pdf-invoices-to-excel/\">invoice guide</a> covers its settings; we have not tested it on statements.</p>\n<p>Free converter websites take the file onto someone else’s server. A statement carries your account number and every payment you made that month, so read the site’s terms before you upload one.</p>\n<p>Our view: download CSV where the bank offers it, run the script on the PDF-only months, and treat any row the balance check flags as unread until a person has looked at it.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>On October 21, 2026, we generated five sample bank statements as PDFs with the ReportLab library from transactions we made up, plus a newest-first copy of one and an image-only copy of one page to stand in for a scan. We ran Camelot, tabula-py and pdfplumber on them through one parser and scored all 154 transactions, planted eight faults in one statement to test the balance check, and ran the script in this guide on the same files, on Linux with Python 3.11. We read each tool’s documentation, QuickBooks’ CSV import rules, Amazon’s Textract pricing and Microsoft’s Excel data source list that day. We did not test any bank’s own export, any paid API or Excel’s PDF import, and our statements are cleaner than real ones, so treat the scores as a best case.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Camelot, <a href=\"https://github.com/camelot-dev/camelot\">README</a> (lattice and stream flavors, text-based PDFs, dependencies), version 2.0.0</li>\n<li>tabula-py, <a href=\"https://tabula-py.readthedocs.io/en/latest/tabula.html\">read_pdf reference</a> (<code>guess</code>, <code>stream</code>, <code>columns</code>), version 2.10.0</li>\n<li>pdfplumber, <a href=\"https://github.com/jsvine/pdfplumber\">README</a> (table strategies, scanned PDFs), version 0.11.10</li>\n<li>OCRmyPDF, <a href=\"https://ocrmypdf.readthedocs.io/en/latest/introduction.html\">Introduction</a></li>\n<li>QuickBooks, <a href=\"https://quickbooks.intuit.com/learn-support/en-global/help-article/bank-transactions/format-csv-files-excel-bank-transactions-quickbooks/L4BjLWckq_ROW_en\">Format CSV files in Excel to get bank transactions into QuickBooks</a>, read October 21, 2026</li>\n<li>Amazon Web Services, <a href=\"https://aws.amazon.com/textract/pricing/\">Amazon Textract pricing</a>, read October 21, 2026</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16\">Power Query data sources in Excel versions</a>, read October 21, 2026</li>\n<li>28creation test, October 21, 2026: five generated statements, 154 transactions, scored by our own script</li>\n</ul>\n",
      "summary": "On five sample statements we made, Camelot read all 154 transactions. The running balance catches lost rows and wrong amounts, not wrong dates.",
      "image": "https://28creation.com/og/bank-statement-pdf-to-excel.jpg",
      "date_published": "2026-10-24T07:00:00.000Z",
      "date_modified": "2026-10-24T07:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/brotli-flaw-python-envoy-istio-bun/",
      "url": "https://28creation.com/articles/brotli-flaw-python-envoy-istio-bun/",
      "title": "Brotli flaw spreads past Node.js to Python, Envoy, Istio and Bun",
      "content_html": "<h2>The short read</h2><p><strong>Installing the Brotli package alone made requests, httpx and aiohttp start asking servers for Brotli in our check.</strong></p><ul><li>The memory bug first patched in Node.js on October 21 lives in Google&apos;s Brotli decoder, and Python, Envoy, Istio and Bun now ship fixes too.</li></ul><h3>What you can do with this</h3><blockquote><p>Run node --version and pip show brotli brotlicffi inside every image you deploy. Move to Node 22.23.4, 24.22.1 or 26.12.1 and Brotli 1.2.1, then rebuild.</p></blockquote><hr><p>The Brotli bug Node.js patched on October 21 is not a Node.js bug. It sits in Google’s Brotli decoder, and <a href=\"https://www.aikido.dev/blog/python-packages-vulnerable-node2shell-brotli-cve-2026-111497\">Aikido Security reported on October 23</a> that Python packages, Envoy, Istio, Bun and bundled copies in curl carry it too. If your services fetch from other servers or accept compressed uploads, every runtime that unpacks Brotli needs a patched build.</p>\n<h2 id=\"one-decoder-many-products\">One decoder, many products</h2>\n<p><a href=\"https://nodejs.org/en/blog/vulnerability/october-2026-security-releases\">Node.js rated CVE-2026-111497 high</a> and fixed it in 22.23.4, 24.22.1 and 26.12.1. Its advisory explains the cause. An invalid block switch was reported as a need for more input, not as an error. So the decoder kept writing bytes into an eight-byte buffer.</p>\n<p>Attacker-supplied Brotli data “can lead to a crash, disclosure of process memory, or remote code execution.” Node.js credits OpenAI Codex Security with the report.</p>\n<p>The <a href=\"https://github.com/advisories/GHSA-9v8h-qqgm-4m35\">GitHub advisory for the Python packages</a>, published October 22, puts the flaw in Google Brotli 1.0.3 through 1.2.0 “on all platforms” and scores it critical, 9.2 on CVSS v4. Fixed releases are Brotli 1.2.1, 1.1.1 and 1.0.10, and brotlicffi 1.2.1.0. <a href=\"https://pypi.org/project/Brotli/#history\">PyPI shows</a> Brotli 1.2.1 went up on October 22, almost a year after 1.2.0.</p>\n<p>Envoy followed on October 23. Its <a href=\"https://github.com/envoyproxy/envoy/security/advisories/GHSA-3wvj-j6qm-62mr\">advisory</a> covers “Envoy deployments that have Brotli decompressor configured,” with fixes in 1.40.0, 1.39.3, 1.38.6, 1.37.8 and 1.36.12. Aikido lists Istio 1.29.9, 1.30.6 and 1.31.2 and Bun 1.4.4 as fixed.</p>\n<p>According to Aikido, OpenAI found the bug in February and the source was fixed then. It was not flagged as a security issue, and no release carried the fix until this week.</p>\n<h2 id=\"why-it-reaches-code-that-never-mentions-brotli\">Why it reaches code that never mentions Brotli</h2>\n<p>Most apps never call a Brotli function themselves. A library does it for them. In Node.js, the built-in <code>fetch()</code> decompresses Brotli responses automatically, <a href=\"https://www.aikido.dev/blog/node2shell-nodejs-brotli-rce-cve-2026-111497\">Aikido’s Node.js write-up</a> says. Express 5 with body-parser 2.x inflates request bodies sent with <code>Content-Encoding: br</code>.</p>\n<p><a href=\"https://www.imperva.com/blog/node2shell-from-brotli-data-to-remote-code-execution/\">Imperva ran code</a> through both paths against the official Node.js 24.22.0 image on Linux arm64. It took several attempts and was tested on arm64 only, and 24.22.1 rejected the same input.</p>\n<p>Python works the same way, and we checked it on October 24. In a fresh Python 3.11 environment, requests 2.34.2, httpx 0.28.1 and aiohttp 3.14.5 each sent <code>Accept-Encoding: gzip, deflate</code>. After <code>pip install brotli==1.2.0</code>, with no code change, all three sent <code>gzip, deflate, br</code>, so any server they call may answer in Brotli.</p>\n<p>Package metadata on PyPI also shows the brotli extras of <a href=\"https://pypi.org/project/urllib3/\">urllib3</a> (<code>brotli&gt;=1.2.0</code>) and <a href=\"https://pypi.org/project/aiohttp/\">aiohttp</a> (<code>Brotli&gt;=1.2</code>) still accept the vulnerable 1.2.0, so an old lockfile keeps it.</p>\n<p>The Python research “didn’t pursue memory corruption or code execution in the Python extension,” Aikido says. For Node.js, no attack in the wild had been reported as of October 22. Aikido still calls it urgent, and we agree: the Brotli package gets 82.8 million downloads a month, by its count.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ul>\n<li><strong>Check inside the images you ship.</strong> Run <code>node --version</code> and <code>pip show brotli brotlicffi</code> in each container and server. Imperva notes package inventories can miss bundled runtimes.</li>\n<li><strong>Upgrade and rebuild.</strong> Node.js 22.23.4, 24.22.1 or 26.12.1; Brotli 1.2.1 or brotlicffi 1.2.1.0; the proxies and Bun at the versions above. Node.js 20.x, 23.x and 25.x get no fix, says Aikido.</li>\n<li><strong>Block Brotli while you wait.</strong> Reject requests whose <code>Content-Encoding</code> includes <code>br</code> at the proxy or load balancer. For outbound calls, Aikido says to drop <code>br</code> from <code>Accept-Encoding</code> and refuse Brotli responses. “Changing Accept-Encoding alone is insufficient,” it warns.</li>\n<li><strong>Or uninstall the Python packages.</strong> Without Brotli, our test clients fell back to gzip and deflate.</li>\n</ul>\n<p>The trade-off is that a blanket block may break clients that legitimately upload compressed data, so Imperva advises testing the rule against normal traffic first. Apache and Nginx do not decode Brotli by default, Aikido says. We read that as a warning: the web server in front passes Brotli bodies on to the app unless you add the rule.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Node.js, October 2026 security releases: <a href=\"https://nodejs.org/en/blog/vulnerability/october-2026-security-releases\">https://nodejs.org/en/blog/vulnerability/october-2026-security-releases</a></li>\n<li>GitHub Advisory GHSA-9v8h-qqgm-4m35, Brotli and brotlicffi: <a href=\"https://github.com/advisories/GHSA-9v8h-qqgm-4m35\">https://github.com/advisories/GHSA-9v8h-qqgm-4m35</a></li>\n<li>Envoy advisory GHSA-3wvj-j6qm-62mr: <a href=\"https://github.com/envoyproxy/envoy/security/advisories/GHSA-3wvj-j6qm-62mr\">https://github.com/envoyproxy/envoy/security/advisories/GHSA-3wvj-j6qm-62mr</a></li>\n<li>Aikido Security, “Node2Shell evolves to HTTP2Shell”: <a href=\"https://www.aikido.dev/blog/python-packages-vulnerable-node2shell-brotli-cve-2026-111497\">https://www.aikido.dev/blog/python-packages-vulnerable-node2shell-brotli-cve-2026-111497</a></li>\n<li>Aikido Security, Node2Shell in Node.js: <a href=\"https://www.aikido.dev/blog/node2shell-nodejs-brotli-rce-cve-2026-111497\">https://www.aikido.dev/blog/node2shell-nodejs-brotli-rce-cve-2026-111497</a></li>\n<li>Imperva, “Node2Shell: from Brotli data to remote code execution”: <a href=\"https://www.imperva.com/blog/node2shell-from-brotli-data-to-remote-code-execution/\">https://www.imperva.com/blog/node2shell-from-brotli-data-to-remote-code-execution/</a></li>\n<li>PyPI release history for Brotli, urllib3 and aiohttp: <a href=\"https://pypi.org/project/Brotli/#history\">https://pypi.org/project/Brotli/#history</a></li>\n</ul>\n",
      "summary": "CVE-2026-111497 sits in Google's Brotli decoder, so any service that unpacks Brotli data needs a patched build, not only Node.js apps.",
      "image": "https://28creation.com/og/brotli-flaw-python-envoy-istio-bun.jpg",
      "date_published": "2026-10-24T05:41:15.000Z",
      "date_modified": "2026-10-24T05:41:15.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/pdf-invoices-to-excel/",
      "url": "https://28creation.com/articles/pdf-invoices-to-excel/",
      "title": "PDF invoices to Excel: free tools tested, paid APIs priced",
      "content_html": "<h2>The short read</h2><p><strong>On 10 sample invoices we made, two free tools read 55 of 58 line items, all but the scan.</strong></p><ul><li>Amazon, Microsoft and Google sell invoice extraction for 1 to 10 cents a one-page invoice, and find the number and total without label rules.</li></ul><h3>What you can do with this</h3><blockquote><p>If your invoices are digital PDFs from a few suppliers, run the script in this guide and check that each invoice&apos;s lines add up to its total. Send every mismatch to a person.</p></blockquote><hr><p>If invoices arrive as PDFs and someone types them into Excel, free tools can do most of that typing. We made 10 sample invoices in nine layouts and ran four free tools on them. The two best, Camelot and tabula-py, read 55 of 58 line items exactly. All four read nothing from the scanned one; after OCR, the two we retried read it fully.</p>\n<p>The paid route is cheap too. Amazon, Microsoft and Google sell invoice extraction for 1 to 10 cents a page or document, and those models find the invoice number and total for you. This guide shows where each free tool broke and why, gives you a script that writes a folder of invoices into a workbook, and prices the paid APIs for three monthly volumes. Prices and documentation were read on October 20, 2026.</p>\n<h2 id=\"our-test-10-invoices-four-free-tools\">Our test: 10 invoices, four free tools</h2>\n<p>We wrote a Python script that drew 10 invoices as PDFs with known values, so every extracted field could be scored against the truth. The vendors and amounts are made up. The layouts are the kinds a small office receives:</p>\n<ul>\n<li><strong>Ruled tables</strong> (lines around every cell), in seven of them.</li>\n<li><strong>Borderless tables</strong> with columns kept apart by spacing, in two.</li>\n<li><strong>A two-page invoice</strong> with 30 lines and the header repeated on page 2.</li>\n<li><strong>Descriptions that wrap</strong> onto a second line inside one cell.</li>\n<li><strong>European formats</strong>: 1.234,56 and 14.10.2026.</li>\n<li><strong>A credit line</strong> written as (120.00).</li>\n<li><strong>Labels set above their values</strong> in a small box, and an invoice number labelled “Reference” instead of “Invoice No”.</li>\n<li><strong>One scan</strong>: invoice 1 rendered as a 150 dpi image, with no text layer.</li>\n</ul>\n<p>We scored two things. A line item counted only when both its description and its amount came out exactly right; there were 58. Then three header fields per invoice (number, date, total), 30 in all, read with one set of label rules we wrote before running anything.</p>\n<table>\n<thead>\n<tr>\n<th>Tool (version)</th>\n<th>Line items right (of 58)</th>\n<th>Header fields right (of 30)</th>\n<th>Where it failed</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Camelot 2.0.0, lattice then stream</td>\n<td>55</td>\n<td>not tested (tables only)</td>\n<td>the scan</td>\n</tr>\n<tr>\n<td>tabula-py 2.10.0</td>\n<td>55</td>\n<td>not tested (tables only)</td>\n<td>the scan</td>\n</tr>\n<tr>\n<td>pdftotext 24.02 <code>-layout</code> plus our rules</td>\n<td>53</td>\n<td>24</td>\n<td>the scan; split 2 wrapped lines</td>\n</tr>\n<tr>\n<td>pdfplumber 0.11.10, default settings</td>\n<td>45</td>\n<td>24</td>\n<td>the scan; found no table in the 2 borderless invoices</td>\n</tr>\n<tr>\n<td>pdfplumber, <code>text</code> strategy</td>\n<td>53</td>\n<td>not rescored</td>\n<td>the scan; split 2 wrapped lines</td>\n</tr>\n</tbody>\n</table>\n<p>After <a href=\"https://ocrmypdf.readthedocs.io/en/latest/introduction.html\">OCRmyPDF</a> added a text layer to the scan, pdftotext and Camelot each read all 3 of its line items, and our label rules found all 3 header fields. The OCR was not perfect: two unit prices came back as <code>49.00.</code> and <code>80.00:</code>, with stray punctuation. The amounts we scored were clean, which is luck, not design.</p>\n<h2 id=\"tables-break-in-three-predictable-places\">Tables break in three predictable places</h2>\n<p>Every free tool here reads the text a PDF already contains and guesses which words belong in which cell. That guess is where they differ, and each failure in our table traces to one of three causes.</p>\n<p><strong>Lines or spacing.</strong> pdfplumber’s table finder <a href=\"https://github.com/jsvine/pdfplumber\">defaults to <code>lines</code></a> for both directions, so it looks for drawn borders. On our two borderless invoices it found no table at all.</p>\n<p>Switching both settings to <code>text</code> fixed those two. Camelot handles this by having two modes: <a href=\"https://github.com/camelot-dev/camelot\">lattice</a> detects the grid from ruled lines, stream uses text alignment and whitespace. Our script tries lattice first and falls back to stream, which is why Camelot read both borderless invoices. tabula-py is <a href=\"https://github.com/chezou/tabula-py\">a wrapper of tabula-java</a>, which has the <a href=\"https://github.com/tabulapdf/tabula-java\">same two modes</a>; with its default settings it read both borderless invoices.</p>\n<p><strong>Wrapped rows.</strong> When a description runs onto a second line, pdftotext prints two lines of text, and the second has no amount. Our line rules then saw the first half of the description as one item and dropped the rest. Camelot and tabula-py kept “Laptop repair, replaced keyboard and battery, tested” together in one cell. Microsoft documents the same problem for Excel’s PDF connector, where multi-line rows <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/pdf\">may need cleanup</a> with Fill Down or Group By.</p>\n<p><strong>Labels.</strong> Header fields are not in a table, so a rule has to find them by their label. Our rules looked for “Invoice No”, “Invoice #” or “Invoice number” followed by a value on the same line. The invoice that put “Invoice number” above “CL-311” in a box beat them, and so did the one that said “Reference”. That is the gap the paid models close.</p>\n<p>Amazon’s AnalyzeExpense maps “bill number, invoice number, receipt number” to one standard field, <a href=\"https://docs.aws.amazon.com/textract/latest/dg/invoices-receipts.html\"><code>INVOICE_RECEIPT_ID</code></a>, and returns a confidence score for every value.</p>\n<p>One tabula-py trap is worth knowing before you start. By default it hands tables to pandas, which turned <code>49.00</code> into the number <code>49.0</code> in our first run. Our scorer expected text, so that run looked like a failure. Passing <code>pandas_options={\"dtype\": str}</code> keeps every cell exactly as printed, and the scores above use that setting.</p>\n<h2 id=\"1-pick-the-route-by-what-your-invoices-look-like\">1. Pick the route by what your invoices look like</h2>\n<p>The test points to a simple rule. Free tools are enough when your invoices are digital PDFs from a known set of suppliers whose layouts rarely change. You write label rules once per supplier and check the totals.</p>\n<p>Pay for an extraction API when you get invoices from many suppliers, when many arrive as scans or phone photos, or when nobody on the team will maintain a script. Microsoft’s invoice model accepts <a href=\"https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/prebuilt/invoice\">phone-captured images, scanned documents and digital PDFs</a> and supports invoices in 27 languages.</p>\n<p>Do not use either route without a check step. Every tool we ran missed something, and the paid models return confidence scores precisely because they can be wrong too.</p>\n<h2 id=\"2-check-whether-each-pdf-has-a-text-layer\">2. Check whether each PDF has a text layer</h2>\n<p>Run <code>pdftotext -layout invoice.pdf -</code> on one invoice. pdftotext is a <a href=\"https://gitlab.freedesktop.org/poppler/poppler/-/blob/master/utils/pdftotext.1\">command-line tool</a> from the free <a href=\"https://poppler.freedesktop.org/\">Poppler</a> PDF library. If it prints the invoice’s words, the PDF has a text layer and every free tool here can read it. If it prints nothing, the PDF is an image, and all four tools returned nothing on ours.</p>\n<p>For image PDFs, add a text layer first: <code>ocrmypdf scan.pdf scan.pdf</code> writes the OCR text into the file. OCRmyPDF uses the open-source Tesseract engine and says plainly that its accuracy <a href=\"https://ocrmypdf.readthedocs.io/en/latest/introduction.html\">“may not match that of commercial OCR solutions”</a>. Our clean, computer-made scan still picked up two stray characters, so expect more on a crooked phone photo.</p>\n<p>Camelot’s own documentation says its built-in parsers <a href=\"https://github.com/camelot-dev/camelot\">“need a text-based PDF”</a>, and pdfplumber’s says it works best on <a href=\"https://github.com/jsvine/pdfplumber\">machine-generated rather than scanned</a> PDFs.</p>\n<h2 id=\"3-try-excels-pdf-connector-if-you-have-microsoft-365-on-windows\">3. Try Excel’s PDF connector if you have Microsoft 365 on Windows</h2>\n<p>Excel can read PDF tables itself through Power Query. Microsoft lists the PDF source for <a href=\"https://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16\">Excel for Microsoft 365 on Windows</a>, where it requires .NET Framework 4.5 or later. The Mac list of data sources does not include PDF, and Excel for the web reads PDFs only from SharePoint.</p>\n<p>We did not test it, because we had no Windows copy of Excel in our test environment. Here is what Microsoft’s <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/pdf\">connector documentation</a> says:</p>\n<ul>\n<li><strong>Steps:</strong> pick the PDF source, choose the file, then in <strong>Navigator</strong> select a table and click <strong>Load</strong>, or <strong>Transform Data</strong> to clean it first.</li>\n<li><strong>Many files:</strong> to import a folder of PDFs at once, use the <strong>Folder</strong> or <strong>SharePoint Folder</strong> connector and combine the files.</li>\n<li><strong>Options</strong> in the <code>Pdf.Tables</code> function: <code>StartPage</code> and <code>EndPage</code> limit the pages read; <code>MultiPageTables</code> (on by default) joins similar tables on consecutive pages; <code>EnforceBorderLines</code> (off by default) makes drawn lines the cell boundaries.</li>\n</ul>\n<p>The connector’s advantage is that the query refreshes. Drop next month’s invoices in the folder, click Refresh, and the table rebuilds. If our wrapped-row results are a guide, check any invoice with long descriptions by hand.</p>\n<h2 id=\"4-run-the-script-on-a-folder-of-invoices\">4. Run the script on a folder of invoices</h2>\n<p>The script below is the one we ran on our 10 test invoices. It reads every PDF in a folder and writes <code>invoices.xlsx</code> with two sheets: Invoices (number, date, total, the sum of the lines and a check) and Lines (one row per item, with the raw row kept for checking). It uses Camelot for line items and pdftotext plus label rules for the header.</p>\n<p>It needs Python and Poppler’s pdftotext. We ran it on Linux with Python 3.11, where pdftotext came installed (the Ubuntu package is <code>poppler-utils</code>), and set it up with:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"bash\"><code><span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">python3</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> -m</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> venv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> invoices-env</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">invoices-env/bin/pip</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> install</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> camelot-py</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> pandas</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> openpyxl</span></span></code></pre>\n<p>Save this as <code>invoices_to_excel.py</code>:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"Read every PDF invoice in a folder into invoices.xlsx: one sheet of</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">invoices, one of line items. Usage: python invoices_to_excel.py &lt;folder&gt;\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re, subprocess, sys</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pathlib </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Path</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> camelot, pandas </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> money</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.sub(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">$€£</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">USD</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">EUR</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">GBP\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s)).strip()</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    neg </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.startswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"(\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.endswith(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\")\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.strip(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"()\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.fullmatch(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{1,3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">,</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s): s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\".\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\".\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.fullmatch(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{1,3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">,</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{3}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, s): s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\",\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">float</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> neg </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> float</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> date</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s):  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># 03/10/2026 is read as March 10 (US order)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> fmt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%m/</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.%m.%Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%Y-%m-</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%B </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, %Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%b</span><span style=\"color:#005CC5;--shiki-dark:#F47067\"> %d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, %Y\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> datetime.strptime(s.strip(), fmt).date()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ValueError</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">pass</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> header</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(text):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    no </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"invoice</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">no</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">number</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">#</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(\\S</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d\\S</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, text, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    dt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.search(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">invoice date</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">date</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">issued</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">([</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">A-Za-z</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{3,9}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> \\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{1,2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{4}|</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[\\d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">./-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{8,10}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, text, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    tots </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.findall(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">total</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> due</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">amount due</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(?:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">USD</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">EUR</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\$</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">([\\d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">.,()</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, text, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"number\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: no.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> no </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: date(dt.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">            \"total\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: money(tots[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tots </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> lines</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    tables </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> camelot.read_pdf(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">pages</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"all\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">flavor</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"lattice\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tables.n </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: tables </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> camelot.read_pdf(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">pages</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"all\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">flavor</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"stream\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> []</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tables:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> t.df.values.tolist():</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [re.sub(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, c).strip() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c.strip()]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(cells) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 2</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> money(cells[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.match(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">sub</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">total</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">amount due</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">.</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">tax\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, cells[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], re.I): </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (amt </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> money(cells[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                out.append({</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"description\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: cells[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: amt, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"row\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" | \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(cells)})</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">folder </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Path(sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]); pdfs </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(folder.glob(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"*.pdf\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdfs: sys.exit(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"No PDF files in </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">folder</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">invoices, items </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [], []</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdf </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdfs:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rec </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name}</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> subprocess.run([</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"pdftotext\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-layout\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">str</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"-\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">capture_output</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#E36209;--shiki-dark:#F69D50\">                              text</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">check</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).stdout</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.strip():</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"no text layer: run ocrmypdf on it first\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; invoices.append(rec); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rec </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header(text)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> lines(pdf); items </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [{</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: pdf.name} </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"lines_sum\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">sum</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"amount\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        missing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [k </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> k </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"number\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"total\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rec[k] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"missing \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \", \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(missing) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> missing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">                        \"ok\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"lines_sum\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"total\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"lines do not add up to total\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    except</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Exception</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> e:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        rec[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"error: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">e</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    invoices.append(rec)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> folder </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"invoices.xlsx\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">with</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd.ExcelWriter(out) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> xl:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    pd.DataFrame(invoices).to_excel(xl, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">sheet_name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Invoices\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    pd.DataFrame(items).to_excel(xl, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">sheet_name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Lines\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pd.DataFrame(invoices)[[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"file\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"number\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"total\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"lines_sum\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"check\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]].to_string(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">index</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(items)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> line items written to </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">out</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span></code></pre>\n<p>Run it with <code>invoices-env/bin/python invoices_to_excel.py invoices/</code>. On our folder, with the scan already passed through OCRmyPDF, it printed:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span>     file      number       date   total  lines_sum                check</span></span>\n<span class=\"line\"><span>inv01.pdf    NWS-1042 2026-10-03  320.00     320.00                   ok</span></span>\n<span class=\"line\"><span>inv02.pdf   2026-0877 2026-10-07  308.91     308.91                   ok</span></span>\n<span class=\"line\"><span>inv03.pdf         NaN       None  740.00     740.00 missing number, date</span></span>\n<span class=\"line\"><span>inv04.pdf       50213 2026-10-14 2473.40    2473.40                   ok</span></span>\n<span class=\"line\"><span>inv05.pdf   KPW-77810 2026-10-12 1292.30    1292.30                   ok</span></span>\n<span class=\"line\"><span>inv06.pdf     FP-5531 2026-10-09  274.00     274.00                   ok</span></span>\n<span class=\"line\"><span>inv07.pdf         NaN 2026-10-15  372.60     345.00       missing number</span></span>\n<span class=\"line\"><span>inv08.pdf DGD/26/4410 2026-10-16  591.50     591.50                   ok</span></span>\n<span class=\"line\"><span>inv09.pdf    NWS-1042 2026-10-03  320.00     320.00                   ok</span></span>\n<span class=\"line\"><span>inv10.pdf    OSL-9001 2026-10-01 1080.00    1080.00                   ok</span></span>\n<span class=\"line\"><span></span></span>\n<span class=\"line\"><span>58 line items written to invoices/invoices.xlsx</span></span></code></pre>\n<p>Eight invoices came out fully right, and the two it could not read say so instead of filling in a wrong value. Our first version did not do that. It took the word “Invoice” from the boxed header as the invoice number, so we made the rule require a digit. Run on the scan before OCR, the check column reads <code>no text layer: run ocrmypdf on it first</code>.</p>\n<p>The script rewrites <code>invoices.xlsx</code> on every run, so run it on a copy of the folder, and close the workbook in Excel first. A PDF that crashes a tool gets an <code>error:</code> row instead of stopping the run. An empty folder stops with a message.</p>\n<h2 id=\"5-teach-the-script-each-suppliers-labels\">5. Teach the script each supplier’s labels</h2>\n<p>The <code>missing</code> rows tell you which rule to extend. For invoice 7, adding <code>reference</code> to the number pattern, as <code>(?:invoice\\s*(?:no\\.?|number|#)|reference)</code>, picked up “CC-2026-118” when we tried it. Its total check would still fail, because that invoice adds 8% sales tax to the lines. Compare the lines with the subtotal on invoices that list tax, or add the tax line to the sum.</p>\n<p>Three settings deserve a look before you trust the output:</p>\n<ul>\n<li><strong>Date order.</strong> The script reads 03/10/2026 as March 10. If your suppliers write the day first, move <code>%d/%m/%Y</code> ahead of <code>%m/%d/%Y</code> in <code>date()</code>, or split suppliers into folders by country.</li>\n<li><strong>Decimal commas.</strong> <code>money()</code> reads both 1,234.56 and 1.234,56, but only with two decimals. An amount printed without cents, such as 1,200, is not read as money.</li>\n<li><strong>Borderless tables.</strong> If a supplier’s table has no lines and Camelot’s stream mode merges columns, try tabula-py with <code>pandas_options={\"dtype\": str}</code>, which read all of ours.</li>\n</ul>\n<h2 id=\"6-price-the-paid-apis-for-your-volume\">6. Price the paid APIs for your volume</h2>\n<p>If the check column fills with <code>missing</code>, the paid models are worth pricing. All three return the invoice number, the total and the line items as named fields (<a href=\"https://docs.aws.amazon.com/textract/latest/dg/invoices-receipts.html\">Amazon</a>, <a href=\"https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/prebuilt/invoice\">Microsoft</a>, <a href=\"https://docs.cloud.google.com/document-ai/docs/processors-list\">Google</a>), with no label rules to write. These are list prices as read on October 20, 2026; the monthly costs assume one-page invoices.</p>\n<table>\n<thead>\n<tr>\n<th>Service</th>\n<th>Price</th>\n<th>100 invoices a month</th>\n<th>500</th>\n<th>2,000</th>\n<th>Free allowance</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Amazon Textract AnalyzeExpense</td>\n<td>$0.01 a page (US West, Oregon)</td>\n<td>$1</td>\n<td>$5</td>\n<td>$20</td>\n<td>100 pages a month for 3 months, new customers</td>\n</tr>\n<tr>\n<td>Azure Document Intelligence, prebuilt invoice (S0)</td>\n<td>$10 per 1,000 pages (East US)</td>\n<td>$1</td>\n<td>$5</td>\n<td>$20</td>\n<td>Free tier: 500 pages a month, only the first 2 pages of each file</td>\n</tr>\n<tr>\n<td>Google Document AI Invoice parser</td>\n<td>$0.10 per document of up to 10 pages</td>\n<td>$10</td>\n<td>$50</td>\n<td>$200</td>\n<td>None listed for the parser; Google Cloud Free Program credit of $300</td>\n</tr>\n</tbody>\n</table>\n<p>The prices come from <a href=\"https://aws.amazon.com/textract/pricing/\">Amazon’s Textract pricing page</a>, whose own example prices AnalyzeExpense at $0.01 a page in US West (Oregon) and $0.008 after one million pages; from Microsoft’s <a href=\"https://prices.azure.com/api/retail/prices\">Azure Retail Prices API</a> for East US, which lists prebuilt pages at $10 per 1,000; and from <a href=\"https://cloud.google.com/document-ai/pricing\">Google’s Document AI pricing page</a>, where one “count” equals up to 10 pages. Azure’s <a href=\"https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/service-limits\">free tier</a> also caps files at 4 MB and one request per second, and its 500-page allowance is on the <a href=\"https://azure.microsoft.com/en-us/pricing/details/ai-document-intelligence/\">pricing page</a>.</p>\n<p>The per-document price makes Google the costliest for one-page invoices and the closest for long ones. A 10-page invoice costs $0.10 at Google and $0.10 at the other two.</p>\n<p>What the table leaves out matters as much. Each service needs a cloud account with billing set up, and each means sending your suppliers’ invoices to that cloud. We did not run any of the three, because that needs a paid account. Cloudflare’s preview invoice model is cheaper still on its free tier, but it <a href=\"https://28creation.com/articles/cloudflare-clairic-invoice-extraction/\">does not accept PDFs</a>, only images.</p>\n<h2 id=\"when-the-free-route-stops-paying\">When the free route stops paying</h2>\n<p>Each new supplier layout costs a rule, and each rule needs a test. Our <a href=\"https://28creation.com/articles/automation-payback-formula/\">payback formula</a> counts that upkeep. At 500 one-page invoices a month, Amazon or Azure would cost $5 a month in API fees. Put the hours you spend writing and fixing label rules against that figure.</p>\n<p>Our view: a script pays only while the layouts stay few and fixed.</p>\n<p>If your invoices arrive by email, our guide to <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments automatically</a> gets the PDFs into a folder for the script. For the fields in the email body itself, see <a href=\"https://28creation.com/articles/email-data-into-excel/\">email data into Excel</a>. More guides like these are on the <a href=\"https://28creation.com/topics/automation/\">automation topic page</a>.</p>\n<h2 id=\"check-the-total-on-every-invoice\">Check the total on every invoice</h2>\n<p>Whichever route you pick, keep the check column. Compare the sum of the lines with the printed total, or with the subtotal when tax is listed, and send every mismatch to a person. In our test, every invoice the script could not read flagged itself that way, and every invoice marked <code>ok</code> was right.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>On October 20, 2026, we generated 10 sample invoices with the ReportLab library, including one rendered as an image to stand in for a scan, and scored Camelot, tabula-py, pdfplumber and pdftotext against their known values; the vendors and amounts are made up. We ran the script in this guide on the same files, on Linux with Python 3.11, and read the vendors’ pricing pages, the Azure price API and each tool’s documentation that day. We did not test Excel’s PDF connector or any paid API, and our invoices are cleaner than real scans, so treat the scores as a best case.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Camelot, <a href=\"https://github.com/camelot-dev/camelot\">README</a> (flavors and text-based PDFs), version 2.0.0</li>\n<li>pdfplumber, <a href=\"https://github.com/jsvine/pdfplumber\">README</a> (table settings and scanned PDFs), version 0.11.10</li>\n<li>Tabula, <a href=\"https://github.com/tabulapdf/tabula-java\">tabula-java README</a> (lattice and stream modes); tabula-py, <a href=\"https://github.com/chezou/tabula-py\">README</a> and <a href=\"https://tabula-py.readthedocs.io/en/latest/getting_started.html\">Getting started</a> (Java 8 or later), version 2.10.0</li>\n<li>Poppler, <a href=\"https://poppler.freedesktop.org/\">project page</a> and <a href=\"https://gitlab.freedesktop.org/poppler/poppler/-/blob/master/utils/pdftotext.1\">pdftotext manual</a>, version 24.02.0</li>\n<li>OCRmyPDF, <a href=\"https://ocrmypdf.readthedocs.io/en/latest/introduction.html\">Introduction</a>, version 17.13.0</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/pdf\">Power Query PDF connector</a>, and Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16\">Power Query data sources in Excel versions</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/prebuilt/invoice\">Invoice data extraction</a> and <a href=\"https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/service-limits\">Service quotas and limits</a>, Document Intelligence</li>\n<li>Microsoft, <a href=\"https://azure.microsoft.com/en-us/pricing/details/ai-document-intelligence/\">Azure Document Intelligence pricing</a> and <a href=\"https://prices.azure.com/api/retail/prices\">Azure Retail Prices API</a> (product “Azure Document Intelligence”, region eastus), read October 20, 2026</li>\n<li>Amazon Web Services, <a href=\"https://aws.amazon.com/textract/pricing/\">Amazon Textract pricing</a> and <a href=\"https://docs.aws.amazon.com/textract/latest/dg/invoices-receipts.html\">Analyzing invoices and receipts</a>, read October 20, 2026</li>\n<li>Google Cloud, <a href=\"https://cloud.google.com/document-ai/pricing\">Document AI pricing</a> and <a href=\"https://docs.cloud.google.com/document-ai/docs/processors-list\">Processor list</a> (Invoice parser fields), read October 20, 2026</li>\n<li>28creation test, October 20, 2026: 10 generated invoices, 58 line items, scored by our own script</li>\n</ul>\n",
      "summary": "On 10 sample invoices we made, two free table readers got 55 of 58 line items right. Paid APIs cost 1 to 10 cents a one-page invoice.",
      "image": "https://28creation.com/og/pdf-invoices-to-excel.jpg",
      "date_published": "2026-10-23T15:00:00.000Z",
      "date_modified": "2026-10-23T15:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/openai-decisions-api-generally-available/",
      "url": "https://28creation.com/articles/openai-decisions-api-generally-available/",
      "title": "OpenAI Decisions API leaves beta with no charge for output",
      "content_html": "<h2>The short read</h2><p><strong>OpenAI&apos;s Decisions API charges nothing for output tokens and claims answers about 10x faster than the Responses API.</strong></p><ul><li>The API left beta on October 22. It runs only gpt-6-luna, takes text, images and JSON, and returns probabilities and choices rather than free text.</li></ul><h3>What you can do with this</h3><blockquote><p>If you run a yes-or-no or pick-one classifier on gpt-6-luna today, price one day of its traffic on both endpoints. Decisions saves the output fee, but it cannot read files or audio.</p></blockquote><hr><p>OpenAI’s Decisions API is <a href=\"https://developers.openai.com/api/docs/changelog\">now generally available</a>, its API changelog says in an October 22 entry. The <a href=\"https://developers.openai.com/api/docs/guides/decisions\">guide</a> prices uncached input at $0.10 per million tokens, and output is not charged at all. If your product routes tickets or scores leads with a language model, those calls can now run cheaper and faster than on the Responses API, with limits worth checking first.</p>\n<p>The API had been in public beta since October 6. <a href=\"https://techcrunch.com/2026/10/22/openai-makes-its-jev-competitor-widely-available-as-decision-model-competition-heats-up/\">TechCrunch reported the launch</a> the same day.</p>\n<h2 id=\"a-classifier-with-typed-answers\">A classifier with typed answers</h2>\n<p>The endpoint is <code>POST /v1/decisions</code>, and <code>gpt-6-luna</code> is “the only currently supported model,” the guide says. Instead of a prompt that asks for text, a request carries up to 400 questions, up from 200 in the beta. Each question has one of three types: a predicate returns a probability from 0 to 1, a choice returns one of your options with a probability per option, and a score rates the item on a scale of 2 to 10 levels.</p>\n<p>The guide claims answers “about 10x faster than the Responses API.” OpenAI did not train a new model for this. TechCrunch reports the team reconfigured Luna for faster inference, and quotes Nikunj Handa, product lead for OpenAI’s APIs: “it’s just plain old Luna that gets the job done.”</p>\n<p>Handa told TechCrunch latency fell from one to three seconds to “100, 120, 180 milliseconds.” Box, Datadog and Ramp were among its early trial users, the report says.</p>\n<h2 id=\"the-price-cut-is-the-output-fee\">The price cut is the output fee</h2>\n<p>The input price is no discount. OpenAI’s <a href=\"https://developers.openai.com/api/docs/pricing\">pricing page</a> lists standard gpt-6-luna rates at the same $0.10 per million input tokens, plus $0.50 per million output tokens and $0.01 for cached input. The guide says Decisions cache reads match those caching rates, and cache writes are charged.</p>\n<p>A hypothetical triage job shows the size of it: 10,000 emails of 800 tokens each is 8 million input tokens, or $0.80 on either endpoint. If Responses wrote 40 output tokens per email, that adds 400,000 tokens, or $0.20. Decisions would cost $0.80 against $1.00. Short answers keep the saving modest; speed is the stronger reason to switch.</p>\n<h2 id=\"what-it-will-not-do\">What it will not do</h2>\n<p>The trade-offs are in the guide. “Files, audio, and item references aren’t supported,” so PDFs need converting first. When you need an object in your own JSON schema, the guide points you back to Structured Outputs on the Responses API.</p>\n<p>Voice works only through GPT-Live, which, per OpenAI’s <a href=\"https://developers.openai.com/api/docs/guides/decisions-voice\">voice guide</a>, “sends the conversation transcript to Decisions automatically.” Regional processing premiums and long-context multipliers also apply. And on two outside rankings cited by TechCrunch, OpenAI’s decision model placed 8th and 14th, behind rival Jev from TypeSafe at 7th and 3rd.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>List your classifier calls.</strong> A prompt that ends in yes or no maps onto a predicate question; one that picks a label maps onto a choice.</li>\n<li><strong>Price a real day.</strong> Take one day of input and output tokens from your usage export and apply $0.10 input with no output fee, against $0.10 and $0.50 on Responses.</li>\n<li><strong>Batch questions per item.</strong> One request can ask up to 400 questions about the same email, so ask them together.</li>\n<li><strong>Keep a shared prefix of 1,024 tokens or more.</strong> Implicit caching needs it, and question text does not count toward it.</li>\n<li><strong>Track it separately.</strong> Usage is reported at <code>GET /v1/organization/usage/decisions</code> with an admin key that has <code>api.usage.read</code>.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>OpenAI API changelog, entries of October 6 and October 22, 2026, read October 23, 2026: <a href=\"https://developers.openai.com/api/docs/changelog\">https://developers.openai.com/api/docs/changelog</a></li>\n<li>OpenAI, Decisions API guide, read October 23, 2026: <a href=\"https://developers.openai.com/api/docs/guides/decisions\">https://developers.openai.com/api/docs/guides/decisions</a></li>\n<li>OpenAI, API pricing, gpt-6-luna rates, read October 23, 2026: <a href=\"https://developers.openai.com/api/docs/pricing\">https://developers.openai.com/api/docs/pricing</a></li>\n<li>OpenAI, Connect voice to Decisions, read October 23, 2026: <a href=\"https://developers.openai.com/api/docs/guides/decisions-voice\">https://developers.openai.com/api/docs/guides/decisions-voice</a></li>\n<li>TechCrunch, Tim Fernholz, “OpenAI makes its Jev competitor widely available as decision model competition heats up,” October 22, 2026: <a href=\"https://techcrunch.com/2026/10/22/openai-makes-its-jev-competitor-widely-available-as-decision-model-competition-heats-up/\">https://techcrunch.com/2026/10/22/openai-makes-its-jev-competitor-widely-available-as-decision-model-competition-heats-up/</a></li>\n</ul>\n",
      "summary": "Classifiers and routers can get typed answers from gpt-6-luna for $0.10 per million input tokens. The saving is the output fee; the bigger change is speed.",
      "image": "https://28creation.com/og/openai-decisions-api-generally-available.jpg",
      "date_published": "2026-10-23T14:47:50.000Z",
      "date_modified": "2026-10-23T14:47:50.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "AI",
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/n8n-self-hosted-free-license/",
      "url": "https://28creation.com/articles/n8n-self-hosted-free-license/",
      "title": "Self-hosted n8n is free while only staff build the workflows",
      "content_html": "<h2>The short read</h2><p><strong>Self-hosted n8n is free for a business whose own people build the workflows, consultants charging clients included.</strong></p><ul><li>The license bars hosting n8n for clients who build their own workflows, embedding a workflow builder in a product, and white-labeling it.</li><li>Sharing, SSO, Git and environments need a paid key. Business showed at 667 euros a month; a 2 GiB server costs $12.</li></ul><h3>What you can do with this</h3><blockquote><p>Write down who will open the n8n editor before you install. If all of them work for you, self-host for free; if clients must build workflows, email license@n8n.io first.</p></blockquote><hr><p>Self-hosted n8n costs nothing to license for a small business that builds its own workflows. The condition fits in one line of the company’s FAQ: “<a href=\"https://docs.n8n.io/n8n-community-license/community-license/license-faq\">If workflows are only created or modified by you or people in your organization</a>, you can use n8n under the Community license for free.” Clients can get the results. They can’t get the editor.</p>\n<p>That leaves two costs. A short list of features sits behind a paid license key, and the server and its upkeep are yours to pay for. We read the <a href=\"https://github.com/n8n-io/n8n/blob/master/LICENSE.md\">license text on GitHub</a> clause by clause, along with n8n’s license FAQ, its feature list and the price pages of two hosting providers, all on October 20, 2026. We also counted the enterprise-licensed files in n8n’s code.</p>\n<p>This is our reading, not legal advice. Where the text leaves room, we say so.</p>\n<h2 id=\"the-license-is-not-open-source-and-one-sentence-decides-most-cases\">The license is not open source, and one sentence decides most cases</h2>\n<p>The code is under the Sustainable Use License, version 1.0, which n8n wrote in 2022. The company says plainly that “<a href=\"https://docs.n8n.io/n8n-community-license/community-license\">we do not call ourselves open source</a>”, because the Open Source Initiative does not accept limits on use. Until March 17, 2022, n8n used Apache 2.0 with the Commons Clause.</p>\n<p>The grant itself is broad: a “royalty-free, worldwide” license to “use, copy, distribute, make available, and prepare derivative works.” Then come the limits. The one that matters for a business reads:</p>\n<blockquote>\n<p>You may use or modify the software only for your own internal business purposes or for non-commercial or personal use.</p>\n</blockquote>\n<p>The second limit covers handing n8n to others: “You may distribute the software or provide it to others only if you do so free of charge for non-commercial purposes.” The third says you may not remove n8n’s licensing or copyright notices. A fourth rule, under Notices, says a modified copy must carry “a prominent notice stating that you have modified the software.”</p>\n<p>Here is the gap. The license never defines “internal business purposes.” The FAQ does, as “the use of n8n within your organization or product where people outside of your business can receive or see what your workflows produce.” So the practical answers live in the FAQ, which is the company’s guidance rather than the license text itself.</p>\n<h2 id=\"consulting-is-allowed-handing-clients-the-editor-is-not\">Consulting is allowed, handing clients the editor is not</h2>\n<p>The FAQ lists what is allowed and what is not. We sorted the cases a small business asks about.</p>\n<table>\n<thead>\n<tr>\n<th>What you want to do</th>\n<th>Allowed without a paid license?</th>\n<th>n8n’s wording</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Run your own company’s automations</td>\n<td>Yes</td>\n<td>“Use n8n community version to help run your business”</td>\n</tr>\n<tr>\n<td>Charge clients to build and maintain workflows</td>\n<td>Yes</td>\n<td>“Charge your customers for workflow creation, setup, and maintenance services”</td>\n</tr>\n<tr>\n<td>Run client workflows on your own instance</td>\n<td>Yes, if clients can’t edit them</td>\n<td>“as long as your clients do not have the possibility to create or edit them”</td>\n</tr>\n<tr>\n<td>Install and manage n8n on a client’s server</td>\n<td>Yes, if you don’t also host it</td>\n<td>“as long as you don’t offer hosting as well”</td>\n</tr>\n<tr>\n<td>Use n8n behind your own product</td>\n<td>Yes, if users only trigger what you built</td>\n<td>“users can use and trigger the automations you built”</td>\n</tr>\n<tr>\n<td>Host n8n so clients build their own workflows</td>\n<td>No</td>\n<td>“Host n8n as a service and allow your clients to build workflows”</td>\n</tr>\n<tr>\n<td>Embed a workflow builder in your product</td>\n<td>No, even with your own interface</td>\n<td>“whether via a custom UI, our API, MCP, or an AI agent”</td>\n</tr>\n<tr>\n<td>White-label n8n</td>\n<td>No</td>\n<td>“White-label n8n by removing our branding and offering n8n as is”</td>\n</tr>\n<tr>\n<td>Fork n8n into your own automation product</td>\n<td>No</td>\n<td>“Fork n8n’s code to launch your own automation product”</td>\n</tr>\n</tbody>\n</table>\n<p>The test is who controls the workflows, not how much you charge. “The key question is not whose credentials are used, but who controls the workflow logic,” the FAQ says. Clients may even connect their own accounts to workflows you built. For agencies, “there is no limit on the number of clients you can serve from a single instance.”</p>\n<h2 id=\"two-setups-need-a-word-with-n8n-first\">Two setups need a word with n8n first</h2>\n<p>Two cases sit in a gray zone, by the company’s own account. Shipping n8n inside a paid product that runs on the customer’s premises “may sit in gray territory,” because a customer with local access may be able to build workflows. The FAQ asks you to confirm that setup with <a href=\"mailto:license@n8n.io\">license@n8n.io</a> before launch. Embedding the n8n editor in your product is a separate deal: <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration\">OEM deployment</a> “requires a separate commercial agreement,” and n8n branding stays.</p>\n<p>The penalty for getting it wrong is the license itself. Use outside the terms “is not licensed, and your license will automatically terminate.” If n8n sends a notice and you stop within 30 days, the license comes back. A second violation after that ends it “automatically and permanently.”</p>\n<h2 id=\"about-4-of-n8ns-files-are-under-a-second-stricter-license\">About 4% of n8n’s files are under a second, stricter license</h2>\n<p>The first lines of LICENSE.md carve out an exception. Files with “.ee.” in their name or “.ee” in a folder name “are NOT licensed under the Sustainable Use License.” They fall under the <a href=\"https://github.com/n8n-io/n8n/blob/master/LICENSE_EE.md\">n8n Enterprise License</a>, which says the code “may only be used in production” with a valid enterprise license. You may “copy and modify the Software for development and testing purposes, without requiring a subscription.”</p>\n<p>We counted them. On n8n’s master branch at commit a5eada9 (October 20, 2026), 1,320 of 31,957 files match that rule, about 4%. Of those, 464 are tests and 856 are not. The folder names read like the paid feature list: sso.ee, ldap.ee, log-streaming.ee, source-control.ee, environments.ee and external-secrets.ee among 19 folder names of that kind.</p>\n<p>That code is in the copy you install. The npm package of n8n 2.44.4, the stable release on October 20, holds 789 such files among its 6,524, in folders such as sso.ee and ldap.ee. So the license key, not the download, decides which of those features switch on. With no key, n8n says, a self-hosted instance “runs as the free <a href=\"https://docs.n8n.io/deploy/host-n8n\">Community edition</a>.”</p>\n<h2 id=\"ten-features-need-a-paid-license-key\">Ten features need a paid license key</h2>\n<p>n8n’s <a href=\"https://docs.n8n.io/deploy/host-n8n/community-edition-features\">edition comparison</a> says the free Community edition “includes almost the complete feature set.” It lists what it leaves out:</p>\n<ul>\n<li>Custom variables</li>\n<li>Environments</li>\n<li>External secrets</li>\n<li>External storage for binary data</li>\n<li>Log streaming (ordinary logging is included)</li>\n<li>Multi-main mode (queue mode is included)</li>\n<li>Projects</li>\n<li>Single sign-on through SAML or LDAP</li>\n<li>Sharing workflows and credentials</li>\n<li>Version control using Git</li>\n</ul>\n<p>We expect the sharing gap to be the first one a small team hits. Without it, “only the instance owner and the user who creates them can access workflows and credentials.” If you register with an email address, a free key adds folders, debug in editor and custom execution data.</p>\n<p>The paid step is large. n8n’s <a href=\"https://n8n.io/pricing/\">pricing page</a> showed us the self-hosted Business plan at 667 euros a month, billed yearly, for 40,000 executions. Its feature list includes SSO, Git version control and 6 shared projects. A Start-up Plan gives companies under 20 employees 50% off, if they qualify. A paid key “must ping our license server daily to stay active,” and that ping reports production execution counts.</p>\n<p>One inconsistency: the Business card lists “Queue mode for scaling” as a feature, while the documentation says queue mode is included in Community. The docs also call the pricing page the source of truth. Check both before you plan around queue mode.</p>\n<h2 id=\"a-2-gib-server-costs-12-to-18-a-month\">A 2 GiB server costs $12 to €18 a month</h2>\n<p>In our cost test on October 20, 2026, n8n peaked at <a href=\"https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/\">884 MiB of memory</a> over 1,000 webhook runs, so we recommended a 2 GiB server. The company’s own <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration/prerequisites\">sizing example</a> gives 320 MB to 2 GB of memory.</p>\n<p>Prices for that size, read October 20, 2026:</p>\n<table>\n<thead>\n<tr>\n<th>Provider</th>\n<th>Plan</th>\n<th>CPU and memory</th>\n<th>Monthly price</th>\n<th>Backups</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>DigitalOcean</td>\n<td>Basic droplet</td>\n<td>1 vCPU, 2 GiB</td>\n<td><a href=\"https://www.digitalocean.com/pricing/droplets\">$12.00</a></td>\n<td>20% of the droplet price weekly, 30% daily</td>\n</tr>\n<tr>\n<td>Hetzner, EU</td>\n<td>CPX12</td>\n<td>1 vCPU, 2 GB</td>\n<td><a href=\"https://www.hetzner.com/cloud/regular-performance/\">€11.99</a> excl. VAT</td>\n<td>From €1.20 a month, priced by server</td>\n</tr>\n<tr>\n<td>Hetzner, US</td>\n<td>CPX11</td>\n<td>2 vCPU, 2 GB</td>\n<td><a href=\"https://www.hetzner.com/cloud/regular-performance/\">€17.99</a> excl. VAT</td>\n<td>From €1.20 a month, priced by server</td>\n</tr>\n</tbody>\n</table>\n<p>n8n’s <a href=\"https://docs.n8n.io/deploy/host-n8n/install-options/use-a-cloud-provider/deploy-to-hetzner\">Hetzner guide</a> says “the CPX11 type is enough” for most usage; Hetzner listed it only in its two US locations. Hetzner’s cheaper cost-optimized plans, from €5.99 for 4 GB, were labeled “not available” when we read the <a href=\"https://www.hetzner.com/cloud/cost-optimized/\">page</a>.</p>\n<p>With weekly backups, the DigitalOcean server comes to $14.40 a month before tax, or $172.80 a year. The Business plan costs 8,004 euros a year at the price we were shown. So the server is the small part of the bill. The license fee is the big one, and a team pays it only once it needs a paid feature such as SSO, Git or shared projects.</p>\n<h2 id=\"the-upkeep-is-where-the-real-cost-sits\">The upkeep is where the real cost sits</h2>\n<p>n8n is blunt about who should do this: “<a href=\"https://docs.n8n.io/deploy/host-n8n/install-options/use-a-cloud-provider/deploy-to-hetzner\">n8n recommends self-hosting for expert users</a>. Mistakes can lead to data loss, security issues, and downtime.” Four jobs come with the server.</p>\n<p><strong>Updates.</strong> n8n “releases a new minor version most weeks,” and its update page says “<a href=\"https://docs.n8n.io/deploy/host-n8n/keep-n8n-running/update-n8n\">try to update at least once a month</a>” and read the release notes for breaking changes. It suggests testing the update in Environments first. Environments is a paid feature. On the free edition, a second instance can serve as the test copy; the FAQ allows running several.</p>\n<p><strong>Backups.</strong> A full backup is the <code>.n8n</code> folder plus your database. The folder holds the encryption key for every stored credential, and without it a restored database “<a href=\"https://docs.n8n.io/deploy/host-n8n/keep-n8n-running/backup-and-restore\">can’t be decrypted</a>.” With the default SQLite database, you stop n8n before copying the folder. n8n recommends a full backup before every update, and the built-in CLI export holds only workflows and credentials, not users or execution history.</p>\n<p><strong>The database.</strong> n8n uses SQLite out of the box. Its Docker Compose guide says SQLite “is fine for trying things out, but for a production instance that must handle more than a handful of users or workflows running around the clock use <a href=\"https://docs.n8n.io/deploy/host-n8n/install-options/install-using-docker-compose\">Postgres</a> instead.” That is a second service to run and back up. n8n supports <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/choose-n8ns-database\">Postgres 16 to 18</a>, and that range shifts each year.</p>\n<p><strong>Scaling.</strong> Queue mode hands executions to separate worker processes through a <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/enable-queue-mode\">Redis</a> queue, and n8n doesn’t recommend it with SQLite. So scaling up means Redis and workers on top of Postgres. A single instance avoids all of that.</p>\n<p>We have not timed this upkeep, so we won’t put an hour figure on it. Our <a href=\"https://28creation.com/articles/automation-payback-formula/\">payback formula</a> shows how to count it once you know your own.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision is about who touches the editor, then about which paid features you can live without.</p>\n<ul>\n<li><strong>You run your own automations, or build them for clients on your instance:</strong> self-host the Community edition. The license allows it, including paid consulting. Budget about $12 a month for a 2 GiB server plus backups, and an update every month.</li>\n<li><strong>Your team needs shared workflows or single sign-on:</strong> the free edition won’t do it. Compare Business at 667 euros a month (half that if the Start-up Plan applies) with n8n Cloud, whose plans we priced in our <a href=\"https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/\">three-tool comparison</a>.</li>\n<li><strong>Clients should build or edit workflows themselves,</strong> in n8n or in your product: that needs a commercial agreement. Write to <a href=\"mailto:license@n8n.io\">license@n8n.io</a> before you build, not after.</li>\n<li><strong>Nobody on the team will own updates and backups:</strong> pay for n8n Cloud, or pick another tool. n8n itself recommends Cloud to anyone not experienced at running servers.</li>\n</ul>\n<p>Before you install, write down who will open the editor. If every name on the list works for you, the license costs nothing.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read both license texts, n8n’s FAQ, docs and pricing page, and two hosts’ price pages on October 20, 2026. We counted the enterprise-licensed files from n8n’s GitHub file list and its npm package. We did not install n8n for this piece; the memory figure comes from our earlier test.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>n8n, <a href=\"https://github.com/n8n-io/n8n/blob/master/LICENSE.md\">LICENSE.md</a> (Sustainable Use License 1.0) and <a href=\"https://github.com/n8n-io/n8n/blob/master/LICENSE_EE.md\">LICENSE_EE.md</a> (n8n Enterprise License), master branch, read October 20, 2026</li>\n<li>n8n Docs, <a href=\"https://docs.n8n.io/n8n-community-license/community-license\">Community license</a> and <a href=\"https://docs.n8n.io/n8n-community-license/community-license/license-faq\">License FAQ</a></li>\n<li>n8n Docs, <a href=\"https://docs.n8n.io/deploy/host-n8n/community-edition-features\">Compare editions</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n\">Host n8n</a> and <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration\">Deploy as an OEM integration</a></li>\n<li>n8n, <a href=\"https://n8n.io/pricing/\">Pricing</a> (Business plan and license key FAQ; shown in euros, read October 20, 2026)</li>\n<li>n8n Docs, <a href=\"https://docs.n8n.io/deploy/host-n8n/keep-n8n-running/update-n8n\">Update n8n</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/keep-n8n-running/backup-and-restore\">Back up and restore</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/choose-n8ns-database\">Choose n8n’s database</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/install-options/install-using-docker-compose\">Install using Docker Compose</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/enable-queue-mode\">Enable queue mode</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/install-options/use-a-cloud-provider/deploy-to-hetzner\">Deploy to Hetzner</a> and <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration/prerequisites\">Prerequisites</a></li>\n<li>DigitalOcean, <a href=\"https://www.digitalocean.com/pricing/droplets\">Droplet pricing</a> (read October 20, 2026)</li>\n<li>Hetzner, <a href=\"https://www.hetzner.com/cloud/regular-performance/\">Regular Performance</a> and <a href=\"https://www.hetzner.com/cloud/cost-optimized/\">Cost-Optimized</a> cloud servers (read October 20, 2026)</li>\n<li>28creation count, October 20, 2026: file list of n8n’s master branch at commit a5eada9 from the GitHub API, and the file list of the n8n 2.44.4 package from npm, both matched against the “.ee” rule in LICENSE.md</li>\n</ul>\n",
      "summary": "The license lets you run n8n and charge clients for workflows, not hand them the editor. SSO, Git and environments need a paid key; a 2 GiB server is about $12.",
      "image": "https://28creation.com/og/n8n-self-hosted-free-license.jpg",
      "date_published": "2026-10-23T07:00:00.000Z",
      "date_modified": "2026-10-23T07:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/arc-browser-shutdown-december-22/",
      "url": "https://28creation.com/articles/arc-browser-shutdown-december-22/",
      "title": "Arc browser stops working December 22; export keeps only links",
      "content_html": "<h2>The short read</h2><p><strong>Arc stops opening on Mac and Windows on December 22, and account data is deleted within 30 days after.</strong></p><ul><li>The Browser Company is ending Arc to put its security team on Dia, its newer browser, which imports Arc&apos;s passwords and extensions.</li></ul><h3>What you can do with this</h3><blockquote><p>If you use Arc for work, import it into Dia or move your passwords to another manager before December 22. Arc&apos;s own export saves only tab links, not logins.</p></blockquote><hr><p>The Browser Company <a href=\"https://arc.net/\">will shut Arc down on December 22, 2026</a>, it said on Arc’s homepage on October 22. That day Arc stops working on macOS and Windows; account data is deleted within 30 days. <a href=\"https://tech.yahoo.com/apps/articles/arc-browser-sunsets-december-22-200151422.html\">TechCrunch</a> and <a href=\"https://www.digitaltrends.com/computing/arc-browser-is-shutting-down-in-december-long-live-ya-trailblazer/\">Digital Trends</a> reported the same dates. Teams that work in Arc have two months to move their saved logins, and by Arc’s own FAQ, its export tool will not carry them.</p>\n<h2 id=\"arc-ends-because-security-work-moved-to-dia\">Arc ends because security work moved to Dia</h2>\n<p>The company’s stated reason is security. “Keeping a browser secure takes constant, dedicated work that is only increasing in the age of AI,” the <a href=\"https://arc.net/\">farewell page</a> says. Its engineering and security teams will focus on Dia, the browser it built after Arc.</p>\n<p>The shift had been visible for a while. TechCrunch reports that active feature work on Arc stopped in 2025, and Digital Trends calls Arc long in “maintenance-only mode.”</p>\n<p>Atlassian <a href=\"https://news.crunchbase.com/ma/atlassian-acquires-browser-ai-knowledge-workers-dia/\">agreed in September 2025</a> to buy The Browser Company for about $610 million in cash. Atlassian’s Mike Cannon-Brookes said then that “today’s browsers weren’t built for work.” Of the two browsers it bought, only Dia remains.</p>\n<p>Do not expect an open-source Arc soon. CEO Josh Miller told TechCrunch the company can’t do that meaningfully without also open-sourcing ADK, the internal toolkit that Dia runs on too.</p>\n<h2 id=\"the-dates-in-order\">The dates, in order</h2>\n<p>The <a href=\"https://arc.net/\">timeline on arc.net</a> runs in three steps:</p>\n<ul>\n<li><strong>October 22:</strong> no new Arc accounts. Existing users see no change yet; the latest Arc version adds a data export option and a smoother Dia import.</li>\n<li><strong>November 23:</strong> on iOS and Android, Browse for Me and the other AI features switch off, and sign-in and tab sync end. Arc Search keeps working as a basic browser.</li>\n<li><strong>December 22:</strong> Arc stops working on Mac and Windows. Arc Search for Android is removed from Google Play and “from the devices it’s installed on.”</li>\n</ul>\n<p>Until December 22, Arc “will continue to work as usual and will receive basic updates,” the FAQ says.</p>\n<h2 id=\"the-export-file-is-a-list-of-links\">The export file is a list of links</h2>\n<p>We read the FAQ closely, and the two exit routes are not equal. Importing into Dia brings over passwords, history, tabs, bookmarks and extensions, with Spaces renamed Profiles and Folders renamed Tab Groups. The timeline says the new release can “export all of your data out of Arc.” But the FAQ’s steps for using tabs in another browser end in “a list of links.”</p>\n<p>The FAQ describes no other way to take saved passwords out of Arc.</p>\n<p>That is the trade-off. Dia is free, and the company says Arc’s features stay free there and “the AI is optional.” But Dia comes from the company that is ending Arc. Leaving for another browser avoids that bet, at the cost of moving logins separately.</p>\n<p>After December 22 the company “will no longer be able to identify or retrieve Arc user data for data requests.” There is no recovery later.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Decide where each Arc user goes</strong> before December 22: Dia, or another browser.</li>\n<li><strong>To move to Dia,</strong> pick Arc as the source when Dia’s setup asks. Existing Dia users on macOS use Dia &gt; Import from Another Browser; on Windows, the Dia app menu’s “Import from another browser.”</li>\n<li><strong>To leave for another browser,</strong> first move saved passwords into a password manager your team controls. Then export tabs: on Mac, Arc &gt; Settings &gt; Advanced &gt; Export Data; on Windows, press Ctrl+, and use Export Data under “Export Spaces and tabs.”</li>\n<li><strong>Check old laptops.</strong> Arc’s FAQ says someone who has not opened Arc in a while can reinstall it and export from inside the app until December 22.</li>\n<li><strong>Ask before the deadline.</strong> Arc’s help chat at <a href=\"https://resources.arc.net\">resources.arc.net</a> stays open until December 22.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>The Browser Company, “A farewell to Arc,” timeline and FAQ, read October 23, 2026: <a href=\"https://arc.net/\">https://arc.net/</a></li>\n<li>TechCrunch via Yahoo Tech, “Arc browser sunsets December 22 as The Browser Company doubles down on Dia,” October 22, 2026: <a href=\"https://tech.yahoo.com/apps/articles/arc-browser-sunsets-december-22-200151422.html\">https://tech.yahoo.com/apps/articles/arc-browser-sunsets-december-22-200151422.html</a></li>\n<li>Digital Trends, “Arc browser is shutting down in December,” October 22, 2026: <a href=\"https://www.digitaltrends.com/computing/arc-browser-is-shutting-down-in-december-long-live-ya-trailblazer/\">https://www.digitaltrends.com/computing/arc-browser-is-shutting-down-in-december-long-live-ya-trailblazer/</a></li>\n<li>Crunchbase News, Atlassian agrees to acquire The Browser Company, September 4, 2025: <a href=\"https://news.crunchbase.com/ma/atlassian-acquires-browser-ai-knowledge-workers-dia/\">https://news.crunchbase.com/ma/atlassian-acquires-browser-ai-knowledge-workers-dia/</a></li>\n</ul>\n",
      "summary": "Arc data deletion starts after December 22. Dia imports passwords and history; Arc's own export saves only a list of tab links.",
      "image": "https://28creation.com/og/arc-browser-shutdown-december-22.jpg",
      "date_published": "2026-10-23T05:40:39.000Z",
      "date_modified": "2026-10-23T05:40:39.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/office-scripts-vs-vba/",
      "url": "https://28creation.com/articles/office-scripts-vs-vba/",
      "title": "Office Scripts vs VBA: what each can still do in 2026",
      "content_html": "<h2>The short read</h2><p><strong>Office Scripts runs in Excel on the web and in Power Automate flows, and VBA runs in neither.</strong></p><ul><li>VBA still reaches Outlook, Word, files and DLLs. Office Scripts sees only the workbook, and flows allow 1,600 Run script calls per user a day.</li><li>Windows blocks macros in internet files by default, and VBScript, which some macros call, will be turned off by default around 2026 or 2027.</li></ul><h3>What you can do with this</h3><blockquote><p>Ask of each macro: does it touch anything outside the workbook? Keep those in VBA. Move the workbook-only jobs that need a browser or a schedule, and search for VBScript.RegExp.</p></blockquote><hr><p>Microsoft’s own comparison page puts the split in one line: VBA macros “are developed for desktop solutions” and Office Scripts “are designed for secure, cross-platform, cloud-based solutions.” For a small office with a folder of .xlsm files, that line sets the limits. A browser tab and a 6 a.m. flow with nobody at the PC are two places VBA can’t go.</p>\n<p>On October 20, 2026, we read Microsoft’s documentation and built the table below from it. Then we wrote one small job both ways. Our view, in short: keep VBA for anything that reaches outside the workbook, and move the workbook-only jobs that must run in a browser or with nobody at the keyboard.</p>\n<h2 id=\"only-office-scripts-runs-in-the-browser-and-in-flows\">Only Office Scripts runs in the browser and in flows</h2>\n<p>Every row comes from a Microsoft page listed under Sources. The <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/resources/vba-differences\">differences page</a> itself was last updated in November 2023, so where a newer page says more, the newer page wins.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>VBA macros</th>\n<th>Office Scripts</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Excel for Windows</td>\n<td>Yes</td>\n<td><a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/testing/platform-limits\">Version 2210 or later</a></td>\n</tr>\n<tr>\n<td>Excel for Mac</td>\n<td>Yes</td>\n<td>Yes</td>\n</tr>\n<tr>\n<td>Excel on the web</td>\n<td>No. Macros <a href=\"https://support.microsoft.com/en-us/office/work-with-vba-macros-in-excel-for-the-web-98784ad0-898c-43aa-a1da-4f0fb5014343\">stay in the file</a> but can’t run or be edited</td>\n<td>Yes</td>\n</tr>\n<tr>\n<td>Excel for iOS</td>\n<td>No</td>\n<td>No</td>\n</tr>\n<tr>\n<td>Word, Outlook, PowerPoint</td>\n<td>Yes</td>\n<td>No, Excel only</td>\n</tr>\n<tr>\n<td>Run by Power Automate</td>\n<td>No connector</td>\n<td>Yes, the <strong>Run script</strong> action</td>\n</tr>\n<tr>\n<td>Language</td>\n<td>VBA</td>\n<td>TypeScript <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/typescript-restrictions\">4.0.3</a>, with no <code>any</code> type and no <code>eval</code></td>\n</tr>\n<tr>\n<td>Where the code lives</td>\n<td>Inside the workbook, saved as <a href=\"https://support.microsoft.com/en-us/office/file-formats-that-are-supported-in-excel-0943ff2c-6014-4e8d-aaea-b83d51d46247\">.xlsm</a> (.xlsx can’t hold it)</td>\n<td><a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/overview/script-storage\">.osts files</a> in the author’s OneDrive, or on SharePoint</td>\n</tr>\n<tr>\n<td>What it can reach</td>\n<td>The whole desktop: COM and OLE, DLLs, the file system</td>\n<td>The workbook only</td>\n</tr>\n<tr>\n<td>Calls to the web</td>\n<td>“No convenient way”</td>\n<td><code>fetch</code> from Excel only; it fails inside Power Automate</td>\n</tr>\n<tr>\n<td>Events such as <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.workbook.open\">open</a> or <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.worksheet.change\">change</a></td>\n<td>Yes</td>\n<td>No</td>\n</tr>\n<tr>\n<td>Starts when</td>\n<td>Someone runs it, or an event fires</td>\n<td>Someone runs it, presses a script button, or a flow calls it</td>\n</tr>\n<tr>\n<td>Recorder</td>\n<td><a href=\"https://learn.microsoft.com/en-us/office/vba/library-reference/concepts/getting-started-with-vba-in-office\">Record Macro</a> on the Developer tab</td>\n<td>Action Recorder on web, Windows, Mac</td>\n</tr>\n<tr>\n<td>License</td>\n<td>Built into desktop Excel</td>\n<td>A business or education plan; Personal and Family in preview</td>\n</tr>\n<tr>\n<td>Admin switch</td>\n<td>Allow all or none, or only signed macros</td>\n<td>On or off for the tenant or one group</td>\n</tr>\n</tbody>\n</table>\n<p>Two rows matter most for a small office. Office Scripts is the only one of the two that runs in Excel on the web. And it is the only one Power Automate can start, so it is the only one that can update a workbook “without even opening Excel.”</p>\n<h2 id=\"vba-reaches-the-whole-pc-and-that-is-the-risk\">VBA reaches the whole PC, and that is the risk</h2>\n<p>VBA’s strength is reach. Microsoft’s <a href=\"https://learn.microsoft.com/en-us/office/vba/library-reference/concepts/getting-started-with-vba-in-office\">VBA guide</a> says it can work with “two or more Office applications at the same time.” Its own samples create an Outlook email and copy an Outlook contact into Word.</p>\n<p><a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/filesystemobject-object\">FileSystemObject</a> “provides access to a computer’s file system.” The <a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/declare-statement\">Declare statement</a> lets it call procedures in a DLL. VBA macros “have the same security clearance as Excel,” says the differences page, which “gives them full access to your desktop.”</p>\n<p>That reach is why Microsoft changed the default. “VBA macros are a common way for malicious actors to gain access to deploy malware and ransomware,” its <a href=\"https://learn.microsoft.com/en-us/microsoft-365-apps/security/internet-macros-blocked\">admin guide</a> says. So Office on Windows now blocks macros in files from the internet.</p>\n<p>The change started rolling out to Current Channel on July 27, 2022. Semi-Annual Enterprise Channel got it on January 10, 2023.</p>\n<p>The test is the Mark of the Web, a tag Windows adds to downloads and email attachments. A macro file that carries it opens with a Security Risk banner, and that banner has no Enable content button. Office on a Mac, on the web, and on iOS and Android devices is not affected.</p>\n<p>The fixes Microsoft lists are concrete. Right-click the file, choose <strong>Properties</strong> and tick <strong>Unblock</strong>.</p>\n<p>Files opened from OneDrive or SharePoint with <strong>Open in Desktop App</strong>, or synced by the OneDrive client, don’t get the mark. A Trusted Location skips the check entirely, which is why Microsoft says to use those “sparingly.” Group policies for this exist only in Microsoft 365 Apps for enterprise, not in Apps for business.</p>\n<h2 id=\"vbscript-is-leaving-windows-and-some-macros-lean-on-it\">VBScript is leaving Windows, and some macros lean on it</h2>\n<p>The second change is slower. In a <a href=\"https://devblogs.microsoft.com/microsoft365dev/how-to-prepare-vba-projects-for-vbscript-deprecation/\">September 2025 post</a>, Microsoft’s Wenjun Gong named two VBA habits that depend on VBScript: running external .vbs files, and referencing VBScript’s library for regular expressions (<code>VBScript.RegExp</code>).</p>\n<p>The plan has three phases. Today VBScript stays on by default. In “approximately 2026 or 2027” it will be turned off by default.</p>\n<p>At an unnamed date it will be removed from Windows, and then “calling .vbs scripts from VBA will no longer be supported.” RegExp references will break “unless the Office client is updated to a supported build.”</p>\n<p>That build is Office Version 2508 (Build 19127.20154) on Windows, which puts RegExp classes into VBA itself. Replying to a reader in the comments, Gong added: “We did not make any change to Scripting.FileSystemObject and Scripting.Dictionary.” So the check is short: search your modules for <code>.vbs</code> and <code>VBScript.RegExp</code>.</p>\n<h2 id=\"office-scripts-sees-only-the-workbook-by-design\">Office Scripts sees only the workbook, by design</h2>\n<p>Office Scripts trades reach for safety. Scripts “only have access to the workbook, not the machine hosting the workbook,” and they get no sign-in token for the user. There is no file system, no other Office app and no DLL.</p>\n<p>Web calls exist, with limits. A script can use <code>fetch</code> when it runs <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/external-calls\">inside Excel</a>. But “there is no way to sign in or use OAuth2,” so keys must be hard-coded or read from somewhere else.</p>\n<p>Scripts stored on SharePoint can’t make the calls at all. In a flow, <code>fetch</code> fails with “fetch is not defined,” and Microsoft points you to Power Automate’s own HTTP actions instead.</p>\n<p>Storage is the quiet trap. Scripts live as .osts files in the author’s OneDrive, under /Documents/Office Scripts/.</p>\n<p>Sharing a script with a workbook links it rather than attaching it, and the script “remain[s] in the script creator’s OneDrive.” When that person leaves, what happens to the script depends on your OneDrive retention policy. Scripts saved to a SharePoint site “are owned by your team,” which makes that the safer home for anything shared, unless the script uses <code>fetch</code>.</p>\n<h2 id=\"power-automate-is-the-scheduler-with-hard-limits\">Power Automate is the scheduler, with hard limits</h2>\n<p>Office Scripts has no workbook events. A script runs when a person starts it or “when a Power Automate flow calls the script.” The built-in scheduler inside Excel is “temporarily disabled,” <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/overview/excel\">Microsoft’s overview</a> says, so a flow is now the way to run a script daily.</p>\n<p>The <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/testing/platform-limits\">platform limits page</a> gives the numbers:</p>\n<ul>\n<li>1,600 Run script calls per user per day, reset at 12:00 a.m. UTC.</li>\n<li>A 120-second timeout for each synchronous call.</li>\n<li>At most 30,000,000 bytes (28.6 MB) of parameters per call.</li>\n<li>5 MB per request or response to the workbook, and 5 million cells per range.</li>\n</ul>\n<p>A hypothetical flow that runs one script on 40 workbooks every hour makes 960 calls a day, inside the cap. For scripts that run long, Microsoft says to speed them up or “split your Excel automation into multiple script calls.”</p>\n<p>Flows also change behavior. Power Automate runs each script “in independent Excel sessions,” so <code>getActiveWorksheet</code> <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/testing/power-automate-troubleshooting\">returns the first sheet</a> unless the script activated another, and <code>getSelectedRange</code> throws. Pivot refreshes do nothing. And “Power Automate always uses UTC,” while a person in Excel gets local time, so “today” can differ by a day.</p>\n<h2 id=\"licenses-vba-comes-with-desktop-excel\">Licenses: VBA comes with desktop Excel</h2>\n<p>VBA “requires no special licensing” in desktop Excel. Office Scripts needs a business or education plan. Microsoft’s list names Office 365 Business and Business Premium, ProPlus, Enterprise E1, E3 and E5, F3, and the education plans A3 and A5.</p>\n<p>The list uses Office 365 plan names, so if your plan goes by another name, look for the <strong>Automate</strong> tab in Excel on the web before you plan a move. Personal and Family plans get Office Scripts only as a preview through the Microsoft 365 Insider program, and can’t share scripts through workbooks.</p>\n<p>Admins have little to do. Office Scripts is <a href=\"https://learn.microsoft.com/en-us/microsoft-365/admin/manage/manage-office-scripts-settings\">“turned on by default”</a> for everyone, and a change to its settings “can take up to 48 hours” to apply.</p>\n<h2 id=\"the-same-job-in-both-languages-differs-in-three-details\">The same job in both languages differs in three details</h2>\n<p>Here is one small job written twice. On a sheet called Invoices, color every unpaid row whose due date has passed. Then copy those rows to a fresh sheet called Overdue. Both assume the table starts in cell A1, with due dates in column C and the status in column D.</p>\n<p>The VBA version:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"vb\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">Sub</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> CopyOverdue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">    ' Table starts in A1: headers, due date in C, status in D.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Dim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> src </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Worksheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, dst </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Worksheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, data </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Range</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Dim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Long</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, outRow </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Long</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">As</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> Variant</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Set </span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">src </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ThisWorkbook.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Worksheets</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Invoices\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Set </span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">data </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> src.UsedRange</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    Application</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.DisplayAlerts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> False</span><span style=\"color:#6A737D;--shiki-dark:#768390\">   ' delete without the prompt</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    On Error Resume Next                </span><span style=\"color:#6A737D;--shiki-dark:#768390\">' no Overdue sheet yet is fine</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    ThisWorkbook.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Worksheets</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Overdue\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).Delete</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    On Error GoTo </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    Application</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.DisplayAlerts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> True</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Set </span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dst </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ThisWorkbook.Worksheets.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">Add</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(After:</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">src)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    dst.Name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"Overdue\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Rows</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).Copy dst.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Range</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"A1\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    outRow </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 2</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    For</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 2</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> To</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.Rows.Count</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Cells</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">3</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).Value</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        If</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> VarType</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(due) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> vbDate </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">Then</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            If</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Date </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">And</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Cells</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">4</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).Text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;&gt;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"Paid\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> Then</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Rows</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r).Interior.Color </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> RGB</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">255</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">199</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">206</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Rows</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r).Copy dst.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Cells</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(outRow, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                outRow </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> outRow </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            End If</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        End If</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    Next</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">End Sub</span></span></code></pre>\n<p>The Office Scripts version:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"typescript\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> main</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">workbook</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#6F42C1;--shiki-dark:#F69D50\"> ExcelScript</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">Workbook</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">  // Table starts in A1: headers, due date in C, status in D.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> src</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> workbook.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getWorksheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Invoices\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> data</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> src?.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getUsedRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">src </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> !</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">data) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"No Invoices sheet, or it is empty\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> values</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getValues</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  workbook.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getWorksheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Overdue\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)?.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">delete</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> dst</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> workbook.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">addWorksheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Overdue\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  dst.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"A1\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">copyFrom</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">));</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">  // Excel stores dates as serial numbers: day 25569 is January 1, 1970.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> now</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Date</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> today</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Date.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">UTC</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(now.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFullYear</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), now.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getMonth</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), now.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 86400000</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 25569</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> outRow </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; r </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> values.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">++</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> due</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> values[r][</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">typeof</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">===</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"number\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> &amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> today </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> values[r][</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">3</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"Paid\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFormat</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setColor</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"#FFC7CE\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      dst.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getCell</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(outRow, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">copyFrom</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      outRow</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">++</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span></code></pre>\n<p>The two look alike, but three details differ. VBA reads a date cell as a Date, while <a href=\"https://learn.microsoft.com/en-us/javascript/api/office-scripts/excelscript/excelscript.range\"><code>getValues</code></a> returns a plain number. So the script builds today’s serial number itself, with the 25569 offset from Microsoft’s <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/resources/samples/javascript-dates\">date samples</a>.</p>\n<p>The macro reads cell by cell, while the script reads every value once, before the loop, as Microsoft’s <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/web-client-performance\">performance guide</a> advises. And run from a flow, its “today” is the UTC date. For an office west of UTC, that date turns over before local midnight, so a late-day run counts invoices due today as overdue.</p>\n<p>What we checked: the script passes the TypeScript compiler in strict mode against the Office Scripts signatures it uses. That was compiler version 5.9.3, not the 4.0.3 that Office Scripts runs.</p>\n<p>We ran it once against a mock workbook of six rows. It marked and copied the two overdue rows and left the paid row, the row due today, the blank date and the text date alone. We did not run either version in Excel.</p>\n<p>Two gaps remain. Neither version clears old highlights, so a row paid since the last run stays colored. And a plain number in column C counts as a date in the script but not in the macro.</p>\n<h2 id=\"python-in-excel-is-not-a-replacement-for-either\">Python in Excel is not a replacement for either</h2>\n<p>Python in Excel “runs the Python code used by Excel in a secure container on the Microsoft Cloud,” <a href=\"https://support.microsoft.com/office/33cc88a4-4a87-485e-9ff9-f35958278327\">Microsoft says</a>. The code “doesn’t have access to your computer, devices, or account” and “doesn’t have network access.” It reads only the data a formula references.</p>\n<p>It <a href=\"https://support.microsoft.com/en-us/office/python-in-excel-availability-781383e6-86b9-4156-84fb-93e786f7cab0\">runs</a> for Enterprise and Business users on Windows and Mac and in Excel on the web, but not on iPad, iPhone or Android. Faster premium compute needs an add-on license. Use it to analyze a range. With no access to the PC or the network, it can’t replace a macro that saves files or sends mail.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision comes down to one question per macro: does it touch anything outside this workbook?</p>\n<p><strong>Keep VBA</strong> when the macro drives Outlook or Word, touches files on disk or a DLL, or runs on a workbook event such as open or save. A script can’t do those itself. A flow around it can cover part of the gap: Microsoft’s <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/resources/samples/samples-overview\">samples</a> pair scripts with flows that send email and Teams messages.</p>\n<p>Clear the two security items while you are there. Keep shared macro files on OneDrive or SharePoint and open them with <strong>Open in Desktop App</strong> or through the sync client, so the Mark of the Web isn’t added. Then search the code for <code>.vbs</code> and <code>VBScript.RegExp</code> before Windows turns VBScript off.</p>\n<p><strong>Move to Office Scripts</strong> when the job stays inside the workbook and people open it in a browser, or when it should run with no one at the PC. Start with the Action Recorder, which captures cell edits and formatting, then tidy the code.</p>\n<p>The traps that bite during a move, all from Microsoft’s own pages:</p>\n<ul>\n<li>No <code>Workbook_Open</code> or change events. Use a scheduled or event-driven flow instead.</li>\n<li><code>getActiveWorksheet</code> and <code>getSelectedRange</code> assume a person is looking. In a flow, name the sheet.</li>\n<li><code>fetch</code> works when you press Run and fails in the flow.</li>\n<li>Dates come back as numbers, and flows run on UTC.</li>\n<li>The script lives in someone’s OneDrive. Move shared scripts to a SharePoint site before that person leaves.</li>\n<li>In Teams, scripts run only in Teams on the web.</li>\n</ul>\n<p>If you are already moving email or file jobs out of macros, our guides to <a href=\"https://28creation.com/articles/email-data-into-excel/\">email data into Excel</a> and <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments</a> use the same Power Automate pieces.</p>\n<p>This week, list every .xlsm file the office uses and mark each macro “outside” or “inside.” Move the inside ones that people open in a browser or want on a schedule first, and leave the rest where they work.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read Microsoft’s documentation and its VBScript post on October 20, 2026, and built the table from them. We type-checked the Office Script and ran it once against a six-row mock workbook in Node. Neither version ran in Excel, and the VBA was not run at all.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/resources/vba-differences\">Differences between Office Scripts and VBA macros</a> (updated November 16, 2023)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/testing/platform-limits\">Platform limits, requirements, and error messages for Office Scripts</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/overview/excel\">Office Scripts in Excel</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/external-calls\">External API call support in Office Scripts</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/overview/script-storage\">Office Scripts file storage and ownership</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/typescript-restrictions\">TypeScript restrictions in Office Scripts</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/power-automate-integration\">Run Office Scripts with Power Automate</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/testing/power-automate-troubleshooting\">Troubleshoot Office Scripts running in Power Automate</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/develop/web-client-performance\">Improve the performance of your Office Scripts</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/dev/scripts/resources/samples/javascript-dates\">JavaScript Date samples</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/javascript/api/office-scripts/excelscript/excelscript.range\">ExcelScript.Range</a>, <a href=\"https://learn.microsoft.com/en-us/javascript/api/office-scripts/excelscript/excelscript.worksheet\">ExcelScript.Worksheet</a> and <a href=\"https://learn.microsoft.com/en-us/javascript/api/office-scripts/excelscript/excelscript.workbook\">ExcelScript.Workbook</a> reference</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/microsoft-365/admin/manage/manage-office-scripts-settings\">Manage Office Scripts settings</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/office/453ab58d-708f-40a9-ab87-99a743bfa69a\">Record your actions as Office Scripts</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/work-with-vba-macros-in-excel-for-the-web-98784ad0-898c-43aa-a1da-4f0fb5014343\">Work with VBA macros in Excel for the web</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/file-formats-that-are-supported-in-excel-0943ff2c-6014-4e8d-aaea-b83d51d46247\">File formats that are supported in Excel</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/vba/library-reference/concepts/getting-started-with-vba-in-office\">Getting started with VBA in Office</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/declare-statement\">Declare statement</a> and <a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/filesystemobject-object\">FileSystemObject object</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/office/vba/excel/concepts/events-worksheetfunctions-shapes/using-events-with-excel-objects\">Using events with Excel objects</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/microsoft-365-apps/security/internet-macros-blocked\">Macros from the internet are blocked by default in Office</a></li>\n<li>Microsoft 365 Developer Blog, Wenjun Gong, <a href=\"https://devblogs.microsoft.com/microsoft365dev/how-to-prepare-vba-projects-for-vbscript-deprecation/\">Prepare your VBA projects for VBScript deprecation in Windows</a> (September 9, 2025)</li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/office/33cc88a4-4a87-485e-9ff9-f35958278327\">Data security and Python in Excel</a> and <a href=\"https://support.microsoft.com/en-us/office/python-in-excel-availability-781383e6-86b9-4156-84fb-93e786f7cab0\">Python in Excel availability</a></li>\n<li>Microsoft Learn, VBA reference: <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.worksheet.delete\">Worksheet.Delete</a>, <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.sheets.add\">Sheets.Add</a>, <a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/vartype-function\">VarType</a>, <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.range.value2\">Range.Value2</a>, <a href=\"https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/date-function\">Date</a>, <a href=\"https://learn.microsoft.com/en-us/office/vba/api/excel.interior.color\">Interior.Color</a></li>\n<li>28creation check, October 20, 2026: the Office Script above, type-checked with the TypeScript 5.9.3 compiler in strict mode against the Office Scripts signatures it uses, and run once against a mock workbook of six rows in Node. Not run in Excel; the VBA was not run.</li>\n</ul>\n",
      "summary": "VBA reaches the whole PC but not the browser. Office Scripts can run on a schedule but sees only the workbook. A table from Microsoft's docs, and what to move.",
      "image": "https://28creation.com/og/office-scripts-vs-vba.jpg",
      "date_published": "2026-10-22T15:00:00.000Z",
      "date_modified": "2026-10-22T15:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/emudhra-exits-tls-trust-store-updates/",
      "url": "https://28creation.com/articles/emudhra-exits-tls-trust-store-updates/",
      "title": "eMudhra quits TLS after key flaw; old trust stores still at risk",
      "content_html": "<h2>The short read</h2><p><strong>eMudhra is leaving web certificates five days after a researcher showed anyone could recover eight of its CA keys.</strong></p><ul><li>Chrome, Firefox, Apple and Microsoft block the eight CAs, but tools that skip revocation checks can still trust them until their CA bundles update.</li></ul><h3>What you can do with this</h3><blockquote><p>Update ca-certificates, Node.js, Java, Go, Python certifi and Docker Desktop to the fixed versions, then rebuild container images. That closes a path for forged certificates against your services.</p></blockquote><hr><p>eMudhra, an Indian certificate authority, <a href=\"https://emudhra.com/en/press-release/emudhra-to-exit-global-ssl-tls-certificate-business\">said on October 22</a> that its board has decided to “voluntarily withdraw from the global SSL/TLS certificate business.” <a href=\"https://finance.yahoo.com/technology/articles/indian-firm-emudhra-exits-global-123540524.html\">Reuters</a> reported the exit the same day. It follows the discovery that eight of its intermediate CA keys could be recovered by anyone. Browsers already block those certificates, so the risk that remains sits in servers and containers that never got the update.</p>\n<h2 id=\"anyone-could-rebuild-the-keys\">Anyone could rebuild the keys</h2>\n<p>On October 17, a researcher named Wayne signed a message with the private key of each of the eight intermediates and reported it. That is the account of the <a href=\"https://emudhra.we-have-a-problem.com/\">disclosure site</a> run by Mozilla’s Dennis Jackson. The keys were generated wrongly in April 2025, so “anyone can recover them from the certificates.” <a href=\"https://www.helpnetsecurity.com/2026/10/21/emudhra-certificates-private-keys-compromised/\">Help Net Security</a> reports each key was smaller than 256, so recovering one took at most 256 guesses.</p>\n<p>eMudhra’s <a href=\"https://emudhra.com/en/press-release/emudhra-statement-distrust-global-tls-intermediate-root-ca\">own statement</a> blames “a flaw in the KGC” that produced weak keys alongside the strong ones.</p>\n<p>With one of those keys, an attacker “could impersonate any website or service,” the disclosure site says. The tooling is public: mitmproxy, a widely used interception proxy, <a href=\"https://github.com/mitmproxy/mitmproxy/commit/169234499088fc2c414644a67efaed3d64ae2e3b\">added an option</a> to “sign certificates with a compromised eMudhra intermediate CA key.”</p>\n<h2 id=\"browsers-moved-in-days-everything-else-did-not\">Browsers moved in days; everything else did not</h2>\n<p>Firefox and Chrome have blocked the certificates since October 17, Apple since October 19 and Microsoft since October 20, per the disclosure site. Chrome used CRLSets, and the Chrome Root Program <a href=\"https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/0R3XLJL6FuE\">wrote</a> that Root Store version 43 will remove the eMudhra root. No Certificate Transparency log holds entries for these intermediates, the site notes. So CT-enforcing browsers “could not have been attacked.”</p>\n<p>But many other clients check neither revocation nor CT. Revocation alone does nothing for them. They need a software update.</p>\n<p>The disclosure site runs test servers signed by the broken intermediates and publishes <a href=\"https://statistics-emudhra.we-have-a-problem.com/stats.json\">how many clients still connect</a>. We read that file at 14:37 UTC on October 22.</p>\n<p>In the 71 minutes since its counter started at 13:26 UTC, it logged 50,866 successful connections from clients that still trusted the chains. The largest groups were iOS WebViews (9,882), curl 8 (8,323), Safari on iOS (7,036 across versions 26 and 27) and the Zabbix monitoring agent (2,273). These count connections rather than devices, from people who chose to run the test. Still, they show that a vendor block does not reach every client at once.</p>\n<h2 id=\"emudhras-account-and-what-stays\">eMudhra’s account, and what stays</h2>\n<p>eMudhra says it revoked the CAs within 14 hours of discovery, that no customer certificates were issued from them and that its infrastructure was not breached. It puts global SSL/TLS at about 1% of revenue and calls the scrutiny “out of proportion.”</p>\n<p>Its G1 RSA roots keep serving customers during the transition. It will replace affected certificates at no cost. Reuters reports the shares fell 40% over two days.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ul>\n<li><strong>Update trust bundles.</strong> Fixed versions on the disclosure site include Ubuntu 24.04 ca-certificates 20260816ubuntu0.24.04.1 and Debian 13 20250419+deb13u1. For runtimes, Node.js 22.23.4 or 24.22.1, Java 21.0.13.0.2 or 17.0.21.0.2, and Go 1.27.3 or 1.26.10. For tools, Python certifi 2026.10.19, pip 26.2.2 and Docker Desktop 4.96.2.</li>\n<li><strong>Rebuild images.</strong> A container keeps the bundle it was built with. Cloudflare’s <a href=\"https://blog.cloudflare.com/emudhra-certificate-compromise\">advice</a> is to update and redeploy, and not to “resolve these errors by disabling verification.”</li>\n<li><strong>Check macOS scripts.</strong> On macOS, the built-in curl, git and python3 read /etc/ssl/cert.pem without checking revocation, and that file has not been updated, per the disclosure site. It gives a command to remove the two affected roots.</li>\n<li><strong>Test your real path.</strong> Run <code>curl https://dv-g3b-emudhra.we-have-a-problem.com/</code> from the machine that matters. A certificate error is the good result.</li>\n</ul>\n<p>The trade-off: the site warns that removing roots by hand is “NOT recommended for ordinary users.” Its commands also drop AAA Certificate Services, the root Sectigo used to cross-sign the eight. Where a vendor update exists, take that instead.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://emudhra.com/en/press-release/emudhra-to-exit-global-ssl-tls-certificate-business\">eMudhra: exit from the global SSL/TLS certificate business, October 22, 2026</a></li>\n<li><a href=\"https://emudhra.com/en/press-release/emudhra-statement-distrust-global-tls-intermediate-root-ca\">eMudhra: statement on distrust of select Global TLS intermediates and one root CA, October 21, 2026</a></li>\n<li><a href=\"https://emudhra.we-have-a-problem.com/\">eMudhra Intermediate CA Compromise disclosure site, October 20, 2026</a></li>\n<li><a href=\"https://statistics-emudhra.we-have-a-problem.com/stats.json\">Disclosure site client statistics, read October 22, 2026, 14:37 UTC</a></li>\n<li><a href=\"https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/0R3XLJL6FuE\">Mozilla dev-security-policy thread, with the Chrome Root Program update</a></li>\n<li><a href=\"https://blog.cloudflare.com/emudhra-certificate-compromise\">Cloudflare: 8 eMudhra certificates are compromised</a></li>\n<li><a href=\"https://github.com/mitmproxy/mitmproxy/commit/169234499088fc2c414644a67efaed3d64ae2e3b\">mitmproxy commit adding the –emudhra option</a></li>\n<li><a href=\"https://finance.yahoo.com/technology/articles/indian-firm-emudhra-exits-global-123540524.html\">Reuters via Yahoo Finance: eMudhra exits global digital certificates business</a></li>\n<li><a href=\"https://www.helpnetsecurity.com/2026/10/21/emudhra-certificates-private-keys-compromised/\">Help Net Security: eMudhra signing keys exposed</a></li>\n</ul>\n",
      "summary": "Browsers already block eMudhra's eight broken CAs. Servers, scripts, containers and macOS command-line tools may still trust them until you update.",
      "image": "https://28creation.com/og/emudhra-exits-tls-trust-store-updates.jpg",
      "date_published": "2026-10-22T14:42:02.000Z",
      "date_modified": "2026-10-22T14:42:02.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/mail-merge-google-sheets/",
      "url": "https://28creation.com/articles/mail-merge-google-sheets/",
      "title": "Mail merge from Google Sheets to Gmail, with a PDF for each row",
      "content_html": "<h2>The short read</h2><p><strong>Gmail&apos;s built-in mail merge sends the same attachment to everyone and can&apos;t put a name in the subject line.</strong></p><ul><li>Our Apps Script turns each sheet row into its own PDF and Gmail message, then marks the row so a second run skips it.</li><li>We tested it on 5 invented rows with stand-ins for Google&apos;s services in Node.js, not in a live Google account.</li></ul><h3>What you can do with this</h3><blockquote><p>Before a real send, set testTo to your own address and run the script once. You get up to 3 sample emails to check, and the sheet stays untouched.</p></blockquote><hr><p>Google gives you two ways to mail merge from a Google Sheet. Gmail’s own <a href=\"https://support.google.com/mail/answer/12921167?hl=en\">mail merge</a> reads names and addresses from a sheet and sends each person a separate copy. It works only on seven Workspace plans, and it attaches the same file to every copy. If each person needs their own document, such as a certificate or an invoice, you need a short Apps Script instead.</p>\n<p>This guide covers both. The script below fills a Google Docs template for each row, saves it as a PDF, emails it from Gmail and writes a status back to the sheet, so a second run skips rows already sent. We tested its logic on 5 sample rows in Node.js, with stand-ins for Google’s services, because we had no Google account to run it in. Limits are from Google’s pages, read on October 20, 2026.</p>\n<h2 id=\"gmails-mail-merge-sends-one-message-many-times\">Gmail’s mail merge sends one message many times</h2>\n<p>Gmail’s mail merge works on the email itself. Google’s help page says “each recipient gets a unique copy of the email in which the merge tags are replaced with your details.” Tags come from your column headers: a column named <code>Course</code> becomes <code>@Course</code> in the message. Recipients can’t see who else got it, and their replies arrive in separate threads.</p>\n<p>That design sets three limits. You “can’t use merge tags in” subject lines or in linked text. A file you attach is the same file for everyone, and Google warns it is stored in every copy: 500 recipients with a 10 MB attachment use “about 5GB of your storage.” Mail merge also can’t be used for scheduled sends.</p>\n<p>So the built-in tool fits an announcement or an event invitation. It does not fit “here is your certificate, Ada”, where the document itself changes per person.</p>\n<p>Google Docs has no built-in merge from a sheet that we could find. Its <a href=\"https://support.google.com/docs/answer/13584275?hl=en\">variable chips</a> let you type a value once and see it everywhere in one document, on eligible work or school accounts. You still enter each value by hand. Google’s own merge into Docs is a <a href=\"https://developers.google.com/docs/api/samples/mail-merge?hl=en\">Docs API sample</a> written in Python, which is a developer project, not a menu.</p>\n<h2 id=\"1-pick-the-route-by-plan-and-attachment\">1. Pick the route by plan and attachment</h2>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Gmail mail merge</th>\n<th>Our Apps Script</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Accounts</td>\n<td>Workspace Individual, Business Standard and Plus, Enterprise Standard and Plus, Education Standard and Plus</td>\n<td>Any Google account</td>\n</tr>\n<tr>\n<td>A document per person</td>\n<td>No: one attachment, same for all</td>\n<td>Yes: a PDF made from a Docs template</td>\n</tr>\n<tr>\n<td>Tags in the subject</td>\n<td>No</td>\n<td>Yes</td>\n</tr>\n<tr>\n<td>Emails a day</td>\n<td>Up to 1,500 recipients</td>\n<td>100 on personal Gmail, 1,500 on Workspace</td>\n</tr>\n<tr>\n<td>Unsubscribe link</td>\n<td>Added to every email</td>\n<td>None; add your own line</td>\n</tr>\n<tr>\n<td>Remembers who got it</td>\n<td>Sent folder banner</td>\n<td><code>Status</code> column in the sheet</td>\n</tr>\n</tbody>\n</table>\n<p>Our view: use Gmail’s mail merge when everyone gets the same message and you have an eligible plan. It needs no code and adds the unsubscribe link for you. Use the script when each person needs their own PDF, or when you are on personal Gmail or Business Starter, which are not on Google’s list of eligible plans.</p>\n<p>Workspace admins also control who you can reach. Google’s <a href=\"https://workspaceupdates.googleblog.com/2023/06/google-sheets-now-integrated-with-gmail.html\">2023 announcement</a> said Enterprise and Education editions limit mail merge to internal recipients by default, while Business Standard and Plus allow external recipients. Admins can turn external sending on for a group or organizational unit.</p>\n<h2 id=\"2-set-up-the-sheet-with-one-row-per-person\">2. Set up the sheet with one row per person</h2>\n<p>Both routes read the same kind of sheet: a header row, then one row per person. Gmail’s mail merge adds two rules. “Contact information must be in the first tab of your spreadsheet and can only contain text.” And a header with characters other than letters or numbers gets a tag by its position, such as <code>@A</code> for the first column, so plain headers like <code>FirstName</code> are easier to use.</p>\n<p>For the script, the tab is named <code>Recipients</code> and needs a column called <code>Email</code>. Any other header becomes a placeholder. Our sample sheet:</p>\n<table>\n<thead>\n<tr>\n<th>Email</th>\n<th>Name</th>\n<th>Course</th>\n<th>Date</th>\n<th>Price</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><a href=\"mailto:ada@lind.example\">ada@lind.example</a></td>\n<td>Ada Lind</td>\n<td>Bookkeeping basics</td>\n<td>10/14/2026</td>\n<td>$1,200.00</td>\n</tr>\n<tr>\n<td><a href=\"mailto:carl@holm.example\">carl@holm.example</a></td>\n<td>Carl Holm Jr.</td>\n<td>Payroll 101</td>\n<td>10/15/2026</td>\n<td>$900.00</td>\n</tr>\n</tbody>\n</table>\n<p>The script reads each cell as it is shown, through <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/range\"><code>getDisplayValues()</code></a>, which “takes into account date, time and currency formatting.” So <code>$1,200.00</code> arrives in the PDF as <code>$1,200.00</code>, not as <code>1200</code>. Format a column in the sheet, and that format is what the reader sees.</p>\n<h2 id=\"3-send-with-gmails-mail-merge-when-one-message-fits-all\">3. Send with Gmail’s mail merge when one message fits all</h2>\n<p>If the built-in route fits, these are Google’s steps:</p>\n<ol>\n<li>In Gmail, click <strong>Compose</strong>. On the right of the <strong>To</strong> line, click <strong>Use mail merge</strong> and turn on <strong>Mail Merge</strong>.</li>\n<li>Click <strong>Add from a spreadsheet</strong>, pick the sheet and click <strong>Insert</strong>.</li>\n<li>Choose the columns for <strong>Email</strong>, <strong>First name</strong> and, if you have one, <strong>Last name</strong>. Click <strong>Finish</strong>.</li>\n<li>In the message, type <code>@</code>, pick a tag such as <code>@Course</code>, and press Enter.</li>\n<li>Send. The message appears in Sent with a “Sent with mail merge” banner.</li>\n</ol>\n<p>If a recipient is missing a value for a tag, Gmail shows an error. You can type a default there, so “Hi @firstname” becomes “Hi friend”, or fix the sheet. Google’s page describes no preview step, so send one merge to yourself first with your own address as the only row.</p>\n<p>Two catches. With recipients from a sheet you “can’t include Cc or Bcc recipients.” And Gmail adds an unsubscribe link to each email but you “can’t get a list of all unsubscribed recipients.” You get one notification email per unsubscribe, so keep a note of them.</p>\n<h2 id=\"4-build-the-docs-template-with-placeholders-in-the-body\">4. Build the Docs template with placeholders in the body</h2>\n<p>The script route starts with a normal Google Doc. Write the certificate, letter or invoice once, and put each column name in double braces where the value should go: <code>{{Name}}</code>, <code>{{Course}}</code>, <code>{{ Date }}</code>. Spaces inside the braces are allowed. The names must match the headers exactly, capitals included.</p>\n<p>Our test template had a header line, four body lines and a footer:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>Example Training</span></span>\n<span class=\"line\"><span>Certificate</span></span>\n<span class=\"line\"><span>{{Name}}</span></span>\n<span class=\"line\"><span>has completed {{Course}} on {{ Date }}.</span></span>\n<span class=\"line\"><span>Fee paid: {{Price}}</span></span>\n<span class=\"line\"><span>Issued to {{Name}}</span></span></code></pre>\n<p>The script fills the body, the header and the footer. It reads only the first tab: Google’s reference says <a href=\"https://developers.google.com/apps-script/reference/document/document\"><code>getBody()</code></a> “retrieves the first tab’s Body”, and the same holds for the header and footer. Keep the template on one tab.</p>\n<p>Copy the Doc’s ID from its address bar, the long string between <code>/d/</code> and <code>/edit</code>. Make a Drive folder for the PDFs and copy its ID, the part of its address after <code>folders/</code>.</p>\n<h2 id=\"5-paste-the-script-and-set-config\">5. Paste the script and set CONFIG</h2>\n<p>Open the sheet and choose <strong>Extensions</strong>, then <strong>Apps Script</strong>. That makes a script <a href=\"https://developers.google.com/apps-script/guides/bound\">bound to the sheet</a>. Paste the code over the empty <code>myFunction</code>, edit <code>CONFIG</code>, save, and reload the sheet. A <strong>Mail merge</strong> menu appears.</p>\n<p>The first run asks you to authorize the script. Sending through <code>GmailApp</code> needs full Gmail access, the <code>https://mail.google.com/</code> scope. Google’s <a href=\"https://developers.google.com/apps-script/samples/automations/mail-merge?hl=en\">own mail merge sample</a> notes you may see “This app isn’t verified”; for a script you wrote, choose <strong>Advanced</strong>, then go to the project.</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"javascript\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Mail merge: each row of a sheet becomes a PDF made from a Docs template and a Gmail message with it attached.</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Paste into Extensions &gt; Apps Script in the spreadsheet, edit CONFIG, reload the sheet, use the Mail merge menu.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  sheet: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Recipients'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,                    </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// tab with a header row and a column named Email</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  templateId: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'PASTE_TEMPLATE_DOC_ID'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,    </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// the Docs template, with {{Column name}} placeholders</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  folderId: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'PASTE_DRIVE_FOLDER_ID'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,      </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// where the PDFs are saved</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  fileName: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Certificate {{Name}}'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  subject: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Your certificate, {{Name}}'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  body: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Hi {{Name}},</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Your certificate for {{Course}} is attached.</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Best regards,</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Anna'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  senderName: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Anna at Example Training'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  optional: [],                           </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// columns that may be empty; any other empty placeholder skips the row</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  skipDuplicates: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">true</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,                   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// a second row with an address already sent to is skipped</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  keepDocs: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">false</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,                        </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// true keeps each filled Doc copy next to its PDF</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  testTo: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,                             </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// your own address: sends up to 3 rows to you and writes nothing back</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  maxPerRun: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">50</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">};</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> STATUS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'Status'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">SENT_AT</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'Sent at'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> TIME_BUDGET_MS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 4.5</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 60</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1000</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// stop well before the 6-minute limit per run</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> onOpen</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getUi</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">createMenu</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Mail merge'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">addItem</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Send pending rows'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'sendMerge'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">addToUi</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> sendMerge</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> lock</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> LockService.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getScriptLock</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">lock.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">tryLock</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">10000</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> say</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Another run is still sending. Try again in a minute.'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> { </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> say</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">run</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()); } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">finally</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> { SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">flush</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); lock.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">releaseLock</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> run</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> started</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Date.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">now</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sheet</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getActiveSpreadsheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getSheetByName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.sheet);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">sheet) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'No tab named '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.sheet);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> data</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDataRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDisplayValues</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> header</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">h</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> h.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">());</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">indexOf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Email'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'No column named Email'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> test</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Boolean</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.testTo);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">STATUS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">SENT_AT</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">test </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">indexOf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) { header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">push</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name); sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, header.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setValue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name); }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> emailCol</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">indexOf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Email'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">statusCol</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">indexOf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">STATUS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">sentAtCol</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">indexOf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">SENT_AT</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> mark</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">row</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">text</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> test </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> console.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">log</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">`row ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">row</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}: ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">text</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, statusCol </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setValue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(text);</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> seen</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();                 </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// address -&gt; first row that was sent, or may have been</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">forEach</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">i</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> { </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(Sent</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Sending</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Check)/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[statusCol] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">norm</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[emailCol]), i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); });</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> unsure</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">i</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(Sending</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Check)/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[statusCol] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)).</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> quota </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> MailApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRemainingDailyQuota</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), sent </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, stop </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; i</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">++</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> row</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> i </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data[i], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">status</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r[statusCol] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(Sent</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Sending</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Check)/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(status)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// done, or waiting for a person to look</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (r.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">every</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">v</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">===</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;                 </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// blank row</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> email</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> norm</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[emailCol]);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">email) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Skipped: no email'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s@,;]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">@</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s@,;]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s@,;]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(email)) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Skipped: not one email address'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.skipDuplicates </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">has</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(email)) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Skipped: same address as row '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">get</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(email)); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (sent </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (test </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 3</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> :</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.maxPerRun)) { stop </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'the per-run limit'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">break</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (Date.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">now</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> started </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> TIME_BUDGET_MS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) { stop </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'the time budget'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">break</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (quota </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) { stop </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'the daily email quota'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">break</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> values</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">create</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">null</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);    </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// no inherited keys, so {{constructor}} is not a column</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">forEach</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">h</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">c</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> { values[h] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (r[c] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); });</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdf, subject, body;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      subject </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> fill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.subject, values);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      body </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> fill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.body, values);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      pdf </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> makePdf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(values);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">catch</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (e) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Error: '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> e.message); </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }     </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// nothing was sent; the next run tries again</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">test) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Sending'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">flush</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); }  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// a crash after this line leaves a visible trace</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      GmailApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">sendEmail</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(test </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.testTo </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> email, subject, body, { attachments: [pdf], name: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.senderName });</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">catch</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (e) {</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">      mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Check: '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> e.message </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '. Look in Sent, then clear this cell to retry.'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    quota</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">--</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; sent</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">++</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(email, row);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">test) { </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">mark</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Sent'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row, sentAtCol </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setValue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Date</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()); }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> `Sent ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">sent</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}`</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (test </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ' test emails to '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.testTo </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (stop </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> `. Stopped at ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">stop</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}; run again to continue.`</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> :</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '.'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (unsure </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ` Rows marked Sending or Check: ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">unsure</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}. Look in Sent before clearing them.`</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> :</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> say</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">msg</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  console.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">log</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(msg);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getActiveSpreadsheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toast</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(msg, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Mail merge'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">15</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> makePdf</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">values</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> folder</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> DriveApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFolderById</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.folderId);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> name</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> fill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.fileName, values);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> copy</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> DriveApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFileById</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.templateId).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">makeCopy</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name, folder);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  try</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> doc</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> DocumentApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">openById</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(copy.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getId</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">());</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> part</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [doc.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getBody</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), doc.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getHeader</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), doc.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFooter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()]) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">part) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> keys</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">([</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">part.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">matchAll</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\{\\{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">{}]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">)</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\}\\}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">g</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)].</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">m</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> m[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]));</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> keys) </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replaceLiteral</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(part, key, </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">lookup</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(values, key));</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> left</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">match</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\{\\{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">{}]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\}\\}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (left) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'template still contains '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> left[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    doc.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">saveAndClose</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> pdf</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> copy.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getAs</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'application/pdf'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '.pdf'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// getAs alone cuts names after a period</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.testTo) folder.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">createFile</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pdf);   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// a test run keeps no PDFs</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pdf;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">finally</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.keepDocs) copy.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setTrashed</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">true</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Inserts the value as plain text. replaceText() would read the template's placeholder as a regular expression.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> replaceLiteral</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">part</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">value</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> pattern</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">{</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">{</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">s*'</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> key.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replace</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[.*+?^${}()|[</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\]\\\\</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">g</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">$&amp;'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">s*</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">findText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pattern); hit; hit </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">findText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pattern)) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> text</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getElement</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">asText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> start</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">isPartial</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getStartOffset</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> end</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">isPartial</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getEndOffsetInclusive</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (value) text.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">insertText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(start, value);   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// an empty optional value only deletes</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    text.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">deleteText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(start </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, end </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> value.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> fill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">template</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">values</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> template.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replace</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\{\\{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">{}]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">)</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\}\\}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">g</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, (</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">_</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> lookup</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(values, key));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> lookup</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">values</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(key </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> values)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'no column named '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> key);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> v</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> values[key];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> !</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.optional.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">includes</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(key)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'empty '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> key);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (v.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">includes</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'{{'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(key </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ' contains {{'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> v;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> norm</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">v</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) { </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> String</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); }</span></span></code></pre>\n<h2 id=\"the-status-column-stops-a-second-run-from-sending-twice\">The status column stops a second run from sending twice</h2>\n<p><strong>The status column is the memory.</strong> Before each send the script writes <code>Sending</code> and calls <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/spreadsheet-app\"><code>SpreadsheetApp.flush()</code></a>, which “applies all pending Spreadsheet changes.” After the send it writes <code>Sent</code> and the time. Any later run skips rows marked <code>Sent</code>, <code>Sending</code> or <code>Check</code>.</p>\n<p><strong>A failed send never retries by itself.</strong> If <code>sendEmail</code> throws an error, the row gets <code>Check</code> and the error text. If the run dies between the send and the <code>Sent</code> mark, the row stays <code>Sending</code>. Neither tells you whether the email left. So the script waits for you: look in Gmail’s Sent folder, and clear the cell only if the email is not there. We chose a missed email over a double one.</p>\n<p><strong>Duplicates and bad cells are skipped.</strong> A second row with the same address, in any capitals, gets <code>Skipped: same address as row 2</code>. Set <code>skipDuplicates</code> to <code>false</code> if one person should get two documents. A cell with two addresses is refused, because <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-app\"><code>sendEmail</code></a> takes “the addresses of the recipient” and would send to both. The check catches a malformed address, not a dead one. Clean old lists first: Gmail’s help for personal accounts says that after many emails bounce, you “may be able to send emails again after 24 hours.”</p>\n<p><strong>Empty values stop the row.</strong> An empty cell behind a placeholder gives <code>Error: empty Name</code> instead of “Hi ,”. List columns that may be empty in <code>optional</code>. A misspelled placeholder such as <code>{{Nmae}}</code> gives <code>Error: no column named Nmae</code>. Error rows are tried again on the next run, since nothing was sent.</p>\n<p><strong>Values go in as plain text.</strong> Apps Script’s <code>replaceText()</code> reads its search pattern as a regular expression, Google’s <a href=\"https://developers.google.com/apps-script/reference/document/body\">Body reference</a> says. It doesn’t say how the replacement treats characters like <code>$</code>. So the script finds each placeholder and inserts the value with <code>insertText()</code> instead. A value like <code>$1,200.00</code> stays as typed.</p>\n<p><strong>PDF names keep their periods.</strong> Google’s <a href=\"https://developers.google.com/apps-script/reference/drive/file\">File reference</a> says <code>getAs()</code> treats the text after the last period as an extension and replaces it: “ShoppingList.12.25.2014” becomes “ShoppingList.12.25.pdf”. A row for “Carl Holm Jr.” would lose part of its name, so the script sets the name again.</p>\n<p><strong>Two runs can’t overlap.</strong> A <a href=\"https://developers.google.com/apps-script/reference/lock/lock-service\">script lock</a> “prevents any user from concurrently running a section of code.” Click the menu twice and the second click waits 10 seconds, then gives up.</p>\n<h2 id=\"6-send-a-test-to-yourself-first\">6. Send a test to yourself first</h2>\n<p>Set <code>testTo</code> to your own address and run <strong>Mail merge</strong>, then <strong>Send pending rows</strong>. The script makes the PDFs for the first 3 sendable rows and emails all 3 to you. It writes nothing to the sheet and saves no PDFs in the folder.</p>\n<p>Open each PDF. Check the name, the dates and the money columns, and that no <code>{{</code> is left anywhere. Then clear <code>testTo</code>.</p>\n<h2 id=\"7-send-for-real-and-read-the-status-column\">7. Send for real and read the Status column</h2>\n<p>Run <strong>Send pending rows</strong> again. A popup in the corner of the sheet reports the result, such as <code>Sent 3.</code> Each row now has one of these:</p>\n<table>\n<thead>\n<tr>\n<th>Status</th>\n<th>Meaning</th>\n<th>What to do</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><code>Sent</code></td>\n<td>Email sent; <code>Sent at</code> has the time</td>\n<td>Nothing</td>\n</tr>\n<tr>\n<td><code>Skipped: ...</code></td>\n<td>No email, two addresses, or a duplicate</td>\n<td>Fix the cell; the next run tries again</td>\n</tr>\n<tr>\n<td><code>Error: ...</code></td>\n<td>Template or data problem; nothing was sent</td>\n<td>Fix it; the next run tries again</td>\n</tr>\n<tr>\n<td><code>Check: ...</code></td>\n<td>Gmail gave an error during the send</td>\n<td>Look in Sent, then clear the cell to retry</td>\n</tr>\n<tr>\n<td><code>Sending</code></td>\n<td>The run stopped mid-send</td>\n<td>Look in Sent, then clear the cell to retry</td>\n</tr>\n</tbody>\n</table>\n<p>The script stops after 50 emails a run, after 4.5 minutes, or when Google’s daily email count runs out. The popup says which. Run it again to continue.</p>\n<h2 id=\"our-test-5-rows-10-cases-no-google-account\">Our test: 5 rows, 10 cases, no Google account</h2>\n<p>We could not run Apps Script itself. So we ran the script unchanged in Node.js 22 with small stand-ins for <code>SpreadsheetApp</code>, <code>DocumentApp</code>, <code>DriveApp</code>, <code>GmailApp</code>, <code>MailApp</code> and <code>LockService</code>, written to follow Google’s reference pages. The 5 rows were invented, on <code>.example</code> addresses: Ada, a row with no email, Carl Holm Jr., Ada again as <code>ADA@lind.example</code> with a trailing space, and Dana Berg with <code>(Q4) $1 \\n</code> in her name.</p>\n<p>What the 10 cases showed:</p>\n<table>\n<thead>\n<tr>\n<th>Case</th>\n<th>Result</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Three runs on the 5 rows</td>\n<td>Run 1: 3 sent, no-email row and duplicate skipped. Run 2: 0 sent. Run 3, after we filled in the missing email: that 1 row sent</td>\n</tr>\n<tr>\n<td>Gmail error on one row, once</td>\n<td>Row marked <code>Check</code>; not resent on 2 more runs</td>\n</tr>\n<tr>\n<td>Same, with <code>skipDuplicates</code> off</td>\n<td>Same result</td>\n</tr>\n<tr>\n<td>Run killed after a send, before <code>Sent</code></td>\n<td>Row stayed <code>Sending</code>; next run sent only the last row</td>\n</tr>\n<tr>\n<td>2 emails left in the day’s quota</td>\n<td>2 sent, stopped; the rest sent on the next run</td>\n</tr>\n<tr>\n<td>Test mode</td>\n<td>3 emails to the tester; sheet unchanged; no PDFs kept</td>\n</tr>\n<tr>\n<td><code>{{Nmae}}</code> in the template</td>\n<td>0 sent; every row <code>Error</code>; Doc copies trashed</td>\n</tr>\n<tr>\n<td><code>{{toString}}</code> in the template</td>\n<td>0 sent; <code>Error: no column named toString</code></td>\n</tr>\n<tr>\n<td>Two addresses in one cell</td>\n<td>Refused</td>\n</tr>\n<tr>\n<td>Lock already held</td>\n<td>Nothing sent</td>\n</tr>\n</tbody>\n</table>\n<p>In the first run, our stand-in sheet already showed <code>Sending</code> for each row at the moment its email went out. Values with <code>$</code>, brackets and a backslash landed in the PDF as typed, and “Carl Holm Jr.” kept his full name in the file name. We also broke the script on purpose six ways, such as removing the duplicate check. Each break made at least one case fail, so the cases catch those bugs.</p>\n<p>What it does not prove: anything about Google’s servers. Our stand-ins follow the documentation, but the real <code>findText()</code>, PDF export, quotas, timing and Gmail delivery were not part of the run. We don’t know how long one row takes in real Apps Script, so 50 a run is our guess, and the 4.5-minute stop is the backstop. Send yourself a test before any real list.</p>\n<h2 id=\"know-the-daily-limits-before-the-list-grows\">Know the daily limits before the list grows</h2>\n<p>Apps Script counts emails by recipient, per user, and the count resets “24 hours after the first request,” according to Google’s <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">quotas page</a> (updated September 3, 2026). That is not midnight. If the day’s first email went out at 3 p.m., the count resets around 3 p.m. the next day.</p>\n<table>\n<thead>\n<tr>\n<th>Limit</th>\n<th>Personal Gmail</th>\n<th>Google Workspace</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Script email recipients a day</td>\n<td>100</td>\n<td>1,500</td>\n</tr>\n<tr>\n<td>Documents created a day</td>\n<td>250</td>\n<td>1,500</td>\n</tr>\n<tr>\n<td>Files converted a day</td>\n<td>2,000</td>\n<td>4,000</td>\n</tr>\n<tr>\n<td>One run</td>\n<td>6 minutes</td>\n<td>6 minutes</td>\n</tr>\n<tr>\n<td>Attachments per email</td>\n<td>25 MB in total</td>\n<td>25 MB in total</td>\n</tr>\n<tr>\n<td>Gmail’s own daily send limit</td>\n<td>500</td>\n<td>2,000</td>\n</tr>\n<tr>\n<td>Gmail mail merge a day</td>\n<td>Not available</td>\n<td>1,500 recipients</td>\n</tr>\n</tbody>\n</table>\n<p>The first line decides most cases. On personal Gmail the script stops at 100 emails a day, even though Gmail itself allows <a href=\"https://support.google.com/mail/answer/22839?hl=en\">500</a>. On Workspace, 1,500 emails and 1,500 Doc copies line up, one each per row.</p>\n<p>We assume each template copy counts as a document created; Google’s page doesn’t say. Each PDF export likely counts as a conversion too, since the File reference points <code>getAs()</code> to that quota. The script checks what is left with <code>MailApp.getRemainingDailyQuota()</code>, which Google <a href=\"https://developers.google.com/apps-script/reference/mail/mail-app\">describes</a> as “the number of recipients you can send emails to for the rest of the day.”</p>\n<p>Gmail’s mail merge shares its 1,500 with your normal email. Once your regular and merge emails reach 1,500 in a day, the rest of the 2,000 “are reserved for regular emails only.” Google’s <a href=\"https://support.google.com/a/answer/166852?hl=en\">Workspace limits page</a> applies its limits “over a rolling 24-hour period” and notes that “Quotas might be different if you’re using Google Apps Script.” It doesn’t say how the script’s 1,500 and Gmail’s 2,000 combine, so plan as if they share one budget.</p>\n<p>Gmail’s own limits block more than the merge. A Workspace user who reaches one “can’t send new messages for up to 24 hours,” and a personal account can send again “within 1 to 24 hours.” Run a big merge on a quiet day, not before a deadline.</p>\n<h2 id=\"keep-newsletters-out-of-both-routes-past-5000-a-day\">Keep newsletters out of both routes past 5,000 a day</h2>\n<p>Both routes send from your own address, so its reputation carries the risk. Google’s <a href=\"https://support.google.com/a/answer/81126?hl=en\">sender guidelines</a> warn that when mail from a domain is often reported as spam, “future messages from you are more likely to be marked as spam.” They ask every sender to keep reported spam “below 0.3%,” and say plainly: “Don’t send messages to people who didn’t sign up to get messages from you.”</p>\n<p>Stricter rules start at volume. Google’s <a href=\"https://support.google.com/a/answer/14229414?hl=en\">sender FAQ</a> defines a bulk sender as one sending “close to 5,000 messages or more to personal Gmail accounts within a 24-hour period,” counted across the whole domain. One person can’t get there with these limits.</p>\n<p>A hypothetical team of four on one company domain, each running a 1,500 merge to customers on gmail.com, would send 6,000 and cross it. And “senders who meet the above criteria at least once are permanently considered bulk senders.”</p>\n<p>Bulk senders need SPF, DKIM and DMARC set up for the domain. Their marketing messages need one-click unsubscribe, done with <code>List-Unsubscribe</code> headers; a link in the body alone doesn’t count. Our script can’t add those headers: the options Google documents for <code>sendEmail</code> have no field for them.</p>\n<p>The rule applies only to marketing. “Transactional messages are excluded,” the FAQ says, naming confirmations as examples.</p>\n<p>That gives a simple split:</p>\n<ul>\n<li><strong>Certificates, invoices, offer letters, one per person:</strong> the script is a good fit. Keep business and promotion apart; Google’s guidelines say not to put promotions in receipts.</li>\n<li><strong>One announcement to people who opted in:</strong> Gmail’s mail merge, which adds the unsubscribe link for you.</li>\n<li><strong>A newsletter, a sales list, or thousands a day from your domain:</strong> a dedicated email service. At that volume Google requires one-click unsubscribe headers on marketing mail, and its guidelines recommend dropping addresses that bounce again and again. You also need a list of who left. Gmail’s merge can’t give you that list, and the script tracks none of it. Check a service’s documentation for those before you move the list, and check that it signs your mail with SPF and DKIM.</li>\n</ul>\n<h2 id=\"check-the-sent-folder-after-the-first-real-run\">Check the Sent folder after the first real run</h2>\n<p>Whatever you pick, the first real run is the one to watch. Open Gmail’s Sent folder and compare it with the rows marked <code>Sent</code>. The counts should match, and each PDF should carry the right name.</p>\n<p>If you also want to fill this kind of sheet from incoming email, our <a href=\"https://28creation.com/articles/gmail-to-google-sheets/\">Gmail to Google Sheets guide</a> has a tested script for that, and <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">our checks for scheduled scripts</a> help a silent failure speak up. More guides are under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Start with a copy of your real sheet holding just two rows, both with your own address. With <code>skipDuplicates</code> on, one row should come back <code>Sent</code> and the other <code>Skipped: same address as row 2</code>.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read Google’s help, quota and sender pages on October 20, 2026. The script ran unchanged in Node.js 22 against stand-ins we wrote for Google’s services, on 5 invented rows and 10 cases. It has not run in Apps Script or sent a real email.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Google, <a href=\"https://support.google.com/mail/answer/12921167?hl=en\">Send personalized emails with mail merge</a> (Gmail Help: plans, steps, limits, unsubscribe, restrictions)</li>\n<li>Google Workspace Updates, <a href=\"https://workspaceupdates.googleblog.com/2023/06/google-sheets-now-integrated-with-gmail.html\">Google Sheets now integrated with Gmail mail merge</a> (June 2023; admin defaults)</li>\n<li>Google, <a href=\"https://support.google.com/docs/answer/13584275?hl=en\">Insert and use variable chips</a> (Docs Help)</li>\n<li>Google, <a href=\"https://developers.google.com/docs/api/samples/mail-merge?hl=en\">Mail merge with Docs API</a> and <a href=\"https://developers.google.com/apps-script/samples/automations/mail-merge?hl=en\">Create a mail merge with Gmail and Google Sheets</a> (developer samples)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">Quotas for Google Services</a> (Apps Script; updated September 3, 2026)</li>\n<li>Google, <a href=\"https://support.google.com/mail/answer/22839?hl=en\">Limits for sending and getting mail</a> (personal Gmail) and <a href=\"https://support.google.com/a/answer/166852?hl=en\">Gmail sending limits in Google Workspace</a></li>\n<li>Google, <a href=\"https://support.google.com/a/answer/81126?hl=en\">Email sender guidelines</a> and <a href=\"https://support.google.com/a/answer/14229414?hl=en\">Email sender guidelines FAQ</a></li>\n<li>Google, Apps Script reference: <a href=\"https://developers.google.com/apps-script/reference/document/body\">Body</a>, <a href=\"https://developers.google.com/apps-script/reference/document/document\">Document</a>, <a href=\"https://developers.google.com/apps-script/reference/drive/file\">File</a>, <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-app\">GmailApp</a>, <a href=\"https://developers.google.com/apps-script/reference/mail/mail-app\">MailApp</a>, <a href=\"https://developers.google.com/apps-script/reference/lock/lock-service\">LockService</a>, <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/range\">Range</a>, <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/spreadsheet-app\">SpreadsheetApp</a>, <a href=\"https://developers.google.com/apps-script/guides/bound\">Container-bound scripts</a></li>\n<li>28creation test, October 20, 2026: 5 invented rows and 10 cases, our script run unchanged in Node.js 22 against stand-ins for Google’s services, not in Apps Script or a live Google account</li>\n</ul>\n",
      "summary": "Gmail's mail merge works on seven Workspace plans and sends one file to all. Our script makes a PDF per row; we tested it on stand-ins, not Google.",
      "image": "https://28creation.com/og/mail-merge-google-sheets.jpg",
      "date_published": "2026-10-22T07:00:00.000Z",
      "date_modified": "2026-10-22T07:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/gitlab-cvss-10-fix-back-to-version-7/",
      "url": "https://28creation.com/articles/gitlab-cvss-10-fix-back-to-version-7/",
      "title": "GitLab CVSS 10 flaw hits every release since 7.0; patch again",
      "content_html": "<h2>The short read</h2><p><strong>GitLab rated a new regex flaw 10.0: anyone, without logging in, could run code on a self-managed server.</strong></p><ul><li>GitLab released 19.5.1, 19.4.3, 19.3.6, 19.2.9, 19.1.9, 19.0.10 and 18.11.13 on October 21; the flaw reaches back to version 7.0.</li></ul><h3>What you can do with this</h3><blockquote><p>Check your version under Help &gt; Help and upgrade to the fixed release for your line today. If you patched on October 12, upgrade again: that release does not cover this flaw.</p></blockquote><hr><p>GitLab released <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-5-1-released/\">seven patch versions</a> on October 21 for CVE-2026-110264, a flaw rated 10.0, the top CVSS score. Under conditions GitLab has not described, an unauthenticated user could run code on a self-managed GitLab server through an integer overflow when it compiles a crafted regular expression. It affects every release from 7.0 on. If your company hosts its own GitLab, this is today’s upgrade.</p>\n<p><a href=\"https://thehackernews.com/2026/10/critical-gitlab-flaw-cvss-100-could.html\">The Hacker News</a> reported it the same day. GitLab.com is already patched, GitLab says, and Dedicated tenants on 19.3.5-internal0 or 19.4.2-internal0 and later already have the fix. Self-managed servers are the ones at risk.</p>\n<h2 id=\"the-fourth-regex-flaw-in-a-month\">The fourth regex flaw in a month</h2>\n<p>This did not come out of nowhere. GitLab’s own advisories show a run of bugs in how the server parses and compiles regular expressions:</p>\n<ul>\n<li><strong>September 23:</strong> <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-1-released/\">two flaws rated 9.9</a>, CVE-2026-89078 and CVE-2026-93577, both needing a logged-in user and a crafted regex in CI/CD configuration.</li>\n<li><strong>October 12:</strong> <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-2-released/\">CVE-2026-97229</a>, rated 10.0 and needing no login, fixed in 19.4.2, 19.3.5 and 19.2.8.</li>\n<li><strong>October 21:</strong> CVE-2026-110264, also 10.0 with no login, and the first that reaches back past 19.2.</li>\n</ul>\n<p>That last point is the trap. The earlier three only affected 19.2 and newer, so a server on an older line had nothing to patch. This one hits everything back to 7.0, which The Hacker News notes came out in 2014. And a server patched on October 12 is still exposed: 19.4.2 is older than 19.4.3.</p>\n<p>The same researcher, joaxcar, is credited on the HackerOne report, and The Hacker News counts four such flaws from that person in a month. The October 21 release also ships a “Ruby regexp repeat ID overflow patch,” per GitLab’s list of changes, which fits The Hacker News’s account that the root cause sits in Onigmo, the regex engine Ruby uses.</p>\n<h2 id=\"why-gitlab-patched-seven-lines-not-three\">Why GitLab patched seven lines, not three</h2>\n<p>GitLab’s <a href=\"https://docs.gitlab.com/policy/maintenance/\">maintenance policy</a> backports security fixes to the current release and the previous two, which would mean 19.5, 19.4 and 19.3. This time it also shipped 19.2.9, 19.1.9, 19.0.10 and 18.11.13. The policy allows that only “in rare cases.” We read the extra four lines as GitLab’s own signal of how serious it thinks this is.</p>\n<p>GitLab says nothing about attacks so far. When we checked CISA’s <a href=\"https://www.cisa.gov/known-exploited-vulnerabilities-catalog\">known exploited vulnerabilities catalog</a> at 05:38 UTC on October 22, this CVE was not on it. The same catalog shows why old servers matter, though: in February 2026 CISA added two GitLab flaws from 2021, CVE-2021-39935 and CVE-2021-22175, which means attackers were still using five-year-old bugs.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Find your version.</strong> Sign in, then open Help &gt; Help; the <a href=\"https://docs.gitlab.com/user/version/\">version shows at the top</a>.</li>\n<li><strong>Upgrade to the fix for your line:</strong> 19.5.1, 19.4.3, 19.3.6, 19.2.9, 19.1.9, 19.0.10 or 18.11.13. GitLab gives no workaround.</li>\n<li><strong>Plan for downtime on a single server.</strong> GitLab says the patch “will cause downtime during the upgrade as migrations must complete before GitLab can start.” Multi-node setups can use the zero-downtime procedure.</li>\n<li><strong>On 18.10 or older, budget more than one step.</strong> You cannot skip <a href=\"https://docs.gitlab.com/update/upgrade_paths/\">required upgrade stops</a> such as 18.2, 18.5, 18.8 and 18.11, and each needs background migrations to finish. GitLab’s <a href=\"https://gitlab-com.gitlab.io/support/toolbox/upgrade-path/\">upgrade path tool</a> lists the stops from your version. If that takes days, consider taking the server off the public internet until you are done.</li>\n<li><strong>Turn on the detections.</strong> GitLab published two rules, “Onigmo native worker crash” and “Org mode render error burst,” in its open <a href=\"https://gitlab.com/gitlab-security-oss/tldr\">TLDR detection repository</a>. If you use SAML with LDAP auto-linking, the same release fixes an account takeover rated 8.2, CVE-2026-15723.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>GitLab, Critical Patch Release 19.5.1, 19.4.3, 19.3.6, 19.2.9, 19.1.9, 19.0.10, 18.11.13, October 21, 2026: <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-5-1-released/\">https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-5-1-released/</a></li>\n<li>GitLab, Critical Patch Release 19.4.2, 19.3.5, 19.2.8, October 12, 2026: <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-2-released/\">https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-2-released/</a></li>\n<li>GitLab, Critical Patch Release 19.4.1, 19.3.3, 19.2.7, September 23, 2026: <a href=\"https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-1-released/\">https://docs.gitlab.com/releases/patches/patch-release-gitlab-19-4-1-released/</a></li>\n<li>NVD, CVE-2026-110264: <a href=\"https://nvd.nist.gov/vuln/detail/CVE-2026-110264\">https://nvd.nist.gov/vuln/detail/CVE-2026-110264</a></li>\n<li>The Hacker News, “Critical GitLab Flaw (CVSS 10.0) Could Allow Unauthenticated RCE Across 12 Years of Releases,” October 21, 2026: <a href=\"https://thehackernews.com/2026/10/critical-gitlab-flaw-cvss-100-could.html\">https://thehackernews.com/2026/10/critical-gitlab-flaw-cvss-100-could.html</a></li>\n<li>GitLab, maintenance policy: <a href=\"https://docs.gitlab.com/policy/maintenance/\">https://docs.gitlab.com/policy/maintenance/</a></li>\n<li>GitLab, upgrade paths: <a href=\"https://docs.gitlab.com/update/upgrade_paths/\">https://docs.gitlab.com/update/upgrade_paths/</a></li>\n<li>GitLab, find your GitLab version: <a href=\"https://docs.gitlab.com/user/version/\">https://docs.gitlab.com/user/version/</a></li>\n<li>CISA, Known Exploited Vulnerabilities Catalog, read October 22, 2026: <a href=\"https://www.cisa.gov/known-exploited-vulnerabilities-catalog\">https://www.cisa.gov/known-exploited-vulnerabilities-catalog</a></li>\n</ul>\n",
      "summary": "A login-free code execution flaw hits self-managed GitLab servers on every release since 7.0. Older 19.x and 18.11 lines got fixes too.",
      "image": "https://28creation.com/og/gitlab-cvss-10-fix-back-to-version-7.jpg",
      "date_published": "2026-10-22T05:41:23.000Z",
      "date_modified": "2026-10-22T05:41:23.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/apps-script-vs-power-automate/",
      "url": "https://28creation.com/articles/apps-script-vs-power-automate/",
      "title": "Google Apps Script vs Power Automate: limits and prices compared",
      "content_html": "<h2>The short read</h2><p><strong>A daily digest would use 8 of Apps Script&apos;s 1,500 email recipients and 5 of Power Automate&apos;s 6,000 actions.</strong></p><ul><li>Both come free with the suite: Apps Script with any Gmail account, Power Automate&apos;s standard connectors with Microsoft 365 plans.</li><li>Power Automate&apos;s Google Sheets connector has no trigger, and Apps Script reaches Microsoft only through Graph code and an app registration.</li></ul><h3>What you can do with this</h3><blockquote><p>Write down what starts the job before you pick. If it starts in Google, use Apps Script; if in Outlook, use Power Automate. Either way, name a second owner.</p></blockquote><hr><p>An office that pays for both Google Workspace and Microsoft 365 already owns two automation tools at no extra cost. Google says Apps Script needs <a href=\"https://workspace.google.com/products/apps-script/?hl=en\">nothing but a Gmail account</a>, and Microsoft says a flow without premium connectors runs on <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">a Microsoft 365 license</a>. Each can reach the other suite. Their limits differ, and the limits decide the fit.</p>\n<p>We read both vendors’ quota and licensing pages on October 20, 2026, along with the connector references, and set the numbers side by side. Then we counted what one ordinary job, a daily email digest of new spreadsheet rows, uses in each tool. The digest costs nothing in either. What separates them is what can start a job and who can fix it when it breaks.</p>\n<h2 id=\"both-cost-nothing-extra-until-a-premium-connector-appears\">Both cost nothing extra until a premium connector appears</h2>\n<p>Apps Script runs on Google’s servers, and Google lists no price for it. Its FAQ asks “Do I need to pay for Google Workspace to use Apps Script?” and answers: “No. All you need is a Gmail account to begin building with Apps Script.” A paid Workspace account buys higher quotas, not access.</p>\n<p>Power Automate is free in a narrower sense. Plans such as Office 365 Business Basic, Business Standard, Business Premium, E3 and E5 include <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">limited use rights</a> that Microsoft describes as meant “for personal productivity scenarios”. Those rights cover cloud flows with <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/deep-dive-on-specific-license\">standard connectors only</a>; premium and custom connectors are “Not included”.</p>\n<p>The step up is Power Automate Premium at <a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">$15.00 per user a month, paid yearly</a>. A Process license costs $150.00 per bot a month (also paid yearly) and gives one flow its own capacity. For a scheduled or automated flow that uses a premium connector, Microsoft says “only the owner needs to have a Premium license”, so one $15 license can cover a flow the whole office relies on.</p>\n<h2 id=\"the-limits-side-by-side-read-october-20-2026\">The limits side by side, read October 20, 2026</h2>\n<p>Every cell below comes from a vendor page read that day, listed under Sources: mainly Google’s <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">quotas page</a> (updated September 3, 2026), its <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">trigger guide</a>, and Microsoft’s <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">limits page</a>, its <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">license types</a> page and the licensing FAQ.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Apps Script, Gmail account</th>\n<th>Apps Script, Workspace account</th>\n<th>Power Automate, Microsoft 365 rights</th>\n<th>Power Automate Premium</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Cost on top of the suite</td>\n<td>$0</td>\n<td>$0</td>\n<td>$0</td>\n<td>$15 per user a month, paid yearly</td>\n</tr>\n<tr>\n<td>Longest single run</td>\n<td>6 minutes</td>\n<td>6 minutes</td>\n<td>30 days</td>\n<td>30 days</td>\n</tr>\n<tr>\n<td>Daily budget</td>\n<td>90 minutes of trigger runtime</td>\n<td>6 hours of trigger runtime</td>\n<td>6,000 actions per user</td>\n<td>40,000 actions per user</td>\n</tr>\n<tr>\n<td>Email from the tool</td>\n<td>100 recipients a day</td>\n<td>1,500 recipients a day</td>\n<td>Each send counts as 1 action</td>\n<td>Each send counts as 1 action</td>\n</tr>\n<tr>\n<td>Calls to other web services</td>\n<td>20,000 URL Fetch calls a day</td>\n<td>100,000 URL Fetch calls a day</td>\n<td>Standard connectors only</td>\n<td>Premium and custom connectors too</td>\n</tr>\n<tr>\n<td>Most frequent schedule</td>\n<td>Every minute</td>\n<td>Every minute</td>\n<td>Every 60 seconds</td>\n<td>Every 60 seconds</td>\n</tr>\n<tr>\n<td>How many per person</td>\n<td>20 triggers per script</td>\n<td>20 triggers per script</td>\n<td>600 flows owned</td>\n<td>600 flows owned</td>\n</tr>\n<tr>\n<td>Idle flows</td>\n<td>No rule listed</td>\n<td>No rule listed</td>\n<td>May be turned off after 90 days without a trigger</td>\n<td>Exempt</td>\n</tr>\n<tr>\n<td>At the limit</td>\n<td>Script throws an error and stops</td>\n<td>Script throws an error and stops</td>\n<td>Runs slow down</td>\n<td>Runs slow down</td>\n</tr>\n</tbody>\n</table>\n<p>Google’s quotas are per user and “reset 24 hours after the first request”. Google also says they can change “at any time, without notice”.</p>\n<p>Microsoft’s action numbers need one footnote. Its limits page lists higher figures for now, 10,000 a day for Microsoft 365 users and 200,000 for Premium. It says they “aren’t guarantees” and sit above the documented limits during a “licensing transition period”. The FAQ adds that during that period per-user limits “aren’t the ones being applied yet”, and each flow gets one shared limit instead.</p>\n<p>We would plan on 6,000 and 40,000, the figures Microsoft lists as the license limits, and treat anything above them as temporary.</p>\n<h2 id=\"power-automate-reaches-google-through-two-standard-connectors\">Power Automate reaches Google through two standard connectors</h2>\n<p>Microsoft’s <a href=\"https://learn.microsoft.com/en-us/connectors/gmail/\">Gmail</a> and <a href=\"https://learn.microsoft.com/en-us/connectors/googlesheet/\">Google Sheets</a> connectors are both classed Standard. So a Microsoft 365 license covers them, with no Premium needed.</p>\n<p><strong>The Google Sheets connector has no trigger.</strong> Its reference lists six actions (Get rows, Get row, Insert row, Update row, Delete Row, Get sheets) and nothing that fires when a row is added. So a new row can’t start a flow; the flow has to check the sheet on a schedule. Get rows takes no filter, starts at the first entry by default, and returns 256 entries unless you raise Top Count. Spreadsheets in shared drives “cannot be selected”, so you paste the file’s ID by hand. Each connection gets 100 calls per 5 minutes.</p>\n<p><strong>The Gmail connector treats consumer accounts differently.</strong> An @gmail.com account “can only connect to a limited set of services”. Google Sheets, Office 365, OneDrive, SharePoint and Teams are on the approved list, and a flow that uses anything else “will end up being saved in a disabled state”. Workspace accounts are not affected.</p>\n<p>The Gmail connector also meters work. Each connection gets 90,000 work units a day, and Send email (V2) costs 100, which works out to at most 900 sent emails a day per connection. The new-email trigger “may skip emails” above 300 in 30 seconds, and on Workspace accounts it “may intermittently skip emails with attachments”. We would not build an invoice inbox on that trigger.</p>\n<h2 id=\"apps-script-reaches-microsoft-only-through-code\">Apps Script reaches Microsoft only through code</h2>\n<p>Apps Script’s <a href=\"https://developers.google.com/apps-script/overview\">built-in libraries</a> cover Google’s own apps, such as Gmail and Drive. To act inside Microsoft 365, a script calls <a href=\"https://learn.microsoft.com/en-us/graph/api/user-sendmail\">Microsoft Graph</a> with <a href=\"https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app\">UrlFetchApp</a>. Google’s guide says Apps Script “doesn’t provide built-in support” for OAuth, the token-based sign-in that Graph requires, and <a href=\"https://developers.google.com/apps-script/guides/services/external\">points to open-source libraries</a>.</p>\n<p>The setup takes four pieces:</p>\n<ol>\n<li>An app registration in Microsoft Entra, which gives the script its <a href=\"https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-auth-code-flow\">client ID</a>.</li>\n<li>The <a href=\"https://github.com/googleworkspace/apps-script-oauth2\">OAuth2 for Apps Script</a> library, with its callback address <code>https://script.google.com/macros/d/{SCRIPT ID}/usercallback</code> registered in Entra.</li>\n<li>The <code>Mail.Send</code> permission and the <code>offline_access</code> scope, without which no refresh token comes back.</li>\n<li>A call to <code>POST /me/sendMail</code> for each email.</li>\n</ol>\n<p>Every part of that has an edge. The library’s samples folder covers dozens of services, Xero and Zoom among them, but has no Microsoft file. Microsoft’s own protocol page says hand-built requests are something “we do not recommend” and points to its own libraries. Refresh tokens have no fixed lifetime but can “expire, are revoked, or lack sufficient privileges”, so the script needs a way to ask someone to sign in again.</p>\n<p>Consent may not be the blocker. By default, Microsoft says, “a user can consent to allow an app to access their mailbox”. An admin can <a href=\"https://learn.microsoft.com/en-us/entra/identity/enterprise-apps/configure-user-consent\">turn that off</a>, and then the script waits until an admin approves the app.</p>\n<p>Often a script needs none of this. If the goal is only an email in an Outlook inbox, <code>MailApp</code> sends ordinary email, and an Outlook address is just another recipient. Graph earns its setup when the message must come from the Microsoft 365 mailbox, which Graph saves “in the Sent Items folder”.</p>\n<h2 id=\"one-task-counted-both-ways-a-daily-digest-of-new-rows\">One task counted both ways: a daily digest of new rows</h2>\n<p>A hypothetical eight-person office logs every customer inquiry as a row in a Google Sheet, with a date column. Every weekday morning it wants one email to all eight Outlook inboxes listing yesterday’s rows.</p>\n<p><strong>In Apps Script</strong>, a script bound to the sheet gets one time-driven trigger at 8 AM. Going by Google’s own example of a 9 AM trigger, it fires at a time Google picks between 8 and 9, then keeps that time. Each run reads the rows, keeps yesterday’s and sends one <code>MailApp</code> email with 8 recipients.</p>\n<table>\n<thead>\n<tr>\n<th>Resource</th>\n<th>Used per run</th>\n<th>Limit (Workspace / Gmail account)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Triggers</td>\n<td>1 in total</td>\n<td>20 per user per script</td>\n</tr>\n<tr>\n<td>Email recipients</td>\n<td>8</td>\n<td>1,500 / 100 a day</td>\n</tr>\n<tr>\n<td>Recipients on one message</td>\n<td>8</td>\n<td>50</td>\n</tr>\n<tr>\n<td>URL Fetch calls</td>\n<td>0 (about 2 if sent through Graph)</td>\n<td>100,000 / 20,000 a day</td>\n</tr>\n<tr>\n<td>Runtime</td>\n<td>One read and one send; we did not time it</td>\n<td>6 minutes a run, 6 hours / 90 minutes a day</td>\n</tr>\n<tr>\n<td>Extra license</td>\n<td>None</td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<p><strong>In Power Automate</strong>, one Microsoft 365 user owns a scheduled cloud flow with five steps: Recurrence (weekdays at 8:00), Google Sheets Get rows, Filter array (date equals yesterday), Create HTML table, and Office 365 Outlook <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Send an email (V2)</a> to the 8 addresses. Microsoft counts “every trigger and action execution” as one action.</p>\n<table>\n<thead>\n<tr>\n<th>Resource</th>\n<th>Used per run</th>\n<th>Limit (Microsoft 365 rights)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Actions</td>\n<td>5 (1 trigger and 4 actions)</td>\n<td>6,000 a day per user</td>\n</tr>\n<tr>\n<td>Share of the daily limit</td>\n<td>0.08%</td>\n<td></td>\n</tr>\n<tr>\n<td>Actions in a month of 22 weekdays</td>\n<td>110</td>\n<td></td>\n</tr>\n<tr>\n<td>Google Sheets calls</td>\n<td>1</td>\n<td>100 per 5 minutes per connection</td>\n</tr>\n<tr>\n<td>Connectors</td>\n<td>Google Sheets, Office 365 Outlook: both Standard</td>\n<td>Standard only</td>\n</tr>\n<tr>\n<td>Extra license</td>\n<td>None</td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<p>Neither tool comes near a limit: 8 of 1,500 recipients (or 100 on a Gmail account), 5 of 6,000 actions.</p>\n<p>The flow’s one real trap is Get rows. It starts at the first entry and stops at 256 by default, so once new rows at the bottom pass row 256, they fall outside the read. The digest then goes out quietly short until someone raises Top Count. Build the flow with an Apply to each loop over the rows instead of Filter array, and Microsoft counts each action inside the loop once per row.</p>\n<h2 id=\"the-downsides-stated-plainly\">The downsides, stated plainly</h2>\n<p><strong>Apps Script is code, and it belongs to one person.</strong> Someone has to read JavaScript to change the digest. Google says installable triggers “always run under the account of the person who created them”, and that one account “can’t see triggers installed from a second account”. When it fails, Apps Script emails a failure summary instead of showing an error on screen. A run also stops at 6 minutes, so a long job has to be split into batches.</p>\n<p><strong>Power Automate charges for reach and slows under load.</strong> Premium connectors are off limits on Microsoft 365 rights, and for a custom connector Microsoft says you “must have a premium or trial license”. When a flow runs too many actions, runs are delayed, and a flow that is “consistently throttled” for 14 days is turned off. On Microsoft 365 rights a failed action gets “up to two retries”, against 12 on Premium. The Outlook connector warns that a retried send can produce duplicate emails.</p>\n<p><strong>Both tie the job to an owner.</strong> If a flow’s owner leaves, the flow “reverts to the Low performance profile”. For a flow outside a solution, the original creator’s limits “can’t be changed”; if the creator leaves, co-owners can export it and import it under a new owner. A flow that fires quarterly also risks the 90-day idle rule on Microsoft 365 rights. A daily digest does not.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The quotas will rarely decide this. Two questions will: where does the event that starts the job happen, and who on staff will fix it.</p>\n<ul>\n<li><strong>The job starts in Google</strong> (a form response, a sheet edit): use Apps Script. It has an <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">installable form submit trigger</a>, and Power Automate’s Sheets connector has no trigger at all.</li>\n<li><strong>The job starts in Outlook, Teams or SharePoint</strong> and writes to a Google Sheet: use Power Automate with the standard Sheets connector, at $0 on Microsoft 365.</li>\n<li><strong>The job runs on a schedule and nobody writes JavaScript:</strong> use Power Automate, and add a second person as co-owner on day one.</li>\n<li><strong>Someone does write JavaScript:</strong> Apps Script skips Microsoft’s action counting and the idle rule. Send plain mail with <code>MailApp</code> unless the message must sit in an Outlook Sent Items folder.</li>\n<li><strong>Before buying Premium,</strong> check each connector’s class on its reference page. For a scheduled flow, only the owner needs the $15 license, which is $180 a year.</li>\n</ul>\n<p>If the job needs apps neither suite connects well, our <a href=\"https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/\">Zapier, Make and n8n price comparison</a> prices a third route. For Apps Script quotas in practice, see <a href=\"https://28creation.com/articles/gmail-to-google-sheets/\">Gmail to Google Sheets</a>, and for a flow on the free Microsoft license, <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments</a>. Our <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">checks for scheduled scripts</a> make a silent failure speak up. More is under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Write your job as one line before you pick: what starts it, which suite it reads and which it writes. Give it to the tool that owns the starting event, and name a second person who can open it.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read Google’s and Microsoft’s quota, licensing and connector pages on October 20, 2026, and built both tables from them. The digest is a hypothetical task: we counted its quota use from the documentation and did not build or run either version, because we have no Workspace or Microsoft 365 tenant for testing.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">Quotas for Google Services</a> (updated September 3, 2026; read October 20, 2026)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/overview\">Apps Script overview</a>, <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">Installable triggers</a>, <a href=\"https://developers.google.com/apps-script/guides/services/external\">External APIs</a> and <a href=\"https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app\">UrlFetchApp reference</a></li>\n<li>Google Workspace, <a href=\"https://workspace.google.com/products/apps-script/?hl=en\">Apps Script product page and FAQ</a></li>\n<li>Google Workspace on GitHub, <a href=\"https://github.com/googleworkspace/apps-script-oauth2\">OAuth2 for Apps Script</a> (samples folder listed October 20, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Limits of automated, scheduled, and instant flows</a> (updated July 17, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Types of Power Automate licenses</a> and <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/faqs\">Power Automate licensing FAQ</a> (both updated September 22, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/deep-dive-on-specific-license\">Seeded and specific licenses</a> (updated October 17, 2026) and <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/api-request-limits-allocations\">Requests limits and allocations</a></li>\n<li>Microsoft, <a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">Power Automate pricing</a> (read October 20, 2026)</li>\n<li>Microsoft Learn, connector references: <a href=\"https://learn.microsoft.com/en-us/connectors/gmail/\">Gmail</a>, <a href=\"https://learn.microsoft.com/en-us/connectors/googlesheet/\">Google Sheets</a>, <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/graph/api/user-sendmail\">user: sendMail</a>, <a href=\"https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-auth-code-flow\">OAuth 2.0 authorization code flow</a> and <a href=\"https://learn.microsoft.com/en-us/entra/identity/enterprise-apps/configure-user-consent\">Configure user consent</a></li>\n</ul>\n",
      "summary": "Both are free with the suite. Apps Script needs code and stops each run at 6 minutes; Microsoft 365 flows get 6,000 actions a day and standard connectors only.",
      "image": "https://28creation.com/og/apps-script-vs-power-automate.jpg",
      "date_published": "2026-10-21T15:00:00.000Z",
      "date_modified": "2026-10-21T15:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/php-fixes-released-cve-2026-109618/",
      "url": "https://28creation.com/articles/php-fixes-released-cve-2026-109618/",
      "title": "PHP fixes are out: 8.5.12, 8.4.27, 8.3.36 and 8.2.35",
      "content_html": "<h2>The short read</h2><p><strong>PHP says there are no complete mitigations for its two critical flaws: updating is the only safe choice.</strong></p><ul><li>PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35 fix two flaws that can allow remote code execution without a login.</li></ul><h3>What you can do with this</h3><blockquote><p>Update PHP today, or ask your host to. On Ubuntu, check the package version with dpkg, not php -v, because Ubuntu backports the fix into older version numbers.</p></blockquote><hr><p>PHP released <a href=\"https://www.php.net/archive/2026.php#2026-10-21-5\">8.5.12, 8.4.27, 8.3.36 and 8.2.35</a> today, fixing two critical flaws that “can allow unauthenticated remote code execution.” PHP’s advisory says every installation serving web requests is at risk, and that “there are no complete mitigations, updating is the only safe choice.” If your company runs a PHP site, the job for today is the update, not a workaround.</p>\n<p>This follows our <a href=\"https://28creation.com/articles/php-critical-security-release-october-2026/\">report this morning</a> on the pre-announcement. Two things are new: the fixed versions, and PHP’s own word on what the stopgaps can and cannot do.</p>\n<h2 id=\"what-the-two-bugs-do\">What the two bugs do</h2>\n<p>Both are rated CVSS v4.0 9.2. The first, CVE-2026-109618, starts when PHP runs out of memory: per the <a href=\"https://www.php.net/archive/2026.php#2026-10-21-5\">advisory</a>, handling that failure can leave request data “only partially initialized,” and cleanup then treats it as valid. The second, CVE-2026-109619, is state that outlives the request it belongs to; the <a href=\"https://github.com/php/php-src/security/advisories/GHSA-6wmf-4rv5-p3rp\">GitHub advisory</a> calls it a use-after-free that a later request in the same worker can reach.</p>\n<p>That is why fixing the site alone is not enough. PHP lists PHP-FPM, Apache with mod_php, php-cgi, FrankenPHP and LiteSpeed as affected, and rates mod_php highest, “with a demonstrated exploit where an unauthenticated request can trigger the attack before any application code runs.” <a href=\"https://blog.cloudflare.com/php-vulnerabilities-waf-protection/\">Cloudflare</a> adds that updating site code or plugins alone does not fix it.</p>\n<p>Calif.io, Wiz, OpenAI and Quarkslab disclosed the flaws, according to Cloudflare. PHP’s advisory does not say the flaws are being exploited.</p>\n<h2 id=\"your-version-number-may-not-change\">Your version number may not change</h2>\n<p>Linux distributions got the patches in advance, PHP says. That creates a trap. <a href=\"https://ubuntu.com/security/CVE-2026-109618\">Ubuntu’s tracker</a> lists the fix in <code>php8.3</code> 8.3.6-0ubuntu0.24.04.13 on 24.04 LTS, <code>php8.1</code> 8.1.2-1ubuntu2.28 on 22.04 LTS and <code>php8.5</code> 8.5.4-0ubuntu1.5 on 26.04 LTS. A patched 24.04 server still reports 8.3.6, not 8.3.36.</p>\n<p>For the PHP 7 packages on Ubuntu 20.04 and 18.04, the status read “Needs evaluation” when we checked at 14:40 UTC.</p>\n<p>For PHP 8.1 and older, the PHP project ships no release. Its security team did publish minimal source backports for <a href=\"https://www.php.net/archive/2026.php#2026-10-21-5\">7.4, 8.0 and 8.1</a> for people who build PHP themselves.</p>\n<h2 id=\"the-stopgaps-and-a-correction-to-ours\">The stopgaps, and a correction to ours</h2>\n<p>Turning off <code>session.upload_progress.enabled</code> blocks “the confirmed session paths but not other variants,” PHP says. It must be set in php.ini, an FPM pool or server configuration; PHP says not to rely on <code>ini_set()</code> or FPM <code>.user.ini</code>. Our earlier piece listed .user.ini as an option. For PHP-FPM, that is wrong.</p>\n<p>For the memory bug, PHP suggests <code>memory_limit=-1</code> and a low <code>post_max_size</code> (8M without uploads, 32M with them). Both have a cost. In our reading, no memory limit lets one request take far more memory, and PHP also suggests a container or cgroup limit as the boundary. A low <code>post_max_size</code> rejects larger uploads.</p>\n<p><a href=\"https://blog.cloudflare.com/php-vulnerabilities-waf-protection/\">Cloudflare</a> has blocked requests aimed at these flaws since 03:02 UTC today on every plan with its firewall turned on, Free included. It says this is an extra layer, not a fix.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Update.</strong> Install 8.5.12, 8.4.27, 8.3.36 or 8.2.35, or your distribution’s patched package, then restart PHP-FPM or Apache so old workers stop.</li>\n<li><strong>Check the package, not the banner.</strong> On Ubuntu, run <code>dpkg -l | grep php</code> and compare against Ubuntu’s tracker. On other distributions, check your vendor’s advisory.</li>\n<li><strong>Ask your host in writing</strong> which PHP version each site runs and when the patch went on. Cloudflare says a host that manages PHP must apply it.</li>\n<li><strong>On 8.1 or older,</strong> get a vendor backport this week and plan the move to 8.3 or newer. PHP 8.2 loses support on December 31, 2026.</li>\n<li><strong>On Cloudflare,</strong> paid plans should confirm the Managed Ruleset is set to block, Cloudflare says.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>PHP, security advisory and release notes for 8.5.12, 8.4.27, 8.3.36 and 8.2.35, October 21, 2026: <a href=\"https://www.php.net/archive/2026.php#2026-10-21-5\">https://www.php.net/archive/2026.php#2026-10-21-5</a></li>\n<li>PHP, GitHub advisory GHSA-6wmf-4rv5-p3rp (CVE-2026-109619): <a href=\"https://github.com/php/php-src/security/advisories/GHSA-6wmf-4rv5-p3rp\">https://github.com/php/php-src/security/advisories/GHSA-6wmf-4rv5-p3rp</a></li>\n<li>PHP, GitHub advisory GHSA-w8hw-w9fx-j79w (CVE-2026-109618): <a href=\"https://github.com/php/php-src/security/advisories/GHSA-w8hw-w9fx-j79w\">https://github.com/php/php-src/security/advisories/GHSA-w8hw-w9fx-j79w</a></li>\n<li>Cloudflare, “Cloudflare deploys protections against php2shell vulnerability,” October 21, 2026: <a href=\"https://blog.cloudflare.com/php-vulnerabilities-waf-protection/\">https://blog.cloudflare.com/php-vulnerabilities-waf-protection/</a></li>\n<li>Ubuntu, CVE-2026-109618, read October 21, 2026, 14:40 UTC: <a href=\"https://ubuntu.com/security/CVE-2026-109618\">https://ubuntu.com/security/CVE-2026-109618</a></li>\n<li>Ubuntu, CVE-2026-109619: <a href=\"https://ubuntu.com/security/CVE-2026-109619\">https://ubuntu.com/security/CVE-2026-109619</a></li>\n</ul>\n",
      "summary": "PHP's two critical flaws hit every PHP web server, and only the update closes them. Ubuntu packages carry the fix under different version numbers.",
      "image": "https://28creation.com/og/php-fixes-released-cve-2026-109618.jpg",
      "date_published": "2026-10-21T14:41:27.000Z",
      "date_modified": "2026-10-21T14:41:27.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/employee-monitoring-gdpr-automation/",
      "url": "https://28creation.com/articles/employee-monitoring-gdpr-automation/",
      "title": "Employee monitoring and GDPR: why consent fails and what works",
      "content_html": "<h2>The short read</h2><p><strong>France&apos;s Conseil d&apos;État cut Amazon&apos;s monitoring fine to 15 million euros; keeping every indicator 31 days still broke the rules.</strong></p><ul><li>Regulators in the EU, UK and Sweden say employee consent rarely works, so recording work usually rests on legitimate interest and an impact assessment.</li><li>In 2024 the CNIL fined a small firm 40,000 euros for idle-time tracking and screenshots every 3 to 15 minutes.</li></ul><h3>What you can do with this</h3><blockquote><p>Before recording anyone, write a legitimate interest assessment and a DPIA, tell staff in writing, and collect group-level data without keystrokes or screenshots. The fined French firm missed the DPIA and the written notice.</p></blockquote><hr><p>In December 2024, France’s data protection authority, the CNIL, fined a small real estate company <a href=\"https://www.cnil.fr/fr/surveillance-excessive-des-salaries-sanction-de-40-000-euros-entreprise-secteur-immobilier\">40,000 euros</a> for watching its staff too closely. Its software counted “idle” minutes and took screenshots of staff computers every 3 to 15 minutes. Any small or mid-sized company that records how people work, even just to find tasks worth automating, answers to the same GDPR rules.</p>\n<p>Screen recordings, task mining logs, app usage logs and email metadata are personal data under the <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">GDPR</a> once they relate to an identifiable person. The obvious first step, asking staff to agree, is the one European regulators say rarely holds up.</p>\n<p>So on October 20, 2026, we read the regulators’ own texts: the GDPR, EU guidance, the UK’s ICO, Sweden’s IMY and two French fines. Below are the texts side by side, the design choices they reward and a checklist for a 20-person company. It is a reading of the sources, not legal advice.</p>\n<h2 id=\"consent-fails-because-staff-cannot-freely-say-no\">Consent fails because staff cannot freely say no</h2>\n<p>The GDPR allows consent, but <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">recital 43</a> says it “should not provide a valid legal ground” where “there is a clear imbalance between the data subject and the controller.” Article 7(4) sets out how to judge whether consent was freely given. The EDPB’s point is simple: an employee who fears what a refusal might cost is not free.</p>\n<p>Every text we read applies this to work, in nearly the same words:</p>\n<table>\n<thead>\n<tr>\n<th>Source</th>\n<th>On consent at work</th>\n<th>On monitoring tools</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td><a href=\"https://ec.europa.eu/newsroom/article29/items/610169\">WP29 Opinion 2/2017</a> (EU advisory body)</td>\n<td>Employees “are seldom in a position to freely give, refuse or revoke consent”</td>\n<td>Software logging keystrokes, screens and apps used is “disproportionate”; a legal ground for recording keystrokes is “very unlikely”</td>\n</tr>\n<tr>\n<td><a href=\"https://www.edpb.europa.eu/system/files/documents/files/file1/edpb_guidelines_202005_consent_en.pdf\">EDPB Guidelines 05/2020</a></td>\n<td>“Problematic for employers”; for most processing at work the basis “cannot and should not be the consent of the employees”</td>\n<td>Names workplace camera monitoring as a request staff are unlikely to answer freely</td>\n</tr>\n<tr>\n<td><a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/data-protection-and-monitoring-workers/\">ICO, monitoring workers</a> (UK)</td>\n<td>“Not usually appropriate in the employment context, due to the imbalance of power”</td>\n<td>Gives keystroke monitoring as an example of high-risk processing; guidance under review after the Data (Use and Access) Act</td>\n</tr>\n<tr>\n<td><a href=\"https://www.imy.se/verksamhet/dataskydd/dataskydd-pa-olika-omraden/arbetsliv/kontroll-och-overvakning-av-anstallda/\">IMY</a> (Sweden, updated June 15, 2026)</td>\n<td>Does not think an employer can use consent as the legal basis for monitoring staff (our translation)</td>\n<td>Using an IT system to monitor in real time, or otherwise regularly, how staff work or take breaks is as a rule not allowed (our translation)</td>\n</tr>\n</tbody>\n</table>\n<p>The 2017 opinion came from the Article 29 Working Party, a European advisory body set up under the old directive. The EDPB’s 2020 guidelines, written for the GDPR, agree.</p>\n<p>There is one narrow exception. The EDPB says employees can consent freely only “when it will have no adverse consequences at all whether or not they give consent.” Its example is a film crew in an office: staff who do not want to be filmed “are not penalised in any way” and get equal desks elsewhere. A recording tool that a manager proposes to a team will rarely clear that bar.</p>\n<h2 id=\"legitimate-interest-is-the-usual-route-with-a-three-part-test\">Legitimate interest is the usual route, with a three-part test</h2>\n<p>With consent out, the regulators point to legitimate interest. <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Article 6(1)(f)</a> allows processing “necessary for the purposes of the legitimate interests pursued by the controller,” unless the person’s rights override them. The 2017 opinion names it as an alternative, then warns that “a legitimate interest in itself is not sufficient to override the rights and freedoms of employees.”</p>\n<p>The ICO breaks the test into three questions. Is there a legitimate interest behind the processing? Is the processing necessary for that purpose? Do the person’s interests, rights or freedoms override it?</p>\n<p>We think finding work to automate can pass the first question. The second and third are the hard part. The ICO says: “If you can achieve your aim in a less intrusive way, you must do so.”</p>\n<p>IMY puts a weight on the scale: security reasons count for more than efficiency reasons (our translation). Automation is an efficiency reason.</p>\n<p>Legitimate interest also gives staff rights. Under <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Article 21</a>, a person can object, and the employer must then stop unless it shows “compelling legitimate grounds.” Article 13(1)(d) requires telling people which legitimate interest you rely on.</p>\n<p>IMY accepts the employment contract as a basis when measuring performance is needed to pay agreed performance-based wages. An automation study is not that, so most projects come back to the balancing test.</p>\n<h2 id=\"the-impact-assessment-is-likely-required\">The impact assessment is likely required</h2>\n<p><a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Article 35</a> requires a data protection impact assessment (DPIA), a written risk analysis done before processing starts, when processing “is likely to result in a high risk.” The EU guidelines on DPIAs (<a href=\"https://ec.europa.eu/newsroom/article29/items/611236\">WP248</a>) list nine risk criteria and say that in most cases, meeting two calls for one. Systematic monitoring is a criterion, and employees can count as vulnerable data subjects, which is another.</p>\n<p>The guidelines then give an example that reads like a task mining pilot: “A company systematically monitoring its employees’ activities, including the monitoring of the employees’ work station, internet activity.” Their answer to whether a DPIA is likely required is yes.</p>\n<p>The ICO says a DPIA is a must for high-risk monitoring and that you “should carry out a DPIA even if there is no specific high risk.” If the DPIA finds a high risk you cannot reduce, the ICO says you must consult it before you start. The French company in our opening skipped this step, and the CNIL counted that as its own breach of Article 35.</p>\n<h2 id=\"national-law-adds-its-own-rules\">National law adds its own rules</h2>\n<p><a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Article 88</a> lets EU countries set more specific rules for employee data “by law or by collective agreements,” with particular regard to “monitoring systems at the work place.” So the GDPR is only part of the answer.</p>\n<p>Two examples show how much this varies. In Germany, a works council has a co-determination right under <a href=\"https://www.gesetze-im-internet.de/betrvg/__87.html\">section 87(1) no. 6</a> of the Works Constitution Act. It covers “the introduction and use of technical devices designed to monitor the behavior or performance of employees” (our translation).</p>\n<p>In Sweden, IMY says an employer’s right to monitor staff is governed first by employment law and collective agreements. The co-determination act (<a href=\"https://www.riksdagen.se/sv/dokument-och-lagar/dokument/svensk-forfattningssamling/lag-1976580-om-medbestammande-i-arbetslivet_sfs-1976-580/\">MBL, section 11</a>) makes an employer with a collective agreement negotiate with the union before an important change to its operations, or to working conditions for the union’s members. IMY notes that such a duty can apply to workplace cameras.</p>\n<p>For the UK, the ICO says you “should seek and document the views of your workers or their representatives (such as trade unions).” Check your own country’s employment law before you buy a tool.</p>\n<h2 id=\"two-french-decisions-show-where-the-line-sits\">Two French decisions show where the line sits</h2>\n<p>The real estate company is the clearer case. Its software marked 3 to 15 minutes without a keystroke or mouse movement as idle, and idle time staff could not justify or make up could be docked from pay. The CNIL noted that time away from the keyboard can be real work, such as meetings or calls, so the tool could not count hours reliably. It found no legal basis for that or for the screenshots, which could capture personal emails, chats and passwords.</p>\n<p>The company had also filmed staff continuously with two cameras. Its written notice to staff fell short, and oral briefings left no record, so the CNIL found breaches of the information rules too.</p>\n<p>The Amazon case shows the other side. In December 2023, the CNIL fined Amazon France Logistique 32 million euros, mainly over indicators drawn from the handheld scanners its warehouse staff use. Amazon appealed, and on December 23, 2025, France’s <a href=\"https://www.conseil-etat.fr/fr/arianeweb/CE/decision/2025-12-23/492830\">Conseil d’État</a> cut the fine to 15 million euros.</p>\n<p>Guess which part survived. The court found that three scanner indicators did not breach the legitimate interest rule. One flagged two scans less than 1.25 seconds apart; another recorded idle time only after more than ten minutes in a row. What it kept was the data minimization breach: storing all the indicators for 31 days, for several purposes at once, when Amazon could not show why each needed that long.</p>\n<p>Findings on staff information and security, which Amazon did not contest, also stood. Our reading: a narrow measurement tied to a clear purpose can pass, while keeping everything for 31 days cannot. Amazon runs warehouses, not an office, and the court judged it on those facts. Breaches of Articles 5 and 6 can cost up to 20 million euros or 4% of worldwide turnover, whichever is higher, under <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Article 83(5)</a>.</p>\n<h2 id=\"design-choices-that-lower-the-risk\">Design choices that lower the risk</h2>\n<p>Each choice below is tied to what a regulator wrote.</p>\n<ul>\n<li><strong>Aggregate where you can.</strong> IMY says the employer must first decide whether the purpose needs follow-up on individuals or whether group level is enough. The ICO suggests “aggregated analytics reports” that “identify trends without identifying individual workers.”</li>\n<li><strong>Keep raw data local and show the employer less.</strong> The 2017 opinion says both what is recorded and “the information that is shown to the employer, should be minimized as much as possible.” It describes systems that record data “without presenting it to the employer.” Processing on the worker’s own machine and sharing only a summary follows that logic, in our reading.</li>\n<li><strong>Skip keystroke logging and screenshots.</strong> The ICO calls keystroke monitoring behavioral biometric data where a worker can be identified by typing rhythm. The CNIL found that screenshots could capture passwords.</li>\n<li><strong>Sample instead of recording all day.</strong> The 2017 opinion lists “sampling instead of continuous monitoring” as a limit and says personal files and communications should not be monitored. Its list of disproportionate software also includes logging which apps were used and for how long, so keep even app logs short and narrow.</li>\n<li><strong>Set a short retention period up front.</strong> The ICO says “you should not retain information just in case.” Amazon could not justify keeping every indicator for 31 days.</li>\n<li><strong>Let individuals opt in, as a safeguard.</strong> A volunteer who picks one task to record is close to the EDPB’s film crew example. Given IMY’s position, we would still document legitimate interest as the basis and treat opt-in as extra protection.</li>\n</ul>\n<h2 id=\"what-this-means-for-you-a-checklist-for-a-20-person-company\">What this means for you: a checklist for a 20-person company</h2>\n<p>The decision is how much detail you need. A tally kept by staff, or a log from systems you already run, carries far less risk. Our pieces on <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">noticing what to automate</a> and <a href=\"https://28creation.com/articles/task-mining-vs-process-mining/\">task mining versus process mining</a> compare those options. If you still want to record work, this order follows the sources:</p>\n<ol>\n<li><strong>Write the purpose in one sentence</strong>, such as “find repeated data entry worth automating.” The ICO’s first test asks for a legitimate interest.</li>\n<li><strong>Try the least intrusive option first</strong> and write down why it is not enough (ICO; WP29 2017).</li>\n<li><strong>Do not use consent as the legal basis</strong> (EDPB 05/2020; ICO; IMY). Write a legitimate interest assessment that answers the ICO’s three questions.</li>\n<li><strong>Run a DPIA before any recording starts</strong> (Article 35; WP248’s workstation example) and keep it on file.</li>\n<li><strong>Check national law and involve staff representatives.</strong> In Germany that means the works council, if you have one (BetrVG section 87); in Sweden, a union you have a collective agreement with (MBL section 11). Elsewhere, at least record staff views, as the ICO advises.</li>\n<li><strong>Tell staff in writing before collection,</strong> including the legitimate interest and their right to object (Articles 13 and 21). Oral notice was not enough in the CNIL case.</li>\n<li><strong>Collect the minimum</strong> for a set sample period: no keystrokes or screenshots, and group reports rather than personal scores (Article 5(1)(c); WP29 2017; ICO; IMY).</li>\n<li><strong>Set and keep a deletion date</strong> for raw data once the candidates are found (Article 5(1)(e); the Conseil d’État on 31 days).</li>\n<li><strong>Keep the data out of pay and performance reviews.</strong> IMY says data collected for one purpose must not be used for other incompatible ones, and performance follow-up must be a stated purpose from the start. In the CNIL case, idle time could be docked from wages.</li>\n</ol>\n<p>The French real estate company removed its software during the inspection. The CNIL chose not to name it because of its small size and that quick removal, and still fined it 40,000 euros. The cheaper version of that lesson is to write the DPIA before installing anything.</p>\n<h2 id=\"how-we-made-this\">How we made this</h2>\n<p>We read the GDPR text, the 2017 working party opinion, the EU guidelines on impact assessments and on consent, the ICO’s monitoring guidance, IMY’s page and the two French decisions on October 20, 2026. The Swedish, German and French quotes are our own translations. We tested no monitoring tool, and none of this is legal advice.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Regulation (EU) 2016/679 (GDPR)</a>: recital 43 and Articles 4, 5, 6(1)(f), 7(4), 13, 21, 35, 83(5) and 88. Text read from the EU Publications Office copy of the Official Journal.</li>\n<li>Article 29 Working Party, <a href=\"https://ec.europa.eu/newsroom/article29/items/610169\">Opinion 2/2017 on data processing at work (WP249)</a>, adopted June 8, 2017.</li>\n<li>Article 29 Working Party, <a href=\"https://ec.europa.eu/newsroom/article29/items/611236\">Guidelines on Data Protection Impact Assessment (WP248 rev.01)</a>, October 2017.</li>\n<li>European Data Protection Board, <a href=\"https://www.edpb.europa.eu/system/files/documents/files/file1/edpb_guidelines_202005_consent_en.pdf\">Guidelines 05/2020 on consent</a>, version 1.1, May 2020, paragraphs 21 to 23.</li>\n<li>Information Commissioner’s Office, <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/data-protection-and-monitoring-workers/\">Data protection and monitoring workers</a> and <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/specific-data-protection-considerations-for-different-ways-or-methods-of-monitoring-workers/\">different methods of monitoring</a>, under review after the Data (Use and Access) Act.</li>\n<li>Integritetsskyddsmyndigheten (IMY), <a href=\"https://www.imy.se/verksamhet/dataskydd/dataskydd-pa-olika-omraden/arbetsliv/kontroll-och-overvakning-av-anstallda/\">Kontroll och övervakning av anställda</a>, updated June 15, 2026.</li>\n<li>CNIL, <a href=\"https://www.cnil.fr/fr/surveillance-excessive-des-salaries-sanction-de-40-000-euros-entreprise-secteur-immobilier\">Surveillance excessive des salariés : sanction de 40 000 euros</a>, decision SAN-2024-021 of December 19, 2024.</li>\n<li>Conseil d’État, <a href=\"https://www.conseil-etat.fr/fr/arianeweb/CE/decision/2025-12-23/492830\">decision no. 492830</a>, December 23, 2025 (Amazon France Logistique v. CNIL, reviewing CNIL decision SAN-2023-021).</li>\n<li>Germany, <a href=\"https://www.gesetze-im-internet.de/betrvg/__87.html\">Betriebsverfassungsgesetz section 87</a>; Sweden, <a href=\"https://www.riksdagen.se/sv/dokument-och-lagar/dokument/svensk-forfattningssamling/lag-1976580-om-medbestammande-i-arbetslivet_sfs-1976-580/\">lag (1976:580) om medbestämmande i arbetslivet</a>.</li>\n</ul>\n",
      "summary": "Recording staff to find work worth automating usually needs a legitimate interest test and an impact assessment, not consent. Regulators' texts, compared.",
      "image": "https://28creation.com/og/employee-monitoring-gdpr-automation.jpg",
      "date_published": "2026-10-21T07:00:00.000Z",
      "date_modified": "2026-10-21T07:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/php-critical-security-release-october-2026/",
      "url": "https://28creation.com/articles/php-critical-security-release-october-2026/",
      "title": "PHP patches two critical bugs today; 8.1 and older get no fix",
      "content_html": "<h2>The short read</h2><p><strong>PHP fixes two critical flaws at 14:00 UTC today, but about a third of PHP sites run unsupported versions.</strong></p><ul><li>Both bugs are rated 9.2 of 10 and may allow remote code execution. Details leaked on Tuesday, before the planned release.</li></ul><h3>What you can do with this</h3><blockquote><p>Run php -v on every server, or ask your host which PHP version your sites use. If it is 8.2 or newer, book the update for 14:00 UTC; if older, ask for a backport today.</p></blockquote><hr><p>The PHP project will release fixes for two critical vulnerabilities today, Wednesday, October 21, at 14:00 UTC (10:00 in New York, 16:00 in Central Europe). PHP’s <a href=\"https://www.php.net/archive/2026.php#2026-10-20-1\">own notice</a> rates both CVSS v4.0 9.2 and asks every PHP user to “prepare now.” If your company runs a site on WordPress or Magento, this is your update.</p>\n<p>The catch is which versions get it. PHP only supports 8.2 and newer. And details leaked before the release, so attackers may move before many sites patch.</p>\n<h2 id=\"what-php-has-said-and-what-leaked\">What PHP has said, and what leaked</h2>\n<p>PHP’s security team posted the notice on Tuesday at 19:33 UTC, <a href=\"https://news-web.php.net/php.announce/509\">signed by Jakub Zelenka</a> on the php.announce list. It says “full details, affected versions, and the fixed versions will be published with the release,” and that PHP may release earlier if details go public or attacks start.</p>\n<p>They already went public in part. A <a href=\"https://news.ycombinator.com/item?id=50167754\">Hacker News post</a> at 15:20 UTC on Tuesday claimed two PHP remote code execution flaws were coming, citing an unnamed source. The EU’s computer emergency team, <a href=\"https://cert.europa.eu/publications/security-advisories/2026-018/\">CERT-EU</a>, lists them as CVE-2026-109618 and CVE-2026-109619 and says a 9.2 score suggests unauthenticated remote code execution. <a href=\"https://sansec.io/research/php2shell\">Sansec</a>, a store security firm, reports that a screenshot of an exploit tool called PHP2Shell leaked and that watchTowr, another security firm, confirmed the bugs.</p>\n<p>Two claims stay unconfirmed. CERT-EU says the flaws “could affect all PHP versions from PHP 5 onwards,” but flags that this comes from social media. Sansec expects mass attacks within 15 minutes of the release; that is a forecast, not an observation.</p>\n<h2 id=\"why-older-php-is-the-real-exposure\">Why older PHP is the real exposure</h2>\n<p>PHP gives each version a fixed <a href=\"https://www.php.net/supported-versions.php\">support window</a>: 8.2 gets security fixes until December 31, 2026, and 8.5 until the end of 2029. Anything older than 8.2 gets nothing from the project. Sansec’s advice for servers on 8.1 or older is to ask the host to backport the fix or move to a supported version.</p>\n<p>That is not a small group. <a href=\"https://w3techs.com/technologies/details/pl-php\">W3Techs</a> counted PHP on 69.3% of websites whose server language it can see, as of today. Of those PHP sites, 27.0% run version 7 and 7.7% version 5, with a further 0.1% on version 4.</p>\n<p>By our arithmetic that is 34.8%, about one PHP site in three, on versions with no official fix. The real share is higher, because PHP 8.0 and 8.1 are also out of support and W3Techs does not split version 8.</p>\n<h2 id=\"the-stopgap-people-are-using-and-its-limit\">The stopgap people are using, and its limit</h2>\n<p>Reports gathered by <a href=\"https://lilting.ch/en/articles/php2shell-session-upload-progress-uaf\">lilting.ch</a> place at least one bug in PHP’s upload progress feature; PHP has not confirmed this. The feature is on by default: the <a href=\"https://github.com/php/php-src/blob/master/ext/session/session.c\">PHP source</a> sets <code>session.upload_progress.enabled</code> to 1. Turning it off stops only progress tracking for file uploads, according to the same report, and uploads still work.</p>\n<p>It is partial cover at best. A researcher quoted in the report says the switch blocks only one of the two bugs.</p>\n<p>The setting cannot be changed from inside a script. PHP’s manual says settings of this kind are set in <a href=\"https://www.php.net/manual/en/configuration.changes.modes.php\">php.ini, .htaccess, httpd.conf or .user.ini</a>.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Find every PHP site you own or pay for.</strong> Run <code>php -v</code> on servers you manage. For hosted sites, ask the host which version runs and when it will apply today’s fix.</li>\n<li><strong>On 8.2 to 8.5,</strong> plan the update for 14:00 UTC. The current releases are <a href=\"https://www.php.net/archive/2026.php\">8.5.11, 8.4.26, 8.3.35 and 8.2.34</a>; PHP will name the fixed ones at release time.</li>\n<li><strong>On 8.1 or older,</strong> ask the host for a backport today, or start the move to 8.2 or newer. Note that 8.2 support ends on December 31.</li>\n<li><strong>Until you patch,</strong> set <code>session.upload_progress.enabled = Off</code> in php.ini or .user.ini if your site has no upload progress bar, and reload PHP. Treat it as partial cover.</li>\n<li><strong>If you cannot patch a public site soon,</strong> CERT-EU suggests isolating it from the internet until you can. That is costly for a shop, so weigh it against how much the site holds.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>PHP, “Upcoming Critical PHP Security Release,” news archive, October 20, 2026: <a href=\"https://www.php.net/archive/2026.php#2026-10-20-1\">https://www.php.net/archive/2026.php#2026-10-20-1</a></li>\n<li>php.announce #509, Jakub Zelenka, October 20, 2026: <a href=\"https://news-web.php.net/php.announce/509\">https://news-web.php.net/php.announce/509</a></li>\n<li>CERT-EU, Security Advisory 2026-018, “Critical Vulnerabilities in PHP”: <a href=\"https://cert.europa.eu/publications/security-advisories/2026-018/\">https://cert.europa.eu/publications/security-advisories/2026-018/</a></li>\n<li>Sansec, “Critical PHP RCE: CVE-2026-109618 and CVE-2026-109619,” October 21, 2026: <a href=\"https://sansec.io/research/php2shell\">https://sansec.io/research/php2shell</a></li>\n<li>Hacker News, “Tell HN: 2x PHP RCE CVEs with CVSS 10.0 Incoming,” October 20, 2026: <a href=\"https://news.ycombinator.com/item?id=50167754\">https://news.ycombinator.com/item?id=50167754</a></li>\n<li>PHP, supported versions: <a href=\"https://www.php.net/supported-versions.php\">https://www.php.net/supported-versions.php</a></li>\n<li>W3Techs, usage statistics of PHP, October 21, 2026: <a href=\"https://w3techs.com/technologies/details/pl-php\">https://w3techs.com/technologies/details/pl-php</a></li>\n<li>lilting.ch, “Disabling session.upload_progress Before the php2shell Patch Lands”: <a href=\"https://lilting.ch/en/articles/php2shell-session-upload-progress-uaf\">https://lilting.ch/en/articles/php2shell-session-upload-progress-uaf</a></li>\n<li>PHP source, ext/session/session.c: <a href=\"https://github.com/php/php-src/blob/master/ext/session/session.c\">https://github.com/php/php-src/blob/master/ext/session/session.c</a></li>\n<li>PHP manual, INI mode constants: <a href=\"https://www.php.net/manual/en/configuration.changes.modes.php\">https://www.php.net/manual/en/configuration.changes.modes.php</a></li>\n</ul>\n",
      "summary": "PHP ships fixes for two remote code execution flaws at 14:00 UTC. Sites on PHP 8.2 or newer can patch; older ones need a host's backport or an upgrade.",
      "image": "https://28creation.com/og/php-critical-security-release-october-2026.jpg",
      "date_published": "2026-10-21T05:40:51.000Z",
      "date_modified": "2026-10-21T05:40:51.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/gemini-small-business-hub-data-terms/",
      "url": "https://28creation.com/articles/gemini-small-business-hub-data-terms/",
      "title": "Gemini small-business hub: which account keeps receipts private",
      "content_html": "<h2>The short read</h2><p><strong>Gemini&apos;s new business tools take your receipts, but on a personal account some chats may go to human reviewers.</strong></p><ul><li>Google announced a Gemini business page with finance skills and apps such as Shopify and Xero, rolling out to all Gemini users soon.</li></ul><h3>What you can do with this</h3><blockquote><p>Before you upload receipts or connect Xero, open Gemini under a Google Workspace account, or turn Keep Activity off on a personal one. Your books then stay out of model training.</p></blockquote><hr><p>Google on Tuesday <a href=\"https://blog.google/innovation-and-ai/products/gemini-app/new-tools-small-business-owners/\">announced</a> a business page inside the Gemini app that turns receipts and purchase records into financial charts and connects to apps such as Shopify and Xero. It reaches “all Gemini app users” in the coming days. The catch for an owner is the account: on a personal Google account, those receipts fall under consumer terms that allow human review and model training.</p>\n<p>Below: what Google shipped, why the account type matters, and the two settings to check before uploading your books.</p>\n<h2 id=\"what-google-shipped-on-tuesday\">What Google shipped on Tuesday</h2>\n<p>The post, by Vishnu Sivaji, senior director for the Gemini app, describes a “new business landing page designed to help small business owners streamline their daily tasks.” It has three parts.</p>\n<ul>\n<li><strong>Brand:</strong> owners upload their website or a brand kit with fonts and color palettes, and Gemini keeps new images in that style with Nano Banana 2.1, which Google calls its latest image model.</li>\n<li><strong>Finance:</strong> Gemini turns “messy spreadsheets” and receipts into “clear financial graphics and interactive models.”</li>\n<li><strong>Connected apps:</strong> three Google services, among them Workspace, plus seven outside apps such as Shopify and Xero.</li>\n</ul>\n<p>Google lists no price. Larger firms that want “centralized administrative controls” are pointed to <a href=\"https://cloud.google.com/gemini-enterprise\">Gemini Enterprise</a>.</p>\n<p>Xero confirmed its part three days earlier. Its <a href=\"https://blog.xero.com/news-events/xero-gemini-integration/\">post</a> says the app “brings live Xero data into your Gemini conversations”: cash position and profit as well as invoices and bills. Anything that changes the books, such as issuing an invoice, needs the user to “review and confirm it before anything happens.”</p>\n<h2 id=\"why-the-account-decides-who-can-read-your-books\">Why the account decides who can read your books</h2>\n<p>Gemini runs under two sets of data terms, and Google’s announcement mentions neither.</p>\n<p>On a personal account, the <a href=\"https://support.google.com/gemini/answer/13594961\">Gemini Apps Privacy Hub</a> says: “A subset of chats are reviewed by human reviewers.” Google uses activity to improve its services, “including training generative AI models,” unless you turn the Keep Activity setting off. “Reviewed chats are retained for up to three years.” Activity is otherwise auto-deleted after 18 months by default.</p>\n<p>On a qualifying Google Workspace edition, the <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">Workspace privacy hub</a> says chats and uploaded files “won’t be reviewed by human reviewers or otherwise used to train generative AI models without your permission.” It adds: “Your interactions with Gemini stay within your organization.” An admin can also set how long chats are kept, from 3 to 36 months.</p>\n<p>Xero says “Xero data isn’t used to train Google Gemini models.” That covers the live data the connector pulls. A receipt you photograph and upload yourself is a different matter, governed by your Gemini account terms. And disconnecting a connected app, the consumer privacy hub notes, does not delete data already saved in your Gemini activity.</p>\n<p>We think this gap matters more than any single feature. Google built the page for “all Gemini app users,” personal accounts included, not only Workspace customers, and its post does not say which terms apply.</p>\n<h2 id=\"what-to-do-before-you-connect-anything\">What to do before you connect anything</h2>\n<ol>\n<li><strong>Check which account you are in.</strong> Open Gemini with your Workspace account if your business has one. Most Workspace editions now include the Gemini app as a core service.</li>\n<li><strong>On a personal account, turn Keep Activity off</strong> before uploading receipts or connecting Xero. Future chats are then not used for training unless you send feedback, and are kept for 72 hours.</li>\n<li><strong>Upload copies with less in them.</strong> Cut card numbers and customer names from receipts and spreadsheets when the chart does not need them.</li>\n<li><strong>In Workspace, set retention.</strong> An admin can choose 3 months instead of the default 18.</li>\n</ol>\n<p>The trade-off is real. With Keep Activity off, chats are kept for only 72 hours, so a finance model you build today will not wait for you next month. And Xero’s planned scheduled tasks, such as daily cash summaries, are “coming soon” and its footnote excludes the EEA and the UK, as well as Switzerland and Nigeria.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Google, <a href=\"https://blog.google/innovation-and-ai/products/gemini-app/new-tools-small-business-owners/\">New Gemini tools for small business owners</a>, October 20, 2026</li>\n<li>Xero, <a href=\"https://blog.xero.com/news-events/xero-gemini-integration/\">Xero + Google Gemini: Trusted numbers for every client, faster</a>, October 17, 2026</li>\n<li>Google, <a href=\"https://support.google.com/gemini/answer/13594961\">Gemini Apps Privacy Hub</a>, updated October 19, 2026</li>\n<li>Google, <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">Generative AI in Google Workspace Privacy Hub</a>, updated October 6, 2026</li>\n</ul>\n",
      "summary": "Google's new Gemini tools read receipts and Xero books. On a personal account, people may review chats; on Workspace, they stay in house.",
      "image": "https://28creation.com/og/gemini-small-business-hub-data-terms.jpg",
      "date_published": "2026-10-20T18:40:11.000Z",
      "date_modified": "2026-10-20T18:40:11.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "AI",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/google-cloud-london-outage-one-zone/",
      "url": "https://28creation.com/articles/google-cloud-london-outage-one-zone/",
      "title": "Google Cloud London outage: one-zone power event hit 38 products",
      "content_html": "<h2>The short read</h2><p><strong>A power event in one Google Cloud zone in London led to almost eight hours of trouble across 38 products.</strong></p><ul><li>Revolut&apos;s services were disrupted for hours, and Google reported errors elsewhere in the europe-west2 region, not only in the zone that lost power.</li></ul><h3>What you can do with this</h3><blockquote><p>List which of your systems and vendors run in europe-west2, and keep spare capacity running in a second zone. Creating it mid-outage may fail, as Google warned on Tuesday.</p></blockquote><hr><p>Google Cloud’s London region, europe-west2, had problems for almost eight hours on Tuesday, October 20. Google’s <a href=\"https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPyzj\">incident record</a> dates the start to 06:08 UTC, names a “power event” in a single zone, europe-west2-b, and lists 38 affected products.</p>\n<p>Google’s own design guide says a one-zone power fault should be survivable by moving to the region’s other zones. On Tuesday, Google still reported errors elsewhere in the region. If you or a supplier run in London on Google Cloud, that gap is your risk.</p>\n<h2 id=\"what-broke-and-who-noticed\">What broke, and who noticed</h2>\n<p>The best-known affected company was Revolut. <a href=\"https://www.thestack.technology/google-cloud-incident-knocks-revolut-offline/\">The Stack</a> reported that the bank told its 80 million users it was “experiencing a service disruption affecting multiple services,” and that its status page showed core services failing. Bloomberg <a href=\"https://www.bloomberg.com/news/articles/2026-10-20/google-cloud-outage-takes-down-revolut-and-other-banks-in-uk\">reported</a> that Revolut blamed Google Cloud, while The Stack noted that Revolut’s own statement did not name a cause.</p>\n<p>Other UK apps reported trouble too. The budgeting app Emma blamed the Google Cloud disruption for its own outage, and Halifax acknowledged problems with its app and website without saying why, according to <a href=\"https://telanganatoday.com/google-cloud-outage-in-london-disrupts-revolut-other-online-services\">IANS</a>. IANS also noted it was unclear whether every affected service relied on Google Cloud.</p>\n<h2 id=\"googles-timeline-from-its-own-updates\">Google’s timeline, from its own updates</h2>\n<p>We read every update in Google’s incident record. The times, in UTC:</p>\n<ul>\n<li><strong>06:08:</strong> the problem starts.</li>\n<li><strong>08:35:</strong> first public update, 2 hours 27 minutes after the start: “elevated errors and intermittent network failures in europe-west2 region.” Workaround: “None for now.”</li>\n<li><strong>09:24:</strong> Google suggests moving affected workloads “to another available location (zone or region).”</li>\n<li><strong>11:12:</strong> the cause appears. “Due to a power event we had suffered an outage within europe-west2-b zone.” Services in that zone are unavailable, and “some services in europe-west2 region may experience elevated errors.”</li>\n<li><strong>11:58:</strong> Google is “prioritizing the recovery of existing workloads. Customers may not be able to yet create new workloads.”</li>\n<li><strong>14:03:</strong> mitigated “for most of the customers.” Google promises a written analysis.</li>\n</ul>\n<p>The list of 38 products covers the basics most apps sit on: Compute Engine, Kubernetes Engine, Cloud SQL, Cloud Storage, Pub/Sub, Cloud Run, Spanner, load balancing and Cloud NAT.</p>\n<h2 id=\"why-one-zone-was-supposed-to-be-enough\">Why one zone was supposed to be enough</h2>\n<p>Google’s own <a href=\"https://docs.cloud.google.com/compute/docs/regions-zones\">regions and zones documentation</a> says it designs zones “to minimize the risk of correlated failures caused by physical infrastructure outages like power, cooling, or networking,” so that if one zone goes down “you can transfer traffic to another zone in the same region.”</p>\n<p>That is the promise multi-zone setups rest on. On Tuesday the fault was a power event, it started in one zone, and Google still reported errors in other parts of the region. The same page says spreading across regions “provides an even higher degree of failure independence.” We do not yet know why the damage spread beyond europe-west2-b; Google has not published its analysis.</p>\n<p>The 11:58 note matters too. Plans that create fresh capacity during an outage, such as autoscaling into a healthy zone, assume you can create new workloads. For part of the recovery, Google said you might not.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Check where you actually run.</strong> List which of your services, and which of your critical vendors, sit in europe-west2. Ask vendors directly; Revolut’s statement did not name its provider.</li>\n<li><strong>Keep spare capacity running in a second zone</strong> rather than relying on creating it when trouble starts.</li>\n<li><strong>Price a second region for the few systems you cannot lose for eight hours,</strong> such as payments or customer login. Google’s documentation says traffic within a region is “always cheaper and faster” than across regions, so this costs money and latency; spend it only where an outage would cost more.</li>\n<li><strong>Follow Google’s status page</strong> at <code>status.cloud.google.com</code>, but do not rely on it. On Tuesday, Google’s first public note came 2 hours 27 minutes after the start, so your own monitoring has to raise the alarm first.</li>\n<li><strong>Read Google’s analysis when it lands,</strong> and check whether the errors outside europe-west2-b came from a shared service you also depend on.</li>\n</ol>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPyzj\">Multiple services are experiencing issues in the europe-west2 region</a>, Google Cloud Service Health, October 20, 2026</li>\n<li><a href=\"https://docs.cloud.google.com/compute/docs/regions-zones\">Regions and zones</a>, Google Cloud documentation</li>\n<li><a href=\"https://www.thestack.technology/google-cloud-incident-knocks-revolut-offline/\">Google Cloud incident knocks Revolut offline</a>, The Stack, October 20, 2026</li>\n<li><a href=\"https://telanganatoday.com/google-cloud-outage-in-london-disrupts-revolut-other-online-services\">Google Cloud outage in London disrupts Revolut, other online services</a>, IANS via Telangana Today, October 20, 2026</li>\n<li><a href=\"https://www.bloomberg.com/news/articles/2026-10-20/google-cloud-outage-takes-down-revolut-and-other-banks-in-uk\">Bloomberg’s report on the outage</a>, October 20, 2026, as summarized on <a href=\"https://www.techmeme.com/261020/p13#a261020p13\">Techmeme</a></li>\n</ul>\n",
      "summary": "A power event in one London zone disrupted Revolut for hours, and Google reported errors beyond that zone. What that means for multi-zone setups.",
      "image": "https://28creation.com/og/google-cloud-london-outage-one-zone.jpg",
      "date_published": "2026-10-20T14:40:22.000Z",
      "date_modified": "2026-10-20T14:40:22.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/xkcd-worth-the-time-upkeep/",
      "url": "https://28creation.com/articles/xkcd-worth-the-time-upkeep/",
      "title": "xkcd's automation table, recomputed with upkeep counted",
      "content_html": "<h2>The short read</h2><p><strong>At an assumed hour a month of upkeep, 5 of 7 tasks in our recomputed xkcd table never pay back.</strong></p><ul><li>xkcd 1205 counts 24-hour days, 365-day years and no upkeep; we redid it for a 260-day working year and subtracted fixes.</li></ul><h3>What you can do with this</h3><blockquote><p>Before you build, multiply minutes saved by runs a year and compare it with the upkeep you expect. If upkeep is larger, do it by hand or keep it a throwaway script.</p></blockquote><hr><p>The xkcd chart <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a> says how long you can spend making a routine task faster before you lose time over five years. It counts 24-hour days, 365-day years and zero upkeep. We recomputed it for a working year and subtracted the hours an automation needs to keep running.</p>\n<p>The change matters to anyone deciding what to automate at a small company, because upkeep is the line the chart leaves blank.</p>\n<p>At an assumed hour a month of it, five of the seven common tasks in our table never pay back at all, however fast you build. This piece shows why the chart reads the way it does, what forces upkeep, and a 28-line Python script you can run with your own numbers.</p>\n<h2 id=\"the-chart-counts-calendar-time-not-working-time\">The chart counts calendar time, not working time</h2>\n<p>Randall Munroe published the chart on <a href=\"https://xkcd.com/1205/info.0.json\">29 April 2013</a>. Rows give the time shaved off one run, from 1 second to 1 day. Columns give how often you do the task, from 50 times a day to yearly. Each cell is the time saved over five years, which is the most you can spend on the fix and still break even.</p>\n<p>The comic never prints its formula, so we checked one.</p>\n<p>We multiplied time shaved by runs per year by five, using 24-hour days and 365-day years, for all 45 cells that hold a number. Every result landed within 27% of the printed cell, and the gaps match the comic’s rounding to whole units. A minute shaved off a daily task gives 30.4 hours; the comic rounds that to “1 day”. The other 9 cells read “n/a”, with no number to check.</p>\n<p>Two of those assumptions do not fit office work. A task you do “daily” at work happens on weekdays, which is 52 times 5, or 260 days a year before holidays and leave, not 365. And a “day” of building is a workday of about 8 hours, not 24.</p>\n<p>Guess which way the two corrections push. They pull in opposite directions. Counting 260 days cuts the hours saved by 29% for anything done daily or more often, and counting 8-hour days triples the number of days those hours buy.</p>\n<p>Net, the minute shaved off a daily task is worth 21.7 hours over five years, or 2.7 workdays, where the comic says 1 day. Weekly, monthly and yearly tasks keep their hours, since they repeat the same number of times either way.</p>\n<p>So the chart is not too strict for office work in days. It is too generous in one larger way: it assumes the automation, once built, costs nothing more.</p>\n<h2 id=\"upkeep-is-forced-by-the-platforms-on-their-schedule\">Upkeep is forced by the platforms, on their schedule</h2>\n<p>The chart’s own sequel says as much. In <a href=\"https://xkcd.com/1319/\">xkcd 1319, “Automation”</a>, the “theory” panel shows coding time ending as the automation takes over the task. In the “reality” panel, writing code turns into debugging, rethinking and “ongoing development”, and the original task gets no time at all.</p>\n<p>That is a joke, but the platforms office workers build on put the same lesson in writing. Microsoft’s <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Power Automate limits page</a> says a cloud flow that “isn’t triggered within a 90 day period might be turned off” unless its owner has a premium or capacity license. Owners are notified 30 days before. A flow whose trigger or actions “fail continuously” for 14 days is turned off too.</p>\n<p>Zapier’s help center says it will <a href=\"https://help.zapier.com/hc/en-us/articles/8496216132621-Zap-is-not-running\">turn off a Zap</a> that errors 95% of the time it runs and has run more than 20 times in the past 7 days. Team plans get a 24-hour grace period and Enterprise plans 72 hours.</p>\n<p>Scripts are not exempt. Google keeps a <a href=\"https://developers.google.com/apps-script/guides/support/sunset\">table of retired Apps Script features</a>, and seven of them reached their sunset date between November 2022 and June 2026. One was the Rhino runtime: “Scripts running on Rhino will continue to function until January 31, 2026, after which they will no longer execute.” Others, such as the Contacts and Sites services, now simply no longer function.</p>\n<p>None of these is a bug in your automation. Each one is a date or a rule set by someone else, and each one costs time you did not put in the chart.</p>\n<h2 id=\"the-upkeep-line-decides-most-cells\">The upkeep line decides most cells</h2>\n<p>Here is the recomputed table. The script below printed it in our sandbox on 20 October 2026. Rows are seven common cells from the comic, with its “1 day” row read as one 8-hour workday. Each column subtracts a different amount of yearly upkeep, meaning time spent checking, fixing and migrating.</p>\n<p>The upkeep levels are inputs we chose, not measurements. We found no published measure of how much upkeep office automations need. Read 2 hours a year as “one small fix a year”, 12 as “an hour a month” and 24 as “two hours a month”.</p>\n<p>Every other number is arithmetic on the comic’s rows. A “day” in the table is 8 working hours.</p>\n<table>\n<thead>\n<tr>\n<th>Saved per run, how often</th>\n<th>Saved per year</th>\n<th>Upkeep 0 h/yr</th>\n<th>Upkeep 2 h/yr</th>\n<th>Upkeep 12 h/yr</th>\n<th>Upkeep 24 h/yr</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>30 seconds, 5 times a day</td>\n<td>10.8 h</td>\n<td>54 h (6.8 days)</td>\n<td>44 h (5.5 days)</td>\n<td>never pays</td>\n<td>never pays</td>\n</tr>\n<tr>\n<td>1 minute, daily</td>\n<td>4.3 h</td>\n<td>22 h (2.7 days)</td>\n<td>12 h (1.5 days)</td>\n<td>never pays</td>\n<td>never pays</td>\n</tr>\n<tr>\n<td>5 minutes, daily</td>\n<td>21.7 h</td>\n<td>108 h (13.5 days)</td>\n<td>98 h (12.3 days)</td>\n<td>48 h (6.0 days)</td>\n<td>never pays</td>\n</tr>\n<tr>\n<td>5 minutes, weekly</td>\n<td>4.3 h</td>\n<td>22 h (2.7 days)</td>\n<td>12 h (1.5 days)</td>\n<td>never pays</td>\n<td>never pays</td>\n</tr>\n<tr>\n<td>30 minutes, weekly</td>\n<td>26.0 h</td>\n<td>130 h (16.2 days)</td>\n<td>120 h (15.0 days)</td>\n<td>70 h (8.8 days)</td>\n<td>10 h (1.2 days)</td>\n</tr>\n<tr>\n<td>1 hour, monthly</td>\n<td>12.0 h</td>\n<td>60 h (7.5 days)</td>\n<td>50 h (6.2 days)</td>\n<td>never pays</td>\n<td>never pays</td>\n</tr>\n<tr>\n<td>1 workday, yearly</td>\n<td>8.0 h</td>\n<td>40 h (5.0 days)</td>\n<td>30 h (3.8 days)</td>\n<td>never pays</td>\n<td>never pays</td>\n</tr>\n</tbody>\n</table>\n<p>Each cell is the most you can spend building it and still break even over five years. “Never pays” means the upkeep alone eats the saving, so even a build that took no time loses hours. The hour-a-month task sits exactly on that line: 12 hours saved, 12 spent.</p>\n<p>The pattern is simple once it is written down. The hours a task saves each year are the most upkeep it can carry. A minute shaved off a daily task saves 4.3 hours a year, so a little over half a day of fixes each year wipes it out. Thirty minutes shaved off a weekly task saves 26 hours a year and survives two hours of upkeep a month.</p>\n<h2 id=\"rare-tasks-carry-an-extra-risk\">Rare tasks carry an extra risk</h2>\n<p>The rules above hit infrequent tasks hardest. A yearly task leaves its flow untriggered for far longer than 90 days, so on a standard license its Power Automate flow is exactly the kind Microsoft says “might be turned off”. A monthly flow stays inside the window.</p>\n<p>The yearly row also has the thinnest margin, 8 hours a year in our working-time version. A migration like Rhino that took a few hours, once in five years, would take a real bite of its 40-hour budget. And a yearly job that breaks is found a year late, when the task is due.</p>\n<p>A hypothetical shows the cost of a silent failure. Say a daily automation that saves 5 minutes stops for a month of 22 workdays before anyone notices. That is 110 minutes of work to catch up by hand, plus the time to find the fault, and it goes in the upkeep column too.</p>\n<p>Our guide to <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">scheduled scripts that fail silently</a> covers how to make a failure reach a person the same day.</p>\n<h2 id=\"the-case-for-automating-anyway\">The case for automating anyway</h2>\n<p>There is an honest argument against all of this arithmetic.</p>\n<p>In a December 2024 post titled <a href=\"https://will-keleher.com/posts/its-not-worth-the-time-yet.html\">Xkcd’s “Is It Worth the Time?” Considered Harmful</a>, the author describes a fix to a pattern in about 10 spots in a codebase. By hand it would have taken a minute; automating it took an hour. “And I think that was the right choice,” the post says, because of what the author learned along the way.</p>\n<p>The claim is that “automating the easy things is how you build the skills, mindset, and muscle-memory to automate the hard things.” The post also grants that doing a task by hand can make sense when all that matters is getting it done quickly. The post drew <a href=\"https://news.ycombinator.com/item?id=43947209\">27 points and 18 comments</a> on Hacker News.</p>\n<p>We think the post is right about one-off scripts that you run yourself and can throw away. Those carry almost no upkeep, because nobody depends on them next month.</p>\n<p>The case weakens for a flow other people rely on, where every broken run lands on a colleague. Learning is a fine reason to build. It is a poor reason to leave something running that costs more than it saves.</p>\n<h2 id=\"what-this-means-for-you-run-the-numbers-before-the-build\">What this means for you: run the numbers before the build</h2>\n<ol>\n<li><strong>Count runs on a working calendar.</strong> Multiply minutes saved per run by runs per year: 260 for a daily task, 52 weekly, 12 monthly. Divide by 60. That is the most upkeep the automation can ever carry.</li>\n<li><strong>List what it depends on.</strong> Each connector, login, file layout and platform rule is a place upkeep comes from. Check the platform’s own limits page: Power Automate’s 90-day and 14-day rules, Zapier’s error rule, Google’s sunset table.</li>\n<li><strong>Guess upkeep high, then compare.</strong> If one hour a month would eat the yearly saving, as it does for five of our seven rows, keep doing the task by hand or make it a one-off script you run yourself.</li>\n<li><strong>Give rare tasks a reminder.</strong> For a yearly or quarterly job, put a calendar check a week before it is due, and test the run then.</li>\n<li><strong>Run the script with your own rows.</strong> Change <code>tasks</code> and <code>upkeep_levels</code> below. It needs Python 3 and nothing else.</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># xkcd 1205 recomputed for a working year, with upkeep subtracted.</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># Inputs: time shaved off each run, runs per year, upkeep hours per year.</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">YEARS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 5</span><span style=\"color:#6A737D;--shiki-dark:#768390\">        # the comic's horizon</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">WORKDAYS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 260</span><span style=\"color:#6A737D;--shiki-dark:#768390\">   # 52 weeks x 5 weekdays, before holidays and leave</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">HOURS_PER_DAY</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 8</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">tasks </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># (label, minutes saved per run, runs per year)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"30 seconds, 5 times a day\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0.5</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">5</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> WORKDAYS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"1 minute, daily\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,           </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,   </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">WORKDAYS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"5 minutes, daily\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,          </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">5</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,   </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">WORKDAYS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"5 minutes, weekly\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,         </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">5</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,   </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">52</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"30 minutes, weekly\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,        </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">30</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,  </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">52</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"1 hour, monthly\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,           </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">60</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,  </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">12</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"1 workday, yearly\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,         </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">60</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> HOURS_PER_DAY</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">upkeep_levels </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">12</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">24</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># hours per year spent fixing and checking</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"| Saved per run, how often | Saved per year | \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">      \" | \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Upkeep </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">u</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> h/yr\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> u </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> upkeep_levels) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" |\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"|---\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(upkeep_levels)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"|\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> label, minutes, runs </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tasks:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    saved </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> minutes </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> runs </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 60</span><span style=\"color:#6A737D;--shiki-dark:#768390\">                      # hours a year</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> []</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> u </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> upkeep_levels:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        budget </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> YEARS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> *</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (saved </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> u)                 </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># build hours that still break even</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        cells.append(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">budget</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:.0f</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> h (</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">budget </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> HOURS_PER_DAY</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:.1f</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> days)\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                     if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> budget </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"never pays\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"| </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">label</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> | </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">saved</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:.1f</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> h | \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" | \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(cells) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" |\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span></code></pre>\n<p>If you have not picked a task yet, start with a <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a> of what you repeat, then run the strongest candidates through the table. More guides are on the <a href=\"https://28creation.com/topics/automation/\">automation page</a>. The comic’s alt text adds that the time spent finding the chart counts too. Upkeep is the same kind of time: small, easy to leave out, and billed again every year the automation runs.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Randall Munroe, <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a>, xkcd 1205, 29 April 2013; transcript and date from <a href=\"https://xkcd.com/1205/info.0.json\">info.0.json</a>. Licensed <a href=\"https://xkcd.com/license.html\">CC BY-NC 2.5</a>; linked, not reproduced.</li>\n<li>Randall Munroe, <a href=\"https://xkcd.com/1319/\">Automation</a>, xkcd 1319.</li>\n<li>Microsoft, <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Limits of automated, scheduled, and instant flows</a>, Power Automate documentation, updated 17 July 2026.</li>\n<li>Zapier, <a href=\"https://help.zapier.com/hc/en-us/articles/8496216132621-Zap-is-not-running\">Zap is not running</a>, help center, updated 15 October 2026.</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/support/sunset\">Apps Script deprecations and sunsets</a>.</li>\n<li><a href=\"https://will-keleher.com/posts/its-not-worth-the-time-yet.html\">Xkcd’s “Is It Worth the Time?” Considered Harmful</a>, will-keleher.com, 13 December 2024; <a href=\"https://news.ycombinator.com/item?id=43947209\">Hacker News discussion</a>.</li>\n<li>The table: our script above, run on 20 October 2026; formula check of all 45 numbered cells against the comic’s transcript, same day.</li>\n</ul>\n",
      "summary": "At an assumed hour a month of upkeep, 5 of 7 common tasks in our recomputed xkcd table never pay back. The script is below, so change the inputs.",
      "image": "https://28creation.com/og/xkcd-worth-the-time-upkeep.jpg",
      "date_published": "2026-10-20T12:30:00.000Z",
      "date_modified": "2026-10-20T12:30:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/",
      "url": "https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/",
      "title": "Zapier, Make and n8n priced for one workflow at 10,000 runs",
      "content_html": "<h2>The short read</h2><p><strong>At 10,000 runs a month, the same lead workflow costs $289 on Zapier and $50 on n8n Cloud, billed yearly.</strong></p><ul><li>Zapier bills every action, Make bills every module including the trigger, and n8n bills one execution per run however many steps it has.</li><li>Self-hosted n8n has no run quota. On our large sandbox machine it peaked at 884 MiB, so we would pick a 2 GiB server.</li></ul><h3>What you can do with this</h3><blockquote><p>Count the action steps in your workflow and multiply by your monthly runs before you pick a plan. Add 25% headroom, because Make and n8n Cloud pause workflows when the allowance runs out.</p></blockquote><hr><p>Zapier, Make and n8n don’t bill the same thing, so one workflow can cost ten times more on one than another. Zapier charges for every action and Make for every module, the trigger included, while n8n charges once per run. So we priced one real workflow on all three, from their pricing pages on October 20, 2026.</p>\n<p>The workflow is a common one. A website form sends a lead, and a filter drops entries without an email. The name and phone get cleaned up, then the lead goes to a CRM, a Google Sheet and a Slack channel.</p>\n<p>At 1,000 runs a month the cheapest yearly plans cost $9 on Make, $20 on n8n Cloud and $80 to $89 on Zapier. At 10,000 runs they cost about $29, $50 and $289.</p>\n<p>We also built the workflow in n8n and ran it 1,000 times on our own machine, to see what self-hosting asks of a server. We did not build it in Zapier or Make, because we have no paid accounts there; those counts come from each vendor’s documentation.</p>\n<h2 id=\"each-tool-counts-a-different-unit\">Each tool counts a different unit</h2>\n<p><strong>Zapier counts tasks.</strong> Zapier’s help center says “<a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261\">Zap triggers never use tasks</a>”, and neither do Filter, Paths or Formatter steps. Each successful action uses one task. Our workflow has three actions (CRM, sheet, Slack), so each lead costs 3 tasks, and a lead the filter stops costs nothing.</p>\n<p><strong>Make counts credits.</strong> Credits are Make’s billing unit, and “<a href=\"https://www.make.com/en/pricing\">most actions consume 1 credit</a>”. Make’s operations guide counts the trigger as <a href=\"https://help.make.com/operations\">one operation each time it runs</a>, whatever it returns. So our workflow costs 4 credits per lead: the trigger plus three modules. In Make a filter sits <a href=\"https://help.make.com/filtering\">on the line between two modules</a>, not in a module, so we count it as free; the Make pages we read don’t say so outright. A rejected lead still costs the trigger’s credit.</p>\n<p><strong>n8n counts executions.</strong> “<a href=\"https://n8n.io/pricing/\">An execution is a single run of your entire workflow</a>”, whatever the number of steps. A webhook trigger counts <a href=\"https://docs.n8n.io/build/understand-workflows/understand-executions/\">one execution for every inbound request</a>, including requests with an empty body. So every lead costs 1 execution, and so does every spam entry the filter rejects.</p>\n<p>That difference decides the result. The more steps your workflow has, the better n8n looks. The more runs end at a filter, the better Zapier looks.</p>\n<h2 id=\"the-bill-at-1000-and-10000-runs-a-month\">The bill at 1,000 and 10,000 runs a month</h2>\n<p>We put each vendor’s price list into a 41-line script and asked it for the cheapest plan that covers each volume. Zapier prices come from <a href=\"https://zapier.com/pricing\">its pricing page</a>, n8n prices from <a href=\"https://n8n.io/pricing/\">its pricing page</a>. Every run is assumed to be a real lead: 3 tasks, 4 credits or 1 execution.</p>\n<table>\n<thead>\n<tr>\n<th>Runs a month</th>\n<th>Zapier Pro, yearly</th>\n<th>Zapier Pro, monthly</th>\n<th>Make Core, yearly</th>\n<th>Make Core, monthly</th>\n<th>n8n Cloud, yearly</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>250</td>\n<td>$19.99</td>\n<td>$29.99</td>\n<td>$9.00</td>\n<td>$10.59</td>\n<td>$20.00</td>\n</tr>\n<tr>\n<td>1,000</td>\n<td>$79.63 to $89.00</td>\n<td>$133.50</td>\n<td>$9.00</td>\n<td>$10.59</td>\n<td>$20.00</td>\n</tr>\n<tr>\n<td>2,500</td>\n<td>$129.00</td>\n<td>$193.50</td>\n<td>$9.00</td>\n<td>$10.59</td>\n<td>$20.00</td>\n</tr>\n<tr>\n<td>5,000</td>\n<td>$189.00</td>\n<td>$283.50</td>\n<td>$16.00</td>\n<td>$18.82</td>\n<td>$50.00</td>\n</tr>\n<tr>\n<td>10,000</td>\n<td>$289.00</td>\n<td>$433.50</td>\n<td>$29.00</td>\n<td>$34.12</td>\n<td>$50.00</td>\n</tr>\n<tr>\n<td>20,000</td>\n<td>$361.25</td>\n<td>$650.25</td>\n<td>$55.00</td>\n<td>$64.71</td>\n<td>none listed</td>\n</tr>\n</tbody>\n</table>\n<p>Prices are in US dollars per month and read October 20, 2026.</p>\n<h2 id=\"three-of-those-prices-come-with-a-caveat\">Three of those prices come with a caveat</h2>\n<p><strong>Make’s prices are the weakest numbers in the table.</strong> Make’s pricing page loads its prices with a script, and Cloudflare’s bot check blocked our browser, so we could not read them there. Make’s own blog confirms the first tier: Core “<a href=\"https://www.make.com/blog/make-vs-zapier\">starts at $9/month (annual billing) for 10,000</a>” credits. The higher tiers and the monthly prices come from <a href=\"https://dev.to/stimlau/make-pricing-in-2026-free-core-pro-teams-and-credit-costs-lcm\">an independent write-up</a> that read the page on October 8, 2026, and that links to Make with an affiliate code. Check them on make.com before you buy.</p>\n<p><strong>Zapier’s 1,000-run price has two answers.</strong> 3,000 tasks don’t fit the 2,000-task tier ($49 a month yearly), so the clean choice is the 5,000-task tier at $89. The script found something cheaper: the 2,000 tier plus pay-per-task billing for the other 1,000 tasks. Zapier’s pricing page says extra tasks cost 1.25 times your base task rate on yearly plans and 2.5 times on monthly ones, up to <a href=\"https://help.zapier.com/hc/en-us/articles/15279018245901-How-pay-per-task-billing-works-in-Zapier\">3 times your plan’s task limit</a>. That makes $79.63. Zapier’s help center adds that the rate “depends on your plan and billing cycle type”, so confirm it in your billing settings.</p>\n<p><strong>n8n Cloud’s figures are yearly.</strong> The page lists Starter at $20 a month for 2,500 executions and Pro at $50 for 10,000, both “billed annually”. Yearly billing saves 17%, it says. It lists no Cloud plan with more than 10,000 executions except Enterprise, through sales. The Business plan, $800 a month for 40,000 executions, is self-hosted only.</p>\n<h2 id=\"the-limits-matter-more-than-the-prices-at-10000-runs\">The limits matter more than the prices at 10,000 runs</h2>\n<p>Two of the three cheap options have no headroom at 10,000 runs, and the tools behave differently at the limit.</p>\n<ul>\n<li><strong>n8n Cloud Pro</strong> includes exactly 10,000 executions. An n8n support answer on the <a href=\"https://community.n8n.io/t/92273\">community forum</a> says workflows are paused at 100% of the quota until the count resets on the 1st of the month. There is no per-execution overage charge. Ten rejected spam posts a day would push our workflow past the line.</li>\n<li><strong>Make Core at 40,000 credits</strong> is also an exact fit. Make says that when you run out, “<a href=\"https://www.make.com/en/pricing\">your scenarios won’t continue to run</a>” until credits are added. You can buy extra credits in bundles of 1,000 or 10,000, and since a November 2025 change they cost <a href=\"https://community.make.com/t/important-update-adjustment-to-plans-and-pricing/94578\">25% more than credits in your plan</a>.</li>\n<li><strong>Zapier</strong> keeps running past the limit if pay-per-task is on, at the higher rate, until 3 times the plan’s tasks. With it off, “<a href=\"https://zapier.com/pricing\">your Zap workflows will pause</a>” until the next usage period.</li>\n</ul>\n<p>So at 10,000 runs, plan on Make’s next tier, reported at $55 a month yearly for 80,000 credits. For n8n, the step past Cloud Pro is Enterprise or self-hosting. Both Make prices stay far below Zapier’s $289.</p>\n<h2 id=\"self-hosted-n8n-has-no-run-quota-but-it-needs-a-real-server\">Self-hosted n8n has no run quota, but it needs a real server</h2>\n<p>The free n8n Community Edition has no execution quota: per n8n’s docs, only <a href=\"https://docs.n8n.io/build/understand-workflows/understand-executions/\">paid plans</a> have one. Its <a href=\"https://raw.githubusercontent.com/n8n-io/n8n/master/LICENSE.md\">Sustainable Use License</a> allows use “only for your own internal business purposes or for non-commercial or personal use”. Running your own lead workflow fits; selling n8n as a hosted service to clients does not.</p>\n<h2 id=\"our-test-1000-leads-through-a-self-hosted-n8n\">Our test: 1,000 leads through a self-hosted n8n</h2>\n<p>To see what a server needs, we installed n8n 2.44.4 from npm on our Linux sandbox (4 vCPUs, 16 GB of memory) with Node.js 24 and the default SQLite database. We built the workflow above as seven nodes: Webhook, If, Edit Fields, three HTTP Request nodes and a webhook reply. A small local server stood in for the CRM, the sheet and Slack, because we have no test accounts there. A script then posted 1,000 made-up leads on <code>.example</code> addresses, five at a time, with every 20th lead missing its email.</p>\n<p>What happened:</p>\n<ul>\n<li><strong>All 1,000 posts got an answer.</strong> 950 leads reached all three stand-in services; 50 were rejected by the filter with a 422 reply. No errors.</li>\n<li><strong>Every post was an execution.</strong> n8n’s database logged 1,002 executions, the 1,000 plus our two manual checks, all marked successful, rejects included. On a paid plan the 50 rejects would count against the quota, as the docs say.</li>\n<li><strong>It took 49.7 seconds</strong>, about 20 runs a second, with a median reply of 253 ms. A month of 10,000 leads is far below that pace.</li>\n<li><strong>Memory was the cost.</strong> The main process used about 754 MiB right after start and about 470 MiB after the first requests. During the run, the main process and its task runner together peaked at 884 MiB. A second run, with the main process’s JavaScript heap capped at 384 MB, finished the same way and peaked at 782 MiB.</li>\n<li><strong>The database files came to about 15 MB</strong> after those 1,002 executions. n8n prunes by default after <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/manage-execution-data\">14 days or 10,000 executions</a>, so it won’t grow forever.</li>\n</ul>\n<h2 id=\"we-would-run-it-on-a-2-gib-server\">We would run it on a 2 GiB server</h2>\n<p>We ran this on a large machine, not on a small cloud server, so read the memory figures as what n8n used, not a proven minimum. n8n’s own <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration/prerequisites\">sizing example</a> gives 320 MB to 2 GB of memory and says memory needs usually outweigh CPU. On DigitalOcean, a <a href=\"https://www.digitalocean.com/pricing/droplets\">1 GiB droplet is $6 a month and a 2 GiB one is $12</a>. With our peak near 900 MiB, we would pick the 2 GiB server.</p>\n<p>$12 a month beats every hosted option above 2,500 runs, on paper. The real price is the hours: updates, backups, an HTTPS front end and someone to call when it stops. n8n’s <a href=\"https://docs.n8n.io/hosting/\">hosting docs</a> mark npm installs as deprecated from n8n 3.0 and point production users to Docker Compose.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision turns on three numbers you can count today: steps per run, runs per month, and how many runs end at a filter.</p>\n<ul>\n<li><strong>Under about 250 runs a month,</strong> use Make’s free plan. It includes 1,000 credits, enough for 250 of our leads. Zapier’s free plan allows only <a href=\"https://zapier.com/pricing\">two-step Zaps</a>, so our three-action workflow needs Pro at $19.99 a month yearly.</li>\n<li><strong>Pick Zapier</strong> when the apps you need only connect well there, or when most runs stop at a filter and cost nothing. Don’t pick it for long workflows at high volume: at 10,000 runs ours costs ten times the reported Make price.</li>\n<li><strong>Pick Make</strong> for the lowest hosted price on short-to-medium workflows, and set a credit alert. Every module and every trigger run counts, so a ten-module scenario costs ten times as much per run as a one-module one.</li>\n<li><strong>Pick n8n Cloud</strong> for long workflows with many steps, where one execution covers them all. Avoid it when much of your traffic is spam or empty webhooks, since each one is billed.</li>\n<li><strong>Self-host n8n</strong> only if someone on the team will own the server. Budget $12 a month for a 2 GiB server, plus regular time for updates and backups.</li>\n</ul>\n<h2 id=\"price-your-own-workflow-from-a-list-of-its-steps\">Price your own workflow from a list of its steps</h2>\n<p>If you are not sure this workflow is the one to automate first, our <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a> helps you find it. For a scheduled report, <a href=\"https://28creation.com/articles/automate-weekly-report/\">the weekly report guide</a> covers the Zapier schedule step by step, and <a href=\"https://28creation.com/articles/email-data-into-excel/\">our email-to-Excel test</a> shows a workflow that may not need any of these tools. More comparisons are under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Before you sign up, write your workflow as a list of steps and mark each as trigger, action or filter. Multiply by your runs and add 25%. That one line tells you which of the three bills you least.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Zapier, <a href=\"https://zapier.com/pricing\">Plans and pricing</a> (task tiers, plan limits, pay-per-task rates; read October 20, 2026)</li>\n<li>Zapier Help Center, <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261\">How is task usage measured in Zapier?</a></li>\n<li>Zapier Help Center, <a href=\"https://help.zapier.com/hc/en-us/articles/15279018245901-How-pay-per-task-billing-works-in-Zapier\">How pay-per-task billing works in Zapier</a></li>\n<li>Make, <a href=\"https://www.make.com/en/pricing\">Pricing and subscription packages</a> (credit rules and FAQ; prices did not load for us)</li>\n<li>Make, <a href=\"https://www.make.com/blog/make-vs-zapier\">Make vs Zapier</a> (Core at $9 a month yearly for 10,000 credits; April 2026)</li>\n<li>Make Help Center, <a href=\"https://help.make.com/operations\">Operations</a> and <a href=\"https://help.make.com/filtering\">Filtering</a></li>\n<li>Make Community, <a href=\"https://community.make.com/t/important-update-adjustment-to-plans-and-pricing/94578\">Important update: adjustment to plans and pricing</a> (November 2025)</li>\n<li>dev.to, <a href=\"https://dev.to/stimlau/make-pricing-in-2026-free-core-pro-teams-and-credit-costs-lcm\">Make pricing in 2026</a> (Make tier prices read October 8, 2026; affiliate links)</li>\n<li>n8n, <a href=\"https://n8n.io/pricing/\">Pricing</a> (read October 20, 2026)</li>\n<li>n8n Docs, <a href=\"https://docs.n8n.io/build/understand-workflows/understand-executions/\">Understand executions</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/configure-n8n/scaling/manage-execution-data\">Manage execution data</a>, <a href=\"https://docs.n8n.io/deploy/host-n8n/deploy-as-an-oem-integration/prerequisites\">Prerequisites</a> and <a href=\"https://docs.n8n.io/hosting/\">Hosting</a></li>\n<li>n8n, <a href=\"https://raw.githubusercontent.com/n8n-io/n8n/master/LICENSE.md\">Sustainable Use License</a></li>\n<li>n8n Community, <a href=\"https://community.n8n.io/t/92273\">Total execution limits</a> (support answer on pausing at the quota)</li>\n<li>DigitalOcean, <a href=\"https://www.digitalocean.com/pricing/droplets\">Droplet pricing</a></li>\n<li>28creation test, October 20, 2026: n8n 2.44.4 on Node.js 24.22.0, SQLite, a Linux sandbox with 4 vCPUs and 16 GB of memory; 1,000 sample webhook posts twice; local stand-ins for the CRM, sheet and Slack; price script run on Node.js 22</li>\n</ul>\n",
      "summary": "One lead workflow at 10,000 runs a month, billed yearly: $289 on Zapier, $50 on n8n Cloud, about $29 on Make (unconfirmed on make.com). We also ran n8n.",
      "image": "https://28creation.com/og/zapier-make-n8n-cost-one-workflow.jpg",
      "date_published": "2026-10-20T11:59:21.000Z",
      "date_modified": "2026-10-20T11:59:21.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/task-mining-vs-process-mining/",
      "url": "https://28creation.com/articles/task-mining-vs-process-mining/",
      "title": "Task mining vs process mining: logs are cheap, screens are risky",
      "content_html": "<h2>The short read</h2><p><strong>EU regulators called software that logs keystrokes and captures screens disproportionate for employers in 2017.</strong></p><ul><li>Process mining rebuilds a process from timestamped system logs; task mining records clicks, keystrokes and often screenshots on a person&apos;s desktop.</li><li>On a 60-invoice sample log we built, 23 lines of free Python found the rework loop and the slowest wait.</li></ul><h3>What you can do with this</h3><blockquote><p>Export one process as a CSV with case ID, activity and timestamp, then mine it before buying anything. If you need click-level detail, have one volunteer record one task they choose.</p></blockquote><hr><p>Process mining and task mining both promise to show where work goes, but they look at different things. Process mining reads the timestamped records your business systems already keep, such as when an invoice was entered, approved and paid. Task mining records what a person does on their own computer, click by click, and some tools <a href=\"https://docs.uipath.com/task-mining/automation-suite/2024.10/user-guide/unassisted-task-minining-introduction\">take a screenshot on each action</a>.</p>\n<p>For a small team that difference decides what data you need and how much legal risk comes with it. When we checked on October 19, every top result for “task mining vs process mining” came from a vendor of one or both.</p>\n<p>So we built a sample event log and mined it with a free library. We also priced the tools from their own pages and read what European and UK regulators say about recording employees. Prices and documentation were read on October 20, 2026.</p>\n<h2 id=\"process-mining-starts-from-records-you-already-have\">Process mining starts from records you already have</h2>\n<p>Process mining needs an event log: one row per step, with three columns. Microsoft’s documentation for its process mining tool names them as a <a href=\"https://learn.microsoft.com/en-us/power-automate/process-mining-processes-and-data\">case ID, an activity name and a timestamp</a>. The case is the thing the process acts on, such as an order or an invoice. Optional columns add the person or system that did the step, or an amount.</p>\n<p>The catch is getting that log. Microsoft warns that many database tables hold only the current state of a record, not its history, and that “not all events that you’re interested in are logged.” Some systems keep the history in a separate table; Microsoft names the Activities table in Dynamics apps as one example.</p>\n<p>When the log exists, the analysis is cheap. To show how cheap, we built a sample log of 60 invoices with a short Python script, so every number below comes from data we made up on purpose, not from a real company.</p>\n<p>Each invoice is received, entered in the accounting system, matched to a purchase order, approved and paid. About one in five fails the match and goes back to the supplier, a few of them more than once. Small invoices skip approval.</p>\n<h2 id=\"free-code-finds-the-loops-and-the-waits\">Free code finds the loops and the waits</h2>\n<p>We ran the log through <a href=\"https://github.com/process-intelligence-solutions/pm4py\">pm4py</a>, an open-source process mining library for Python, version 2.7.23.11. This is the whole analysis, 23 lines:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pandas </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd, pm4py</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">df </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd.read_csv(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"invoice_log.csv\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">df[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pd.to_datetime(df[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">log </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pm4py.format_dataframe(df, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">case_id</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"case_id\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">activity_key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"activity\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">timestamp_key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"pm4py\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, pm4py.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">__version__</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"># Variants (path through the process, number of invoices)\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> path, n </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pm4py.get_variants(log).items(), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=lambda</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> x: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">x[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]):</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(n, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" &gt; \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(path))</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"># Directly-follows counts\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dfg, starts, ends </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pm4py.discover_dfg(log)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (a, b), n </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(dfg.items(), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=lambda</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> x: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">x[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]):</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">n</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:3d</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">  {</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">a</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> -&gt; </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">b</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"># Median wait before each step (hours)\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">df </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> df.sort_values([</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"case_id\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">df[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"wait_h\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> df.groupby(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"case_id\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].diff().dt.total_seconds() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 3600</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(df.groupby(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"activity\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"wait_h\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].median().round(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).dropna().sort_values(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">ascending</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">False</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).to_string())</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dur </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> df.groupby(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"case_id\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"timestamp\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].agg(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">lambda</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s: (s.max() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> s.min()).total_seconds() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 86400</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">rework </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> df[df.activity </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"Ask supplier for fix\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].case_id.unique()</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"># Throughput time (days), median\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"all invoices:\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(dur.median(), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"| with supplier fix:\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(dur[dur.index.isin(rework)].median(), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">),</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">      \"| without:\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(dur[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">~</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dur.index.isin(rework)].median(), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"| invoices with a fix:\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rework))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">net </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pm4py.discover_petri_net_inductive(log)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"># Inductive-miner model fitness on its own log:\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">round</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pm4py.fitness_token_based_replay(log, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">net)[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"log_fitness\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">3</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span></code></pre>\n<p>Guess how many different paths 60 invoices took through a five-step process. Our sample produced seven. The first lines of the real output:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span>39 Receive invoice &gt; Enter in accounting &gt; Match to PO &gt; Approve &gt; Pay</span></span>\n<span class=\"line\"><span>12 Receive invoice &gt; Enter in accounting &gt; Match to PO &gt; Ask supplier for fix &gt; Enter in accounting &gt; Match to PO &gt; Approve &gt; Pay</span></span>\n<span class=\"line\"><span>5 Receive invoice &gt; Enter in accounting &gt; Match to PO &gt; Pay</span></span></code></pre>\n<p>The other four paths held one invoice each, all with one to three trips back to the supplier. The rest of the output, abridged:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span># Median wait before each step (hours)</span></span>\n<span class=\"line\"><span>Approve                 58.0</span></span>\n<span class=\"line\"><span>Enter in accounting     21.7</span></span>\n<span class=\"line\"><span>Pay                     17.3</span></span>\n<span class=\"line\"><span>Match to PO              2.0</span></span>\n<span class=\"line\"><span>Ask supplier for fix     1.4</span></span>\n<span class=\"line\"><span></span></span>\n<span class=\"line\"><span># Throughput time (days), median</span></span>\n<span class=\"line\"><span>all invoices: 4.7 | with supplier fix: 7.1 | without: 3.7 | invoices with a fix: 16</span></span></code></pre>\n<p>Read it the way you would read a real log. Sixteen invoices went back to a supplier, and those took a median 7.1 days against 3.7 for the rest. The longest median wait sits before approval, at 58 hours. Those are the two questions process mining answers well: where work loops and where it waits.</p>\n<p>One honest limit: these patterns are in the output because our generator put them there. The test shows what the method reports, not how real invoice processes behave. The library also prints a license notice when it loads: pm4py is under the <a href=\"https://github.com/process-intelligence-solutions/pm4py\">AGPL-3.0</a>, and its maintainer, a Fraunhofer FIT spin-off, sells a separate license for closed-source commercial use. Running it on your own data is a different case from building it into software you sell or host for others; ask whoever handles licenses before you do that.</p>\n<h2 id=\"task-mining-sees-the-clicks-a-log-never-records\">Task mining sees the clicks a log never records</h2>\n<p>Process mining could not tell us why entering an invoice takes a person ten minutes. The accounting system logs that the invoice was saved, not the copying and pasting that came before. That gap is what task mining fills.</p>\n<p>To show the difference, we built a second sample: 341 desktop actions for one clerk’s morning, the kind of trail a recorder produces. Again it is made up by our script, not recorded from a person. Thirty invoices each take eight steps, from opening the email to clicking Save, with other work and interruptions mixed in. Then we counted repeated runs of actions:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span>most frequent run of 8 actions, seen 12 times:</span></span>\n<span class=\"line\"><span>   Outlook | open email</span></span>\n<span class=\"line\"><span>   Outlook | open PDF attachment</span></span>\n<span class=\"line\"><span>   PDF viewer | copy invoice number</span></span>\n<span class=\"line\"><span>   Accounting app | click New supplier invoice</span></span>\n<span class=\"line\"><span>   Accounting app | paste invoice number</span></span>\n<span class=\"line\"><span>   PDF viewer | copy amount</span></span>\n<span class=\"line\"><span>   Accounting app | paste amount</span></span>\n<span class=\"line\"><span>   Accounting app | click Save</span></span>\n<span class=\"line\"><span></span></span>\n<span class=\"line\"><span>app switches: 238 | paste actions: 60</span></span></code></pre>\n<p>The routine ran 30 times, but an exact count of all eight steps found it only 12 times, because an interruption anywhere inside it breaks the match. A four-step prefix showed up 22 times.</p>\n<p>That is one reason the commercial tools group similar actions for you (Microsoft’s tool now <a href=\"https://learn.microsoft.com/en-us/power-automate/process-advisor-processes\">does this automatically</a>) and ask for a lot of data. Still, the output names the automation: copy two fields from a PDF into the accounting app, 30 times a morning. Our guide to <a href=\"https://28creation.com/articles/email-data-into-excel/\">getting email data into Excel</a> covers that kind of extraction.</p>\n<h2 id=\"task-mining-needs-far-more-data-and-some-of-it-is-screenshots\">Task mining needs far more data, and some of it is screenshots</h2>\n<p>The two task mining tools we read up on differ in who presses Record. In Microsoft’s Power Automate, a person starts the desktop recorder, <a href=\"https://learn.microsoft.com/en-us/power-automate/process-advisor-processes\">performs the task and selects Finish</a>, and can pause or delete steps. The recorder stores screenshots and text entries, and Microsoft’s documentation shows how to <a href=\"https://learn.microsoft.com/en-us/power-automate/process-advisor-protect\">delete screenshots and remove text</a> before analysis.</p>\n<p>UiPath’s unassisted task mining runs in the background instead. It collects “the screenshot and log data upon each user action,” such as a <a href=\"https://docs.uipath.com/task-mining/automation-suite/2024.10/user-guide/unassisted-task-minining-introduction\">mouse click or keystroke</a>. Its <a href=\"https://docs.uipath.com/task-mining/automation-suite/2024.10/user-guide/best-practices-utm\">best-practice page</a> recommends 2 to 7 users per project and at least 10,000 actions before analysis, with 50,000 ideal. It says an active user produces 200 to 1,000 actions an hour.</p>\n<p>Work that out for a small team. At 1,000 actions an hour, the 10,000-action minimum takes 10 hours of active recording; at 200, it takes 50. Each analysis also uses 5,000 of UiPath’s AI Units. Our 341-action sample is about 3% of the minimum, which is why it took exact matching and a tidy routine to find anything.</p>\n<h2 id=\"prices-start-at-15-a-user-and-jump-to-5000-a-month\">Prices start at $15 a user and jump to $5,000 a month</h2>\n<p>Here is what each option costs, from its own pages on October 20, 2026:</p>\n<table>\n<thead>\n<tr>\n<th>Option</th>\n<th>What it does</th>\n<th>Price shown</th>\n<th>Downsides</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>pm4py (open source)</td>\n<td>Process mining in Python</td>\n<td>Free under <a href=\"https://github.com/process-intelligence-solutions/pm4py\">AGPL-3.0</a>; commercial license sold separately</td>\n<td>Needs someone who writes Python; no task mining</td>\n</tr>\n<tr>\n<td>Power Automate Premium</td>\n<td>Process and task mining</td>\n<td><a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">$15 a user a month</a>, paid yearly, with 50 MB of process mining data per license</td>\n<td>Text/CSV upload <a href=\"https://learn.microsoft.com/en-us/power-automate/process-mining-processes-and-data\">needs OneDrive for Business</a></td>\n</tr>\n<tr>\n<td>Power Automate Process Mining add-on</td>\n<td>100 GB of process mining data</td>\n<td><a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">$5,000 a tenant a month</a>, paid yearly, Premium only</td>\n<td>Priced for large companies</td>\n</tr>\n<tr>\n<td>Celonis</td>\n<td>Process mining</td>\n<td><a href=\"https://www.celonis.com/pricing\">No prices published</a>; a contact form for sales, free training through its Academy</td>\n<td>No way to price it before talking to sales</td>\n</tr>\n<tr>\n<td>UiPath Task Mining</td>\n<td>Background task mining</td>\n<td>Not listed on the documentation pages we read; 5,000 AI Units per analysis</td>\n<td>Screenshots on every action; 10,000-action minimum</td>\n</tr>\n</tbody>\n</table>\n<p>Our 353-event sample log was 16.6 KB as a CSV. A five-person team on Premium holds 50 MB per license, so a log of that kind fits thousands of times over. Size is rarely the small team’s limit. Getting a clean history out of its systems is.</p>\n<h2 id=\"recording-screens-is-where-the-legal-risk-sits\">Recording screens is where the legal risk sits</h2>\n<p>Process mining logs often name the person who did each step, as ours did in its resource column. Under the GDPR that is personal data, so the usual rules apply even to a spreadsheet. Article 88 of the <a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">GDPR</a> also lets EU member states set more specific rules for employee data, with particular regard to “monitoring systems at the work place.” Check your own country’s law and any collective agreement.</p>\n<p>Task mining raises the stakes.</p>\n<p>In 2017 the EU’s Article 29 Working Party, the group of national data protection authorities that preceded today’s EDPB, published its <a href=\"https://ec.europa.eu/newsroom/article29/items/610169\">opinion on data processing at work</a>. It describes software that logs “keystrokes and mouse movements, screen capturing (either randomly or at set intervals), logging of applications used.” Its verdict: the processing is disproportionate, and the employer “is very unlikely to have a legal ground under legitimate interest” for recording keystrokes and mouse movements. The opinion predates the GDPR, but its reasoning on consent carries over: employees “are seldom in a position to freely give, refuse or revoke consent.”</p>\n<p>The UK regulator says much the same in its guidance on <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/data-protection-and-monitoring-workers/\">monitoring workers</a>. Consent “is not usually appropriate in the employment context, due to the imbalance of power.”</p>\n<p>Employers must tell workers about monitoring and “select the least intrusive means” for their purpose, and must carry out a data protection impact assessment before processing likely to cause high risk. The ICO’s page on <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/specific-data-protection-considerations-for-different-ways-or-methods-of-monitoring-workers/\">monitoring methods</a> adds that keystroke monitoring counts as behavioral biometric data where a worker is identifiable. Covert monitoring is unlikely to be justified “in usual circumstances.” The ICO notes that the guidance is under review after the UK’s Data (Use and Access) Act.</p>\n<p>Our reading: a person choosing to record one task, then deleting what they do not want analyzed, sits far from background capture of a whole team. It is still processing of personal data, and this is not legal advice. Background screenshot capture on every click is the case both regulators describe, and a small company should expect to have to justify it in writing.</p>\n<h2 id=\"what-this-means-for-you-mine-the-logs-first\">What this means for you: mine the logs first</h2>\n<p>Most small teams should start with process mining, because it reuses data that already exists and records nobody’s screen. Task mining earns its cost only when you already know which task to study and need the clicks inside it.</p>\n<ol>\n<li><strong>Export one process.</strong> Ask whoever runs your accounting, ticketing or CRM system for a CSV with three columns: case ID, activity and timestamp. Our sample of 60 cases was enough to show the main paths and the loop.</li>\n<li><strong>Drop or replace names.</strong> If you keep the resource column, swap people’s names for roles unless you need them. You will see waits and loops without rating anyone.</li>\n<li><strong>Run the 23 lines above,</strong> or upload the CSV to Power Automate if your company already pays for Premium. Look at two things: the paths that loop and the step with the longest wait before it.</li>\n<li><strong>Study one task, with one volunteer.</strong> If a step looks slow and the log cannot say why, have the person who does it record that task themselves, or keep a <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a>. Tell them what is captured and who sees it. Skip background recording.</li>\n<li><strong>Check what the delay is made of</strong> before you build. In our sample the supplier loop added 3.4 days at the median, and most of that was waiting for the supplier, which no automation on your side shortens.</li>\n</ol>\n<p>More on choosing and building the automation itself is on the <a href=\"https://28creation.com/topics/automation/\">automation hub</a>.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/process-mining-processes-and-data\">Prepare processes and data</a> (process mining data requirements, August 31, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/process-mining-overview\">Overview of process mining in Power Automate</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/task-mining-overview\">Overview of task mining</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/process-advisor-processes\">Prepare processes and recordings</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/process-advisor-protect\">Protect your data</a></li>\n<li>Microsoft, <a href=\"https://www.microsoft.com/en-us/power-platform/products/power-automate/pricing\">Power Automate pricing</a> (read October 20, 2026)</li>\n<li>UiPath documentation, <a href=\"https://docs.uipath.com/task-mining/automation-suite/2024.10/user-guide/unassisted-task-minining-introduction\">Unassisted Task Mining introduction</a> and <a href=\"https://docs.uipath.com/task-mining/automation-suite/2024.10/user-guide/best-practices-utm\">best practices</a> (Automation Suite 2024.10)</li>\n<li>Celonis, <a href=\"https://www.celonis.com/pricing\">pricing page</a> (read October 20, 2026)</li>\n<li>Process Intelligence Solutions, <a href=\"https://github.com/process-intelligence-solutions/pm4py\">pm4py on GitHub</a> (license and maintainer)</li>\n<li>Article 29 Data Protection Working Party, <a href=\"https://ec.europa.eu/newsroom/article29/items/610169\">Opinion 2/2017 on data processing at work, WP249</a> (June 23, 2017)</li>\n<li>Information Commissioner’s Office, <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/data-protection-and-monitoring-workers/\">Data protection and monitoring workers</a> and <a href=\"https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/employment/monitoring-workers/specific-data-protection-considerations-for-different-ways-or-methods-of-monitoring-workers/\">Specific considerations for different monitoring methods</a></li>\n<li><a href=\"https://eur-lex.europa.eu/eli/reg/2016/679/oj\">Regulation (EU) 2016/679, the GDPR</a>, Article 88</li>\n<li>28creation test, October 20, 2026: a 60-invoice event log (353 events) and a 341-action desktop log, both generated by seeded Python scripts in our sandbox, analyzed with pm4py 2.7.23.11 and pandas; the output above is copied from that run</li>\n</ul>\n",
      "summary": "Process mining reads logs your systems already keep; task mining records desktops. We mined sample logs of both kinds and read the prices and GDPR texts.",
      "image": "https://28creation.com/og/task-mining-vs-process-mining.jpg",
      "date_published": "2026-10-20T11:30:14.000Z",
      "date_modified": "2026-10-20T11:30:14.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/gmail-to-google-sheets/",
      "url": "https://28creation.com/articles/gmail-to-google-sheets/",
      "title": "Gmail to Google Sheets automatically, with and without code",
      "content_html": "<h2>The short read</h2><p><strong>In our stand-in test, a label-per-thread script missed 2 of 13 sample form emails and wrote 4 rows wrong.</strong></p><ul><li>Gmail gives scripts whole conversations, so labeling a thread hides or duplicates its next email. We tested outside live Gmail.</li><li>A free 96-line script that records each message ID, Zapier, or Workspace Studio can fill the sheet, each with its own limits.</li></ul><h3>What you can do with this</h3><blockquote><p>Paste the script into Extensions, Apps Script, set the sender and labels, and run setup. Then send your form to yourself twice, once without a phone, and check for two rows.</p></blockquote><hr><p>If you copy details from Gmail into Google Sheets by hand, three tools can do it for you. A free Apps Script runs inside the sheet; Zapier and Google’s Workspace Studio need no code. The script route has one trap. Gmail hands scripts whole conversations, not single emails, so a script that labels each conversation can lose the second email in it.</p>\n<p>We wrote the script below and ran it on 13 sample emails we built, using stand-ins for Gmail and Sheets in Node.js, because our sandbox has no Google account. In that test a simple label-based script missed 2 of the 13 emails and wrote 4 rows wrong. Our script got all 13. Below are the steps for each route, the code, and the limits Google and Zapier document, all read on October 20, 2026.</p>\n<h2 id=\"gmail-gives-scripts-conversations-not-emails\">Gmail gives scripts conversations, not emails</h2>\n<p>Apps Script reaches Gmail through <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-app\"><code>GmailApp.search()</code></a>, which returns threads, the conversations you see in your inbox. Gmail <a href=\"https://support.google.com/mail/answer/5900?hl=en\">groups replies into one conversation</a>, up to 100 emails before it starts a new one.</p>\n<p>A simple script does three things. It searches for unprocessed mail, reads the first message of each thread, and marks the thread with a label such as <code>Processed</code>. That label is the weak spot. Google’s reference says <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-label\"><code>addToThread()</code></a> “adds this label to the given thread”, so the mark sits on the whole conversation.</p>\n<p>So when a second email joins a thread that is already marked, the script has two choices, and both are wrong. If the search skips labeled threads, the new email never reaches the sheet. If the search returns the thread again, the script reads its first message a second time and writes a duplicate. We could not check which of the two real Gmail does, so we tested both.</p>\n<p>The fix is to stop marking threads. Our script reads every message in each matching thread and writes the message’s own ID in column A. Before it writes a row, it checks whether that ID is already in the sheet.</p>\n<h2 id=\"our-test-13-sample-emails-two-scripts-no-google-account\">Our test: 13 sample emails, two scripts, no Google account</h2>\n<p>We built 11 threads with 14 messages: 13 web-form notifications from one sender and 1 reply from a colleague. The names, companies and addresses are invented, on <code>.example</code> domains. The cases:</p>\n<table>\n<thead>\n<tr>\n<th>Emails</th>\n<th>What they test</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1, 6, 8</td>\n<td>Plain <code>Label: value</code> lines; email 6 has a phone starting with <code>+</code>, email 8 one starting with <code>0</code></td>\n</tr>\n<tr>\n<td>2</td>\n<td>A message three lines long</td>\n</tr>\n<tr>\n<td>3</td>\n<td>No phone line</td>\n</tr>\n<tr>\n<td>4</td>\n<td><code>Company Name:</code> above <code>Name:</code></td>\n</tr>\n<tr>\n<td>5</td>\n<td>Each value on the line below its label</td>\n</tr>\n<tr>\n<td>7</td>\n<td>A message field that starts with <code>=HYPERLINK(</code></td>\n</tr>\n<tr>\n<td>9</td>\n<td>The label spelled <code>E-mail:</code></td>\n</tr>\n<tr>\n<td>10, 12, 14</td>\n<td>Three form emails in one conversation, with a colleague’s reply (email 11) between the first two</td>\n</tr>\n<tr>\n<td>13</td>\n<td>A CSV file and a logo attached</td>\n</tr>\n</tbody>\n</table>\n<p>We could not run Apps Script itself. So we ran the code unchanged in Node.js 22 with small stand-ins for <code>GmailApp</code>, <code>SpreadsheetApp</code>, <code>ScriptApp</code> and <code>Utilities</code> that follow Google’s reference. That tests our logic, not Google’s servers. Gmail’s own conversion of HTML email to plain text, its search, and real quotas were not part of it.</p>\n<p>We compared our script with a short label-based script that does those three things. Both ran once, then again after email 14 arrived in the old conversation:</p>\n<table>\n<thead>\n<tr>\n<th>Method</th>\n<th>Emails with a row (of 13)</th>\n<th>Rows fully right</th>\n<th>What went wrong</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Label-based script, labeled thread skipped</td>\n<td>11</td>\n<td>6</td>\n<td>Emails 12 and 14 missed; 4 rows wrong; 1 formula</td>\n</tr>\n<tr>\n<td>Label-based script, labeled thread returned</td>\n<td>11, plus 1 duplicate</td>\n<td>6</td>\n<td>Same, and email 10 written twice</td>\n</tr>\n<tr>\n<td>Our script</td>\n<td>13</td>\n<td>13</td>\n<td>Nothing in this run</td>\n</tr>\n</tbody>\n</table>\n<p>The 4 wrong rows: email 2 lost two lines of its message, email 4 got <code>Dahl CPA</code> as the person’s name, email 5 came out blank, and email 9 had no email address. The formula is email 7. Google’s reference for both <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/sheet\"><code>appendRow()</code></a> and <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/range\"><code>setValues()</code></a> says: “If a cell’s content begins with =, it’s interpreted as a formula.” Anyone who can fill in your form can put a working formula in your sheet.</p>\n<p>One caveat is ours. We wrote the script while looking at these cases, so 13 of 13 is a best case. A sender with a layout we did not plan for will break it until you add the label.</p>\n<h2 id=\"1-pick-the-route-by-account-and-budget\">1. Pick the route by account and budget</h2>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Apps Script</th>\n<th>Zapier</th>\n<th>Workspace Studio</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Cost</td>\n<td>Free with any Google account</td>\n<td><a href=\"https://zapier.com/pricing\">Free</a>: 100 tasks a month; Professional from $19.99 a month billed yearly</td>\n<td>Available in <a href=\"https://support.google.com/workspace-studio/answer/16444479?hl=en\">eligible Workspace editions</a></td>\n</tr>\n<tr>\n<td>Personal Gmail</td>\n<td>Yes</td>\n<td>Not checked by us</td>\n<td>Only through Google Workspace Experiments</td>\n</tr>\n<tr>\n<td>How fast</td>\n<td>Every 1 to 30 minutes</td>\n<td>Checks every 15 minutes on Free, 2 on paid plans</td>\n<td>When the email arrives</td>\n</tr>\n<tr>\n<td>Splits fields</td>\n<td>Yes, with the labels you list</td>\n<td>Needs a third step, so a paid plan</td>\n<td>Yes, with its AI Extract step</td>\n</tr>\n<tr>\n<td>Old mail</td>\n<td>Yes, anything the search finds</td>\n<td>New mail, as the trigger’s name says</td>\n<td>New mail only</td>\n</tr>\n<tr>\n<td>Main limit</td>\n<td>90 minutes of trigger runtime a day on personal accounts</td>\n<td>Tasks per month</td>\n<td>25 flows; a daily run limit</td>\n</tr>\n</tbody>\n</table>\n<p>Our view: start with the script if you have a personal Gmail account or the emails follow a fixed layout. It is free, it backfills old mail, and you can read what it does. Use Workspace Studio if your company is on an eligible Workspace edition and the layouts vary. Use Zapier when the data must also go to tools outside Google.</p>\n<h2 id=\"2-install-the-script-in-the-sheet\">2. Install the script in the sheet</h2>\n<ol>\n<li><strong>Open the target spreadsheet</strong> and choose <a href=\"https://developers.google.com/apps-script/guides/bound\"><strong>Extensions</strong>, then <strong>Apps Script</strong></a>. A script made this way belongs to that sheet.</li>\n<li><strong>Paste the code below</strong> over the empty <code>myFunction</code>, then save.</li>\n<li><strong>Edit <code>CONFIG</code>.</strong> Set <code>from</code> to the sender’s address and <code>query</code> to a <a href=\"https://support.google.com/mail/answer/7190?hl=en\">Gmail search</a>, for example <code>from:noreply@yourform.example newer_than:7d</code>. Under <code>fields</code>, list every label your senders use for each column.</li>\n<li><strong>Select <code>setup</code> in the toolbar and choose Run.</strong> It creates a time-driven trigger and fills the sheet from the mail the search finds. Google asks you to authorize access first: every <code>GmailApp</code> method needs the full <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-app\"><code>https://mail.google.com/</code> scope</a>.</li>\n<li><strong>Check the sheet.</strong> A <code>Leads</code> tab appears with one row per email. The <code>Missing</code> column names any field the script did not find.</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"javascript\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Gmail to Google Sheets: copy labeled fields from matching emails into a sheet.</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Paste into Extensions &gt; Apps Script in the target spreadsheet, edit CONFIG, run setup() once.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> CONFIG</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  from: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'noreply@yourform.example'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,       </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// only messages from this sender; '' for any sender</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  query: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'from:noreply@yourform.example newer_than:7d'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// Gmail search, same syntax as the search box</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  sheet: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Leads'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  fields: {                                </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// column: the labels senders use for it</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    Name:    [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Name'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Full name'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    Email:   [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Email'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'E-mail'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    Phone:   [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Phone'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Mobile'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    Company: [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Company'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Company name'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    Message: [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Message'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Comments'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  },</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  multiline: [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Message'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],                  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// these keep reading until the next label</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  csvSheet: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'CSV import'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,                  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// rows from .csv attachments go here; '' to skip</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">};</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> QUOTE</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2,}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(Forwarded message</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Original Message)</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2,}|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">On </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">.</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> wrote:)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">im</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> setup</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {                         </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// run once: creates the sheets and a 15-minute trigger</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  ScriptApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getProjectTriggers</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">forEach</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">t</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ScriptApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">deleteTrigger</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(t));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  ScriptApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">newTrigger</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'run'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">timeBased</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">everyMinutes</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">15</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">create</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">  run</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> run</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ss</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getActiveSpreadsheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sheet</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> getSheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(ss, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.sheet, [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Message ID'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Received'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'From'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">keys</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.fields), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Missing'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> seen</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Set</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, Math.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">max</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getLastRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getValues</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> String</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(r[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])));</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> rows</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">csvRows</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> csvHeader </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> thread</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> GmailApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">search</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.query, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">100</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> msg</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> thread.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getMessages</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()) {   </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// every message, so a reply in an old thread still counts</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> id</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getId</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">has</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(id) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> !</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFrom</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">includes</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.from.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">())) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// done, or a reply from you</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      seen.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">add</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(id);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> row</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> parse</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getPlainBody</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">());</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> cols</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">keys</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.fields);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      rows.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">push</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">([id, msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getFrom</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">cols.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">c</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> asText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row[c] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                 cols.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">c</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> !</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">row[c]).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">join</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">' '</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)]);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.csvSheet) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> file</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getAttachments</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">({includeInlineImages: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">false</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">})) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">          if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\.</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">csv</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">i</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(file.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">())) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">          const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> text</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> file.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDataAsString</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'UTF-8'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replace</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\uFEFF</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// drop Excel's byte order mark</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">          const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> table</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Utilities.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">parseCsv</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(text);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">          csvHeader </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> table[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">];</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">          table.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">slice</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">forEach</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> csvRows.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">push</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">([id, file.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">r.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(asText)]));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    }</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">  write</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(sheet, rows);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (csvRows.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">write</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getSheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(ss, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.csvSheet, [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Message ID'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'File'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">csvHeader]), csvRows);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  console.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">log</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">`added ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">rows</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">} rows, ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">csvRows</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">} CSV rows`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> parse</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">text</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> parts</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">split</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">QUOTE</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (parts.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> &gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parts[parts.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">];    </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// forwarded or replied: read only the original</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> labels</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {};</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">col</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">ls</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">entries</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.fields)) ls.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">forEach</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">l</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> labels[l.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> alts</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">keys</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(labels).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">sort</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">a</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">b</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> b.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> -</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> a.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">l</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> l.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replace</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[.*+?^${}()|[</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\]\\\\</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">g</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">$&amp;'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">));</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> rx</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> RegExp</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'^</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">s*('</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> alts.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">join</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'|'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ')</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">s*:</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\\\</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">s*(.*)$'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'i'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> row</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {};</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> null</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> raw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> of</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">split</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> line</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> raw.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">replace</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">(&gt;</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">)</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(), </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">hit</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">match</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rx);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (hit) {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> labels[hit[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row) { col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> null</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; }            </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// keep the first value of each field</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">      row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">row[col]) row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line;                      </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// value on the line below its label</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      else</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">CONFIG</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.multiline.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">includes</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(col)) row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ' '</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">      else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> null</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">line </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row[col]) col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> null</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> asText</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">v</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {        </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// the apostrophe keeps =, +, -, @ and leading zeros as typed text</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> String</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(v);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[=+\\-@]</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|^</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">0</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(v) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"'\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> v </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">:</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> v;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> getSheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">ss</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">name</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">header</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  let</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sheet </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ss.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getSheetByName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">sheet) { sheet </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ss.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">insertSheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name); sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">appendRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(header); }</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sheet;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> write</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">sheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">rows</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) {  </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// one call for all rows: much faster than appendRow in a loop</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">rows.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> width</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Math.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">max</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">rows.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  rows </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">concat</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">Array</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(width </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">fill</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)));</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(sheet.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getLastRow</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, rows.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, width).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">setValues</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(rows);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span></code></pre>\n<h3 id=\"how-the-script-handles-the-traps\">How the script handles the traps</h3>\n<p>Labels match only at the start of a line, longest first, so <code>Name</code> doesn’t match inside <code>Company Name</code>. An empty label line takes the next line as its value. A message from anyone other than <code>from</code> is skipped, so your own replies in the thread never become rows.</p>\n<p>The apostrophe in <code>asText</code> is there to keep such values as text. Our stand-in can’t show how a live sheet treats it, so check one row with a <code>+</code> phone number after your first run.</p>\n<p>Rows are written in one <code>setValues()</code> call per run rather than one call per row. Google’s <a href=\"https://developers.google.com/apps-script/guides/support/best-practices\">best-practices page</a> says “Your scripts will run faster if you minimize service calls.” In its example, 10,000 cells written one at a time took about 70 seconds; batched, 1 second.</p>\n<h3 id=\"what-it-printed-on-our-sample-emails\">What it printed on our sample emails</h3>\n<p>This is the real output of three runs in our stand-in: the first run, a second run with no new mail, and a third after email 14 arrived in the old conversation.</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>--- run 1 (setup)</span></span>\n<span class=\"line\"><span>trigger: run every 15 minutes</span></span>\n<span class=\"line\"><span>added 12 rows, 2 CSV rows</span></span>\n<span class=\"line\"><span>--- run 2, no new mail</span></span>\n<span class=\"line\"><span>added 0 rows, 0 CSV rows</span></span>\n<span class=\"line\"><span>--- run 3, one new form email lands in an old conversation</span></span>\n<span class=\"line\"><span>added 1 rows, 0 CSV rows</span></span></code></pre>\n<p>Row 3 came out as <code>Cara Lund | cara@lund.example | | Lund Bakery | No phone, email me. | Phone</code>: a blank phone, and the gap named in <code>Missing</code> instead of hidden. Email 7’s message was written as <code>'=HYPERLINK(...)</code>, as text.</p>\n<h2 id=\"know-googles-quotas-before-the-trigger-runs-all-day\">Know Google’s quotas before the trigger runs all day</h2>\n<p>Apps Script limits are per user, and they reset “24 hours after the first request”, according to Google’s <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">quotas page</a> (updated September 3, 2026):</p>\n<table>\n<thead>\n<tr>\n<th>Limit</th>\n<th>Personal Gmail</th>\n<th>Google Workspace</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Trigger runtime, total</td>\n<td>90 minutes a day</td>\n<td>6 hours a day</td>\n</tr>\n<tr>\n<td>Gmail read and write (not sending)</td>\n<td>20,000 a day</td>\n<td>50,000 a day</td>\n</tr>\n<tr>\n<td>One run</td>\n<td>6 minutes</td>\n<td>6 minutes</td>\n</tr>\n<tr>\n<td>Triggers</td>\n<td>20 per user per script</td>\n<td>20 per user per script</td>\n</tr>\n</tbody>\n</table>\n<p>The runtime line matters most. A 15-minute trigger runs 96 times a day, so on a personal account each run must average under 56 seconds (5,400 seconds divided by 96) to stay inside 90 minutes. If your runs take longer, set the trigger to 30 minutes in <code>setup</code>. <a href=\"https://developers.google.com/apps-script/reference/script/clock-trigger-builder\"><code>everyMinutes()</code></a> accepts only 1, 5, 10, 15 or 30.</p>\n<p>When a quota runs out, the script stops with an error such as “Service using too much computer time for one day”. A failed trigger sends you an email titled <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">“Summary of failures for Apps Script”</a>, and the project’s Executions page shows each run.</p>\n<p>Three more limits come from the code itself:</p>\n<ul>\n<li><strong>100 threads a run.</strong> The script asks for at most 100 threads per search. If more than 100 matching conversations arrive between runs, narrow the query or run more often.</li>\n<li><strong>The trigger runs as you.</strong> Installable triggers <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">“always run under the account of the person who created them”</a>. It reads your mailbox, not a shared one, and it stops if your account goes.</li>\n<li><strong>The sheet has a ceiling.</strong> Google Sheets holds <a href=\"https://support.google.com/drive/answer/37603\">up to 20 million cells</a>. At 9 columns that is about 2.2 million rows, more than most inboxes will fill.</li>\n</ul>\n<h2 id=\"3-pull-csv-attachments-into-their-own-tab\">3. Pull CSV attachments into their own tab</h2>\n<p>Some systems email a CSV report every morning instead of one email per record. The script handles that too. For every attachment whose name ends in <code>.csv</code>, it parses the text with <a href=\"https://developers.google.com/apps-script/reference/utilities/utilities\"><code>Utilities.parseCsv()</code></a>. The rows go to a tab named <code>CSV import</code>, after the message ID and file name.</p>\n<p>Two details came from our test file. A CSV saved by Excel as UTF-8 can start with an invisible byte order mark, which the script strips so the first header doesn’t carry it. And the test file had a quoted value with a comma, <code>\"Lind, Anna\"</code>, which landed in one cell as it should.</p>\n<p>The script skips inline images such as logos by passing <code>includeInlineImages: false</code> to <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-message\"><code>getAttachments()</code></a>; Google’s default includes them. Set <code>csvSheet</code> to <code>''</code> if you don’t want this part.</p>\n<h2 id=\"4-without-code-zapier\">4. Without code: Zapier</h2>\n<p>Zapier’s <a href=\"https://zapier.com/apps/gmail/integrations/google-sheets\">Gmail and Google Sheets page</a> lists the pieces: the trigger <strong>New Email Matching Search</strong> and the action <strong>Create Spreadsheet Row</strong>.</p>\n<ol>\n<li>Create a Zap with Gmail, trigger event <strong>New Email Matching Search</strong>, and enter the same search you would type in Gmail, for example <code>from:noreply@yourform.example</code>.</li>\n<li>Add Google Sheets, action event <strong>Create Spreadsheet Row</strong>. The sheet needs a header row first. Map the email’s date, sender and body to columns.</li>\n<li>Test, then publish.</li>\n</ol>\n<p>That two-step Zap fits the <a href=\"https://zapier.com/pricing\">Free plan</a>: 100 tasks a month, two-step Zaps only, and a check for new mail every 15 minutes. Zapier counts a task as <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261-Tasks-in-Zapier\">“any successful action”</a>, and triggers are free, so each new row costs one task. A form that brings in five leads a day uses about 150 tasks a month and outgrows the free plan.</p>\n<p>Splitting the body into Name, Phone and the rest takes a step in the middle. Zapier says its Formatter steps don’t count as tasks, but a three-step Zap needs a paid plan. Professional starts at $19.99 a month billed yearly for 750 tasks, or $29.99 billed monthly, and checks every 2 minutes. If you only want the whole email body in one cell, the free plan is enough.</p>\n<p>When you run out of tasks without pay-per-task billing, Zapier <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261-Tasks-in-Zapier\">“holds new Zap runs until your billing cycle resets”</a>. They are not lost; you can replay them later. Note that your email text passes through Zapier’s servers, which the Apps Script route avoids.</p>\n<h2 id=\"5-without-code-workspace-studio\">5. Without code: Workspace Studio</h2>\n<p>Google’s own no-code tool, <a href=\"https://support.google.com/workspace-studio/answer/16444479?hl=en\">Workspace Studio</a>, builds a flow from a starter and steps. Its <a href=\"https://support.google.com/workspace-studio/answer/16765661?hl=en\">guide to starters and steps</a> lists the three you need:</p>\n<ol>\n<li><strong>Starter: When I get an email.</strong> Filter on From, Subject has, Has the words or Has attachment. Custom labels can’t be used here; only Gmail’s default labels work.</li>\n<li><strong>Step: Extract.</strong> Pick the email as content and name what to pull out (name, phone, company). This is an AI step, so it copes with layouts that move around.</li>\n<li><strong>Step: Add a row.</strong> Pick the spreadsheet and tab, and map each extracted item to a column.</li>\n</ol>\n<p>It is the only route of the three that reacts when the email arrives instead of checking on a timer. The limits, from Google’s <a href=\"https://support.google.com/workspace-studio/answer/16765942?hl=en\">limits page</a>: at most 25 flows per person, 20 steps per flow, and 25 active flows started from Gmail. A daily run limit covers all your flows together; the page doesn’t give the number. When you hit it, “your active flows stop running” until it resets 24 hours later.</p>\n<p>The catch is access. It works on business, enterprise and education editions whose admin has turned on Gemini, and on personal accounts only through Google Workspace Experiments. It reacts to new mail, so it won’t backfill the emails already in your inbox.</p>\n<p>Extract is AI, too: a wrong value looks exactly like a right one, so spot-check rows against their emails for the first week. We could not test it without a Google account.</p>\n<h2 id=\"check-the-missing-column-each-week\">Check the Missing column each week</h2>\n<p>Whichever route you pick, the failure to fear is the quiet one: a row that arrives half empty, or an email that never arrives at all. Sort the sheet by <code>Missing</code> once a week and open the emails behind any entry. If one sender’s rows start missing fields, its layout changed and you need a new label in <code>CONFIG</code>.</p>\n<p>For the Outlook and Excel side of the same job, see <a href=\"https://28creation.com/articles/email-data-into-excel/\">email data into Excel</a>, which tests the same field-splitting idea on 20 emails. Before you build, our <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a> shows whether this is the task worth automating first. Once the sheet fills, the <a href=\"https://28creation.com/articles/automate-weekly-report/\">weekly report guide</a> can send a summary of it on a schedule, and <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">our checks for scheduled scripts</a> make a silent failure speak up. More guides are under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Start by sending your own form to yourself twice, once with the phone left out, and run <code>setup</code>. Two rows with one <code>Missing</code> entry mean your labels match. Then reply to one of those emails and run it again: no new row should appear.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-app\">GmailApp reference</a> (search, authorization scope; updated April 13, 2026)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-label\">GmailLabel reference</a> (labels apply to threads)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/gmail/gmail-message\">GmailMessage reference</a> (getPlainBody, getAttachments)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/sheet\">Sheet reference</a> and <a href=\"https://developers.google.com/apps-script/reference/spreadsheet/range\">Range reference</a> (appendRow, setValues, formulas)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/utilities/utilities\">Utilities reference</a> (parseCsv)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/reference/script/clock-trigger-builder\">ClockTriggerBuilder reference</a> (everyMinutes)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">Quotas for Google Services</a> (updated September 3, 2026)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">Installable triggers</a> (updated September 3, 2026)</li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/bound\">Container-bound scripts</a></li>\n<li>Google, <a href=\"https://developers.google.com/apps-script/guides/support/best-practices\">Apps Script best practices</a></li>\n<li>Google, <a href=\"https://support.google.com/mail/answer/7190?hl=en\">Gmail search operators</a> and <a href=\"https://support.google.com/mail/answer/5900?hl=en\">conversation view</a></li>\n<li>Google, <a href=\"https://support.google.com/drive/answer/37603\">Files you can store in Google Drive</a> (Sheets cell limit)</li>\n<li>Google, Workspace Studio help: <a href=\"https://support.google.com/workspace-studio/answer/16444479?hl=en\">overview</a>, <a href=\"https://support.google.com/workspace-studio/answer/16765661?hl=en\">starters and steps</a>, <a href=\"https://support.google.com/workspace-studio/answer/16765942?hl=en\">limits</a></li>\n<li>Zapier, <a href=\"https://zapier.com/apps/gmail/integrations/google-sheets\">Gmail and Google Sheets integrations</a>, <a href=\"https://zapier.com/pricing\">pricing</a> and <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261-Tasks-in-Zapier\">Tasks in Zapier</a></li>\n<li>28creation test, October 20, 2026: 14 sample messages in 11 threads, built by us; our script and a label-based script run in Node.js 22 against stand-ins for Google’s services, not in Apps Script or a live Gmail account</li>\n</ul>\n",
      "summary": "A free 96-line Apps Script, Zapier or Workspace Studio can fill a sheet from Gmail. We tested the script on 13 sample emails in a stand-in, not live Gmail.",
      "image": "https://28creation.com/og/gmail-to-google-sheets.jpg",
      "date_published": "2026-10-20T11:29:37.000Z",
      "date_modified": "2026-10-20T11:29:37.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/automation-payback-formula/",
      "url": "https://28creation.com/articles/automation-payback-formula/",
      "title": "Automation payback: a formula that counts the time to fix it",
      "content_html": "<h2>The short read</h2><p><strong>In our worked example, a monthly task looked like a 19-month payback until upkeep was counted; then it never paid.</strong></p><ul><li>Our tested template adds yearly upkeep and failure hours to the usual sum, priced at $36.63 an hour for a median bookkeeping clerk.</li></ul><h3>What you can do with this</h3><blockquote><p>Before building an automation, fill in the template with your numbers and log every fix. If payback reads never, simplify or skip it and keep the build hours.</p></blockquote><hr><p>The simple payback sum for an automation counts minutes saved and hours to build, then stops. It leaves out the hours spent fixing it afterward.</p>\n<p>Those hours are built into the platforms. Power Automate <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">turns off a cloud flow</a> whose trigger or actions fail continuously for 14 days, and Zapier <a href=\"https://help.zapier.com/hc/en-us/articles/8496216132621-Zap-is-not-running\">turns off a Zap</a> that errors 95% of the time it runs. Someone then has to notice it and redo the missed work.</p>\n<p>That matters most for the small tasks people automate first. Much of upkeep is fixed: reconnecting a login or adjusting to a renamed column takes the same hour whether the task runs 12 times a year or 3,000. So upkeep can eat a small saving whole.</p>\n<p>This guide gives the formula, a spreadsheet template we built and tested, and three worked examples priced with US government wage data.</p>\n<h2 id=\"the-formula-has-four-parts-and-upkeep-is-the-one-people-skip\">The formula has four parts, and upkeep is the one people skip</h2>\n<p>Write the yearly saving in hours first, then turn hours into money. Here is the whole model, with each input named:</p>\n<ul>\n<li><strong>Hours saved per year, before upkeep</strong> = runs per year × (minutes by hand − minutes still spent after automating) ÷ 60</li>\n<li><strong>Upkeep hours per year</strong> = fixed upkeep hours + failures per year × minutes per failure ÷ 60</li>\n<li><strong>Net value per year</strong> = (hours saved − upkeep hours) × loaded cost per hour − tool cost per year</li>\n<li><strong>Payback in months</strong> = build hours × loaded cost per hour ÷ net value per year × 12</li>\n</ul>\n<p>“Minutes still spent after automating” is the check you keep doing: opening the sheet and fixing a typo the tool copied in. It is rarely zero. “Fixed upkeep” is the work that comes whether or not anything breaks, such as reconnecting an expired login or re-testing after a tool update. “Minutes per failure” covers noticing the failure, fixing it and doing the missed runs by hand.</p>\n<p>If net value per year is zero or below, the automation never pays back. No build cost is small enough to save it.</p>\n<h2 id=\"price-an-hour-at-what-it-costs-the-employer-not-the-wage\">Price an hour at what it costs the employer, not the wage</h2>\n<p>A saved hour is worth what the employer pays for it, which is more than the hourly wage. The US Bureau of Labor Statistics measures both halves.</p>\n<p>The wage comes from the Occupational Employment and Wage Statistics survey. In the <a href=\"https://www.bls.gov/oes/special-requests/oesm25nat.zip\">May 2025 national file</a>, which we downloaded and read on 20 October 2026, the median hourly wage for bookkeeping, accounting and auditing clerks is $24.36. For general office clerks it is $21.64, and for all occupations $24.51.</p>\n<p>The rest comes from the Employer Costs for Employee Compensation release. In <a href=\"https://www.bls.gov/news.release/ecec.t02.htm\">June 2026</a>, wages and salaries made up 66.5% of total compensation for civilian workers in office and administrative support jobs, where total compensation averaged $38.27 an hour. The other third is benefits. For all <a href=\"https://www.bls.gov/news.release/ecec.nr0.htm\">private industry workers</a> the wage share was 70.0%.</p>\n<p>So the loaded cost per hour is the wage divided by the wage share. For a bookkeeping clerk at the median: $24.36 ÷ 0.665 = $36.63. Treat it as an approximation, because it applies a group average to one occupation’s median.</p>\n<h2 id=\"the-template-with-every-formula-written-out\">The template, with every formula written out</h2>\n<p>Put labels in column A and your numbers in column B. Rows 2 to 12 are inputs; rows 14 to 23 are formulas. We tested them in LibreOffice Calc; they use only IF and MAX, which Excel and Google Sheets also have. You can also <a href=\"https://28creation.com/downloads/automation-payback-template.xlsx\">download the template</a> as an .xlsx file, with our three examples filled in and a fifth column to type over.</p>\n<table>\n<thead>\n<tr>\n<th>Row</th>\n<th>Column A</th>\n<th>Column B</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>2</td>\n<td>Runs per year</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>3</td>\n<td>Minutes per run by hand</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>4</td>\n<td>Minutes per run still spent after automating</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>5</td>\n<td>Hours to build and test</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>6</td>\n<td>Upkeep hours per year</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>7</td>\n<td>Failures per year</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>8</td>\n<td>Minutes per failure (notice, fix, redo by hand)</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>9</td>\n<td>Tool cost per year ($)</td>\n<td>your number</td>\n</tr>\n<tr>\n<td>10</td>\n<td>Hourly wage ($)</td>\n<td>24.36</td>\n</tr>\n<tr>\n<td>11</td>\n<td>Wages as share of total pay cost</td>\n<td>0.665</td>\n</tr>\n<tr>\n<td>12</td>\n<td>Horizon (years)</td>\n<td>3</td>\n</tr>\n<tr>\n<td>14</td>\n<td>Loaded cost per hour ($)</td>\n<td><code>=B10/B11</code></td>\n</tr>\n<tr>\n<td>15</td>\n<td>Hours saved per year, before upkeep</td>\n<td><code>=B2*(B3-B4)/60</code></td>\n</tr>\n<tr>\n<td>16</td>\n<td>Upkeep and failure hours per year</td>\n<td><code>=B6+B7*B8/60</code></td>\n</tr>\n<tr>\n<td>17</td>\n<td>Net hours saved per year</td>\n<td><code>=B15-B16</code></td>\n</tr>\n<tr>\n<td>18</td>\n<td>Net value per year ($)</td>\n<td><code>=B17*B14-B9</code></td>\n</tr>\n<tr>\n<td>19</td>\n<td>Build cost ($)</td>\n<td><code>=B5*B14</code></td>\n</tr>\n<tr>\n<td>20</td>\n<td>Payback (months)</td>\n<td><code>=IF(B18&lt;=0,\"never\",B19/B18*12)</code></td>\n</tr>\n<tr>\n<td>21</td>\n<td>Net over horizon ($)</td>\n<td><code>=B18*B12-B19</code></td>\n</tr>\n<tr>\n<td>22</td>\n<td>Most build hours that still break even</td>\n<td><code>=MAX(0,B18*B12/B14)</code></td>\n</tr>\n<tr>\n<td>23</td>\n<td>Payback if upkeep is ignored (months)</td>\n<td><code>=IF(B15*B14-B9&lt;=0,\"never\",B19/(B15*B14-B9)*12)</code></td>\n</tr>\n</tbody>\n</table>\n<p>Row 22 answers the question people usually ask first: how long can I spend building this? It is the build budget over your horizon, after upkeep. Row 23 is there for comparison. It is the simple sum without upkeep, and the gap between rows 20 and 23 is the cost of ignoring upkeep.</p>\n<p>If someone else builds it at a different rate, scale the build hours. Ten hours from a contractor who costs twice the loaded rate go in row 5 as 20.</p>\n<h2 id=\"three-examples-with-assumed-tasks-and-sourced-prices\">Three examples, with assumed tasks and sourced prices</h2>\n<p>These are examples. The task counts, minutes, build hours and failure rates below are our assumptions, chosen to show how the formula behaves; they are not measurements. The wage, the wage share and the tool price are real and sourced above. All three use a three-year horizon.</p>\n<ul>\n<li><strong>Example A, <a href=\"https://28creation.com/articles/email-data-into-excel/\">invoice emails into a sheet</a>.</strong> 12 a day on 250 working days, so 3,000 runs a year. 4 minutes by hand, 30 seconds of checking after. 30 hours to build, 10 hours of upkeep a year, 24 failures a year at 30 minutes each. It runs on Zapier’s Professional plan, which the <a href=\"https://zapier.com/pricing\">pricing page</a> listed at $19.99 a month billed annually ($239.88 a year) when we read it on 20 October 2026. That plan includes 750 tasks a month. Zapier <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261-How-is-task-usage-measured-in-Zapier\">counts a task</a> for each successful action, not for the trigger, so a two-action Zap uses about 500 a month here.</li>\n<li><strong>Example B, a weekly report.</strong> 52 runs, 45 minutes by hand, 5 minutes of checking after. 12 hours to build, 8 hours of upkeep, 6 failures a year at an hour each. No tool cost, assuming one of the <a href=\"https://28creation.com/articles/automate-weekly-report/\">schedulers that come with Google Workspace or Microsoft 365</a>.</li>\n<li><strong>Example C, a monthly export.</strong> 12 runs, 30 minutes by hand, 5 minutes of checking. 8 hours to build, 4 hours of upkeep, 2 failures at an hour each. No tool cost.</li>\n</ul>\n<h2 id=\"upkeep-barely-moves-the-big-task-and-erases-the-small-one\">Upkeep barely moves the big task and erases the small one</h2>\n<p>Before reading the table, guess which one upkeep hurts most. LibreOffice Calc returned these results from the template:</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Example A</th>\n<th>Example B</th>\n<th>Example C</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Hours saved per year, before upkeep</td>\n<td>175</td>\n<td>34.7</td>\n<td>5</td>\n</tr>\n<tr>\n<td>Upkeep and failure hours per year</td>\n<td>22</td>\n<td>14</td>\n<td>6</td>\n</tr>\n<tr>\n<td>Net hours saved per year</td>\n<td>153</td>\n<td>20.7</td>\n<td>−1</td>\n</tr>\n<tr>\n<td>Net value per year</td>\n<td>$5,364.75</td>\n<td>$757.05</td>\n<td>−$36.63</td>\n</tr>\n<tr>\n<td>Build cost</td>\n<td>$1,098.95</td>\n<td>$439.58</td>\n<td>$293.05</td>\n</tr>\n<tr>\n<td>Payback, upkeep counted</td>\n<td>2.5 months</td>\n<td>7.0 months</td>\n<td>never</td>\n</tr>\n<tr>\n<td>Payback, upkeep ignored</td>\n<td>2.1 months</td>\n<td>4.2 months</td>\n<td>19.2 months</td>\n</tr>\n<tr>\n<td>Net over three years</td>\n<td>$14,995.31</td>\n<td>$1,831.58</td>\n<td>−$402.95</td>\n</tr>\n<tr>\n<td>Most build hours that still break even</td>\n<td>439</td>\n<td>62</td>\n<td>0</td>\n</tr>\n</tbody>\n</table>\n<p>Upkeep took 13% of Example A’s gross saving and barely moved its payback. It took 40% of Example B’s and stretched payback by two thirds. Example C saves 5 hours a year and costs 6 to keep alive. Counted without upkeep, it looks like a 19-month payback; counted honestly, every hour spent building it is lost.</p>\n<p>The sensitivity is steep for mid-sized tasks. Holding everything else in Example B fixed, we varied its total upkeep and failure hours:</p>\n<table>\n<thead>\n<tr>\n<th>Example B upkeep and failure hours per year</th>\n<th>0</th>\n<th>7</th>\n<th>14</th>\n<th>28</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Payback</td>\n<td>4.2 months</td>\n<td>5.2 months</td>\n<td>7.0 months</td>\n<td>21.6 months</td>\n</tr>\n</tbody>\n</table>\n<p>At 34.7 hours of upkeep a year, Example B’s whole saving is gone. Example A could absorb 168 hours before it stopped paying.</p>\n<h2 id=\"how-we-built-and-checked-the-template\">How we built and checked the template</h2>\n<p>We wrote the template with Python’s openpyxl library, with formulas in the cells rather than typed results. LibreOffice 24.2 in headless mode opened and calculated it, then exported the values to CSV. A separate Python function recomputed the same model from the inputs. All 30 results (ten outputs for three examples) matched to within a cent:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"text\"><code><span class=\"line\"><span>Example C</span></span>\n<span class=\"line\"><span>  Net hours saved per year                      LO=          -1 PY=      -1.0 OK</span></span>\n<span class=\"line\"><span>  Payback (months)                              LO=       never PY=     never OK</span></span>\n<span class=\"line\"><span>  Payback if upkeep is ignored (months)         LO=        19.2 PY=      19.2 OK</span></span></code></pre>\n<p>The check catches typing slips in the formulas, not wrong assumptions. If your inputs are guesses, the payback is a guess with decimals.</p>\n<h2 id=\"measure-upkeep-instead-of-guessing-it\">Measure upkeep instead of guessing it</h2>\n<p>The weakest input is upkeep, and the platforms do not total it for you. Power Automate <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">keeps run history for 30 days</a>, so a failure from last quarter is gone unless you wrote it down. Keep a two-column log next to the automation: date, minutes spent. Every fix and every manual redo goes in it.</p>\n<p>Some upkeep is forced on you. Power Automate may turn off a cloud flow that has not been triggered for 90 days, unless the owner has a premium or capacity license. It notifies owners 30 days before, so someone has to read that email. A flow that runs quarterly or less often is the one this rule catches.</p>\n<p>Zapier’s shutoff needs volume too: beside the 95% error rate, it lists a Zap that has run more than 20 times in the past 7 days. Either way, a turned-off automation saves nothing until someone turns it back on.</p>\n<p>For a new automation with no history, borrow numbers from the old ones. If you have none, start with the hours you spent building it and assume a share comes back every year. Then run row 23 against row 20 and see how far apart they land. Our guide to <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">scheduled scripts that fail silently</a> covers making failures visible, which shortens the “notice” part of every failure.</p>\n<h2 id=\"what-the-formula-leaves-out\">What the formula leaves out</h2>\n<p>Money saved is real only if the freed hours go to other paid work. If a clerk’s time would sit idle, the saving is time, not cash. The formula also ignores the cost of an error the automation makes and nobody catches, which can exceed everything else. Our piece on <a href=\"https://28creation.com/articles/boring-tools-win/\">five tests before buying automation</a> covers that risk.</p>\n<p>Time-only charts, such as xkcd’s <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a>, give a build budget from frequency and minutes saved, with no upkeep line. The same comic site’s <a href=\"https://xkcd.com/1319/\">Automation</a> strip draws the missing line as ongoing development. The template puts a number on it, and our <a href=\"https://28creation.com/articles/xkcd-worth-the-time-upkeep/\">recomputed xkcd table</a> applies upkeep to the comic’s own cells.</p>\n<h2 id=\"run-it-on-one-task-this-week\">Run it on one task this week</h2>\n<p>Pick the automation you are about to build, or the one already running that annoys you most. Fill column E of the downloaded <a href=\"https://28creation.com/downloads/automation-payback-template.xlsx\">template</a> with its numbers, and start the upkeep log the same day. If row 20 says “never”, stop or simplify it before spending more hours. If you still need a candidate, the <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally method</a> finds the tasks that repeat most; more guides sit under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>US Bureau of Labor Statistics: <a href=\"https://www.bls.gov/oes/special-requests/oesm25nat.zip\">Occupational Employment and Wage Statistics, May 2025 national data (XLSX)</a>, downloaded 20 October 2026</li>\n<li>US Bureau of Labor Statistics: <a href=\"https://www.bls.gov/news.release/ecec.t02.htm\">Employer Costs for Employee Compensation, June 2026, table 2 (civilian workers by occupational group)</a></li>\n<li>US Bureau of Labor Statistics: <a href=\"https://www.bls.gov/news.release/ecec.nr0.htm\">Employer Costs for Employee Compensation news release, June 2026</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Limits of automated, scheduled and instant flows, Power Automate</a></li>\n<li>Zapier: <a href=\"https://help.zapier.com/hc/en-us/articles/8496216132621-Zap-is-not-running\">Zap is not running</a></li>\n<li>Zapier: <a href=\"https://help.zapier.com/hc/en-us/articles/8496196837261-How-is-task-usage-measured-in-Zapier\">How is task usage measured in Zapier</a></li>\n<li>Zapier: <a href=\"https://zapier.com/pricing\">Plans and pricing</a>, read 20 October 2026</li>\n<li>Randall Munroe: <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a>, xkcd 1205, and <a href=\"https://xkcd.com/1319/\">Automation</a>, xkcd 1319</li>\n</ul>\n",
      "summary": "Much of upkeep does not shrink with a task's size, so it sinks small automations first. A tested template, priced with US wage data.",
      "image": "https://28creation.com/og/automation-payback-formula.jpg",
      "date_published": "2026-10-20T11:29:31.000Z",
      "date_modified": "2026-10-20T11:29:31.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/email-data-into-excel/",
      "url": "https://28creation.com/articles/email-data-into-excel/",
      "title": "Email data into Excel automatically: three methods tested",
      "content_html": "<h2>The short read</h2><p><strong>In our emulation, the tutorial email-to-Excel recipe failed on 8 of 20 test emails and botched 4 more.</strong></p><ul><li>It splits the text on labels like Name:, so a table layout, a renamed label or a missing field breaks it.</li><li>An 85-line Python script and an AI prompt each read all 100 fields, with their own limits.</li></ul><h3>What you can do with this</h3><blockquote><p>Build one flow per sender format with the guarded expression below, add a Missing column, and check it weekly. Rows that break then show up instead of disappearing.</p></blockquote><hr><p>If web forms, order systems or suppliers send you emails with the same fields every time, you can stop copying those fields into Excel by hand. A Power Automate flow, a short script or an AI prompt can fill the table for you. The catch is the method nearly every tutorial teaches: cut the text after a label such as <code>Name:</code>. It works until one sender changes its layout.</p>\n<p>We built 20 sample emails in six layouts that real inboxes get and ran them through three methods. The label-split recipe, as tutorials write it, got 8 of the 20 rows fully right. A script we wrote got all 20, and so did an AI prompt.</p>\n<p>Below are the steps for each, the code, and the limits Microsoft documents. All documentation was read on October 20, 2026.</p>\n<h2 id=\"why-the-label-split-recipe-breaks\">Why the label-split recipe breaks</h2>\n<p>The no-code recipe has three moves. A flow starts when an email arrives, turns the email’s HTML into plain text, then cuts out each field with text functions. In Power Automate that means the Office 365 Outlook trigger and the <strong>Html to text</strong> action from the <a href=\"https://learn.microsoft.com/en-us/connectors/conversionservice/\">Content Conversion connector</a>. Then the <a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\"><code>split()</code> function</a> takes everything after <code>Phone:</code>, and everything before the next line break.</p>\n<p>That logic assumes every email has the label, spelled the same way, with the value on the same line. Microsoft’s documentation shows three ways the assumption fails before any sender changes anything.</p>\n<ul>\n<li><strong>Tables lose their colons.</strong> Html to text puts <a href=\"https://learn.microsoft.com/en-us/connectors/conversionservice/\">3 spaces between data table columns</a>, so a form that sends a two-column table arrives as <code>Phone   +1 213 555 0160</code>, with no <code>Phone:</code> to find.</li>\n<li><strong>Links grow a tail.</strong> A link becomes <code>text[link]</code> unless the text and the address are the same. By that documented rule, an email address sent as a mailto link comes out as <code>dpark@parkdental.example[mailto:dpark@parkdental.example]</code>.</li>\n<li><strong>Long lines are cut.</strong> The converter’s “max line length is 80 characters, afterwards a line break will follow”. A message longer than that loses everything after the 80th character, because the recipe stops at the first line break.</li>\n</ul>\n<p>A missing label is worse. When the delimiter is not in the text, <code>split()</code> <a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\">returns an array with a single element</a>, so the tutorial expression asks for a second item that does not exist. In our emulation that counts as a failed run, and the row never reaches Excel.</p>\n<h2 id=\"our-test-20-emails-six-layouts-three-methods\">Our test: 20 emails, six layouts, three methods</h2>\n<p>We wrote 20 web-form notification emails as standard .eml files with Python’s email library, one lead per email, five fields each (Name, Email, Phone, Company, Message). That makes 100 values to get right. The names and companies are invented, on <code>.example</code> addresses. The layouts:</p>\n<table>\n<thead>\n<tr>\n<th>Emails</th>\n<th>Format</th>\n<th>What it tests</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1 to 8</td>\n<td>Plain text, <code>Name: value</code> on each line</td>\n<td>The happy path; email 5 has a three-line message, email 7 has no phone line</td>\n</tr>\n<tr>\n<td>9 to 12</td>\n<td>HTML table, label in one cell, value in the next</td>\n<td>No colons after conversion; email address as a mailto link</td>\n</tr>\n<tr>\n<td>13 to 15</td>\n<td>Plain text with <code>Company Name:</code> above <code>Name:</code>, and <code>E-mail:</code></td>\n<td>A label inside another label; a spelling variant</td>\n</tr>\n<tr>\n<td>16 and 17</td>\n<td>HTML, label in one paragraph, value in the next</td>\n<td>Value on the line below the label</td>\n</tr>\n<tr>\n<td>18 and 19</td>\n<td>Forwarded by a colleague</td>\n<td>Extra header lines above the form</td>\n</tr>\n<tr>\n<td>20</td>\n<td>Replied to, with the colleague’s signature on top</td>\n<td>A second <code>Phone:</code> and <code>Email:</code> before the real ones</td>\n</tr>\n</tbody>\n</table>\n<p>We could not run Power Automate in our sandbox, which has no Microsoft 365 tenant. So we emulated it. Our Python code converts HTML by the rules Microsoft documents for Html to text, then applies the same split logic with the labels from the first layout. Plain-text emails went to the split step directly.</p>\n<p>We tested it two ways. The tutorial form fails when a label is missing. Our guarded form, shown in step 2, returns an empty cell instead. The script ran for real, and we gave the AI prompt to a Claude model from Anthropic in our sandbox. The results:</p>\n<table>\n<thead>\n<tr>\n<th>Method</th>\n<th>Fields right (of 100)</th>\n<th>Rows fully right (of 20)</th>\n<th>What went wrong</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Label split, tutorial form</td>\n<td>47</td>\n<td>8</td>\n<td>8 runs fail, so those rows never arrive; 4 rows written wrong or blank</td>\n</tr>\n<tr>\n<td>Label split, guarded</td>\n<td>63</td>\n<td>10</td>\n<td>No failures, but 6 rows arrive blank, 3 lose two fields and 1 message is cut</td>\n</tr>\n<tr>\n<td>Our Python script</td>\n<td>100 (99 on the first run)</td>\n<td>20</td>\n<td>One missing space, fixed</td>\n</tr>\n<tr>\n<td>AI prompt</td>\n<td>100</td>\n<td>20</td>\n<td>Nothing in this run</td>\n</tr>\n</tbody>\n</table>\n<h2 id=\"the-tutorial-recipe-broke-in-seven-ways\">The tutorial recipe broke in seven ways</h2>\n<p>The failures follow the layouts exactly. The tutorial split failed on the missing phone line (email 7), on all four tables, and on all three <code>E-mail:</code> emails.</p>\n<p>It read <code>Carter CPA</code> as Ben Carter’s name, because <code>Name:</code> first appears inside <code>Company Name:</code>. It kept only the first line of email 5’s three-line message. It wrote empty rows for the two stacked layouts. And on the reply, it filled the lead’s Email and Phone with the colleague’s signature.</p>\n<p>The guarded version fixed three of those. The missing phone no longer stops the run, and by reading the last <code>Name:</code> and the last <code>Phone:</code>, it got the names and the reply right. Its failures are quieter, which is a different problem. A failed run shows up in the flow’s history; a blank row looks like a lead who skipped the form.</p>\n<p>Two caveats are ours. We wrote the script while looking at all six layouts, so it is a best case: a new seventh layout will break it until you add the label. And we ran the AI prompt once. Twenty emails is a small sample, so 100 out of 100 says the method can work on messy emails, not that it never errs.</p>\n<h2 id=\"1-pick-the-method-by-how-many-formats-you-receive\">1. Pick the method by how many formats you receive</h2>\n<p>The number of different layouts in your inbox decides this more than price does.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Power Automate flow</th>\n<th>Python script</th>\n<th>AI prompt in a flow</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Best for</td>\n<td>One sender, one fixed layout</td>\n<td>A backlog, any mailbox, mixed layouts</td>\n<td>Layouts that change or come from many senders</td>\n</tr>\n<tr>\n<td>Runs</td>\n<td>By itself when mail arrives</td>\n<td>When you run it</td>\n<td>By itself when mail arrives</td>\n</tr>\n<tr>\n<td>Cost</td>\n<td><a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Free license</a> for work and school accounts</td>\n<td>Free; needs Python 3</td>\n<td><a href=\"https://learn.microsoft.com/en-us/ai-builder/credit-management\">Credits per run</a>, billed by tokens</td>\n</tr>\n<tr>\n<td>Breaks when</td>\n<td>A label changes, moves or goes missing</td>\n<td>A sender uses a label you did not list</td>\n<td>Not in our one run; each run costs credits</td>\n</tr>\n<tr>\n<td>Main limit</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">25 MB workbook</a>; don’t edit it while the flow writes</td>\n<td>You save the emails yourself first</td>\n<td>Email text goes to an AI service</td>\n</tr>\n</tbody>\n</table>\n<p>Our view: if one website form sends all the email, build the flow. If three suppliers each send their own layout, build three flows, each filtered to one sender, rather than one flow with clever expressions. Reach for AI when the layouts are too many to keep up with.</p>\n<h2 id=\"2-build-a-flow-for-one-senders-format\">2. Build a flow for one sender’s format</h2>\n<p>This needs a Microsoft 365 work or school account. All three connectors used here, <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook</a>, <a href=\"https://learn.microsoft.com/en-us/connectors/conversionservice/\">Content Conversion</a> and <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Excel Online (Business)</a>, are standard connectors, which the <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">free license</a> covers. That license doesn’t let you share the flow, so it runs in one person’s account.</p>\n<ol>\n<li><strong>Make the table first.</strong> Create a workbook in OneDrive for Business or SharePoint. Type the headers <code>Received</code>, <code>Name</code>, <code>Email</code>, <code>Phone</code>, <code>Company</code>, <code>Message</code>, <code>Missing</code> in row 1, select them and choose <strong>Insert</strong> and then <strong>Table</strong>. The connector’s <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Add a row action needs a table</a>, in an .xlsx or .xlsb file. Avoid headers that are only a number; Microsoft lists them as a cause of unexpected behavior.</li>\n<li><strong>Add the trigger.</strong> Create an automated cloud flow with <strong>When a new email arrives (V3)</strong>. Set <strong>From</strong> to the one sender address, for example <code>noreply@yourform.example</code>, and <strong>Subject Filter</strong> to text that is always in the subject. The trigger reacts to new mail, so it fills the table from today; mail already in the inbox needs step 3.</li>\n<li><strong>Convert the body.</strong> Add <strong>Html to text</strong> and give it the trigger’s <strong>Body</strong>. Its <a href=\"https://learn.microsoft.com/en-us/connectors/conversionservice/\">documented limits</a>: 5 MB of content, and lines broken at 80 characters. Microsoft also says line breaks in the input “will be collapsed into space”. If your sender’s emails are plain text, check the trigger’s <strong>Is HTML</strong> output on a test run; when it is false, skip this step and split the <strong>Body</strong> itself.</li>\n<li><strong>Cut out each field with a guard.</strong> Add a <strong>Compose</strong> action per field, named after the field, with this expression for Phone:</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>if(contains(body('Html_to_text'), 'Phone:'), trim(first(split(last(split(body('Html_to_text'), 'Phone:')), decodeUriComponent('%0A')))), '')</span></span></code></pre>\n<p><code>body('Html_to_text')</code> must match your conversion action’s name.</p>\n<p><a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\"><code>contains()</code></a> checks that the label is there before anything is split, and it is case-sensitive, so <code>phone:</code> will not match. <code>last()</code> takes the text after the last <code>Phone:</code>, which skips a colleague’s signature at the top of a reply. <code>decodeUriComponent('%0A')</code> is a line break. Copy it for each field and change the label.</p>\n<ol start=\"5\">\n<li><strong>Flag the gaps.</strong> Add one more <strong>Compose</strong> named <code>Missing</code> that lists empty fields, for example <code>concat(if(empty(outputs('Name')), 'Name ', ''), if(empty(outputs('Phone')), 'Phone ', ''))</code>, extended to every field.</li>\n<li><strong>Write the row.</strong> Add Excel Online (Business) <strong>Add a row into a table</strong>, pick the file and table, and map each column to its Compose output. Map <code>Received</code> to the trigger’s <strong>Received Time</strong>.</li>\n</ol>\n<h2 id=\"test-the-flow-with-three-emails-before-trusting-it\">Test the flow with three emails before trusting it</h2>\n<p>Send the form to yourself three times: once filled in fully, once with the phone left out, and once with a message longer than 80 characters. Then look at all three rows. The first should be complete, the second should name Phone in the Missing column, and the third shows whether the converter cut your message.</p>\n<h2 id=\"know-the-excel-limits-before-the-table-grows\">Know the Excel limits before the table grows</h2>\n<p>Microsoft’s <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Excel Online (Business) reference</a> lists these, and the first two cause most of the trouble:</p>\n<ul>\n<li>“Simultaneous file modifications made by other connectors, or manual edits are not supported.” If someone keeps the workbook open and edits it while mail arrives, rows can conflict. Keep the flow’s table in its own workbook, and point your reports at it.</li>\n<li>A file “may be locked for an update or delete up to 6 minutes since the last use of the connector”, and a new row can take up to 30 seconds to appear.</li>\n<li>The workbook can be at most 25 MB, and each connection gets 100 calls a minute. The Outlook connector allows <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">300 a minute</a>.</li>\n<li>Heavy formulas in the table can time out the connector, and its retry can then insert the same row more than once. Keep calculations on another sheet, or set the workbook to manual calculation, as Microsoft suggests.</li>\n</ul>\n<h2 id=\"3-run-the-script-on-saved-emails\">3. Run the script on saved emails</h2>\n<p>The script works on any mailbox, including a personal one, the new Outlook and Outlook on the web. It also clears a backlog the flow can’t see.</p>\n<p>Its weakness is the first step: you save the emails yourself. In Outlook on the web, select a message, then <strong>More actions</strong> and <strong>Download</strong>, which saves it as <a href=\"https://support.microsoft.com/en-US/Outlook/mail/save-an-outlook-message-as-a-eml-file-a-pdf-file-or-as-a-draft\">an EML or MSG file</a>. The script reads only .eml files.</p>\n<ol>\n<li>Save the emails as .eml files into one folder, for example <code>Downloads/leads</code>.</li>\n<li>Save the script below as <code>email_to_csv.py</code>. Edit <code>FIELDS</code> so each column lists every label your senders use for it.</li>\n<li>Run <code>python3 email_to_csv.py Downloads/leads leads.csv</code>.</li>\n<li>In Excel, choose <strong>Data</strong>, then <a href=\"https://support.microsoft.com/en-us/office/import-or-export-text-txt-or-csv-files-5250ac4c-663c-47ce-937b-339e391393ba\"><strong>From Text/CSV</strong></a> in the Get &amp; Transform Data group, pick <code>leads.csv</code>, choose <strong>Import</strong>, check the preview and choose <strong>Load</strong>.</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"Pull labelled fields out of a folder of .eml files into one CSV for Excel.</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Usage: python3 email_to_csv.py &lt;folder of .eml files&gt; &lt;output.csv&gt;</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Edit FIELDS for your own forms. Standard library only.</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> csv, re, sys, pathlib, email, email.policy</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> email.utils </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parsedate_to_datetime</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> html.parser </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> HTMLParser</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">FIELDS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {                       </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># column: the labels senders use for it</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"Name\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:    [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Name\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Full name\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Contact name\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"Email\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:   [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Email\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"E-mail\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Email address\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"Phone\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:   [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Phone\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Telephone\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Mobile\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"Company\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Company\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Company name\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Organization\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">    \"Message\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: [</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Message\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Comments\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">],</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">MULTILINE</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Message\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}          </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># these keep reading until the next label</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">QUOTE</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.compile(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">^(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2,}</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Forwarded message</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Original Message</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2,}|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">On </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">.</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> wrote:</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, re.I </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.M)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">STOP</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.compile(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">^(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Sent from </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">|</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">--</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">$)</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, re.I)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">class</span><span style=\"color:#6F42C1;--shiki-dark:#F69D50\"> Text</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#6F42C1;--shiki-dark:#6CB6FF\">HTMLParser</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">):          </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># HTML to lines; a table row becomes \"label: value\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    def</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> __init__</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(self):</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">        super</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">__init__</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(); </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.out, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cell </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [], </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> handle_starttag</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(self, tag, attrs):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"br\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cell </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" \"</span><span style=\"color:#6A737D;--shiki-dark:#768390\">    # line break inside a cell</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"p\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"div\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"br\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"h1\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"h2\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"h3\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"li\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.out.append(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"tr\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> []</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"td\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"th\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">): </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cell </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> handle_endtag</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(self, tag):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"td\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"th\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells.append(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cell.strip())</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> tag </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"tr\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells:</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.out.append(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].rstrip(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\":\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \": \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:]) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">            self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> handle_data</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(self, data):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cells </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.cell </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> data</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">self</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.out.append(data)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> body_text</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(msg):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    part </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg.get_body((</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"plain\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"html\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">return</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_content()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_content_subtype() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"html\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        p </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> Text(); p.feed(text); text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(p.out)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\r\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).replace(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\xa0</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> label_pattern</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">():</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    alts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {re.escape(l.lower()): col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col, ls </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> FIELDS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.items() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> l </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ls}</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rx </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.compile(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">^\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"|\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(alts, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">key</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">reverse</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\")</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">:</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\s</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">(.</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">)$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, re.I)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rx, alts</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> parse</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(text):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    m </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> list</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">QUOTE</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.finditer(text))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> m: text </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text[m[</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].end():]       </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># forwarded or replied: read only the original</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    rx, alts </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> label_pattern()</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    lines </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [l.strip() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> l </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> text.split(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    row, col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {}, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> lines:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        hit </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rx.match(line)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> alts[re.escape(hit.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).lower())]</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row: col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">; </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">continue</span><span style=\"color:#6A737D;--shiki-dark:#768390\">   # keep the first value of each field</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hit.group(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).strip()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> STOP</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.match(line):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row[col]: row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line                       </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># value on the next line</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            elif</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> MULTILINE</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: row[col] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \" \"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            else</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        elif</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> line </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row.get(col): col </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> main</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(src, out):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    files </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(pathlib.Path(src).glob(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"*.eml\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    with</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> open</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(out, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"w\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">newline</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">encoding</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"utf-8-sig\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">as</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> f:   </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># utf-8-sig: Excel reads å, ä, ö</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        w </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> csv.writer(f)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        w.writerow([</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"File\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Received\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">FIELDS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Missing\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> path </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> files:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            msg </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> email.message_from_bytes(path.read_bytes(), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">policy</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">email.policy.default)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            when </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parsedate_to_datetime(msg[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]).strftime(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%Y-%m-</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> %H:%M\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            row </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parse(body_text(msg))</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            missing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> FIELDS</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> row.get(c)]</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            w.writerow([path.name, when, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">*</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row.get(c, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> c </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> FIELDS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">), </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" \"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(missing)])</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">            print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">path.name</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">FIELDS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(missing)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> of </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">FIELDS</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> fields\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\", missing </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">' '</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.join(missing)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> missing </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">))</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"wrote </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">len</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(files)</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> rows to </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">out</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> __name__</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"__main__\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    main(sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])</span></span></code></pre>\n<h3 id=\"what-it-printed-on-our-20-emails\">What it printed on our 20 emails</h3>\n<p>This is its real output, run with Python 3.11 on Linux. Three runs took 0.07 to 0.08 seconds each. Email 7 has no phone line, and the script says so instead of failing:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>01.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>02.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>03.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>04.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>05.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>06.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>07.eml: 4 of 5 fields, missing Phone</span></span>\n<span class=\"line\"><span>08.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>09.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>...</span></span>\n<span class=\"line\"><span>20.eml: 5 of 5 fields</span></span>\n<span class=\"line\"><span>wrote 20 rows to leads.csv</span></span></code></pre>\n<p>The first version got 99 of 100 fields. Email 11 has a two-line message inside a table cell, split by an HTML line break, and the script glued the lines together as <code>April.Can</code>. We added the line that turns a break inside a cell into a space, and the second run got all 100. That is the honest picture of this method: it is exact once you have taught it every quirk, and every new quirk is a fix you make yourself.</p>\n<p>How it handles the traps from the test: labels are matched only at the start of a line, so <code>Name</code> doesn’t match inside <code>Company Name</code>. When a label line is empty, it takes the next line as the value. And if the email contains a forwarding or reply marker, it reads only the text after the last one, which drops the colleague’s signature.</p>\n<h2 id=\"4-use-an-ai-prompt-when-layouts-keep-changing\">4. Use an AI prompt when layouts keep changing</h2>\n<p>The AI route swaps the expressions for one instruction. This is the prompt we gave, once per email, with the raw body as input:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>This is the body of an email notifying us of a website form submission.</span></span>\n<span class=\"line\"><span>It may be HTML, forwarded or replied to. Extract the details of the person</span></span>\n<span class=\"line\"><span>who filled in the form: Name, Email, Phone, Company, Message. Use only text</span></span>\n<span class=\"line\"><span>in the email. If a field is not present, return an empty string. Keep the</span></span>\n<span class=\"line\"><span>message text exactly as written, joining lines with a single space.</span></span>\n<span class=\"line\"><span>Return JSON only.</span></span></code></pre>\n<p>It returned all 100 values correctly, including the tables and the stacked layouts. It also skipped the colleague’s signature in the reply; the prompt only asked for “the person who filled in the form”. That run used a Claude model in our sandbox, not Microsoft’s AI tools, so treat it as evidence for the method, not for any one product.</p>\n<p>Inside Power Automate, the same idea is an AI Builder prompt action, and that is where the cost comes in. Microsoft’s <a href=\"https://learn.microsoft.com/en-us/ai-builder/credit-management\">credit management page</a>, updated October 8, 2026, says prompts always consume credits outside testing, priced on input and output tokens and the model you pick.</p>\n<p>New customers can no longer buy the AI Builder credit add-on and must buy Copilot Credits instead. The 5,000 AI Builder credits a month that came with each Power Automate Premium license are granted only up to November 1, 2026. When an environment runs out of AI Builder credits and has no Copilot Credits to fall back on, AI Builder actions fail with errors such as <code>QuotaExceeded</code>.</p>\n<p>Two more trade-offs. Every email body goes to an AI service, so check your company’s rules before you send customer data there. And a wrong answer from a model looks exactly like a right one, so keep the Missing column and spot-check rows against the emails.</p>\n<h2 id=\"5-skip-power-querys-exchange-connector-for-new-workbooks\">5. Skip Power Query’s Exchange connector for new workbooks</h2>\n<p>Excel can also read a mailbox directly. Power Query’s <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/microsoft-exchange-online\">Microsoft Exchange Online connector</a> imports mail into a workbook, and <a href=\"https://learn.microsoft.com/en-us/powerquery-m/text-betweendelimiters\"><code>Text.BetweenDelimiters</code></a> can cut a field out of the body. That is the same label logic as the flow, with the same weak spots.</p>\n<p>The bigger problem is the date. Microsoft’s page says the connector “will be replaced by new Exchange Mail and Calendar connectors in Excel Power Query by April 2027”. Workbook owners must migrate connections by hand.</p>\n<p>The message center notice, <a href=\"https://mc.merill.net/message/MC1461158\">MC1461158</a>, says existing workbooks “will not automatically transition”, and that the new connectors need Microsoft 365 Apps or Office 2024. Our view: don’t start a new workbook on a connector with a retirement date. If you already have one, list it now so the April deadline doesn’t surprise you.</p>\n<h2 id=\"check-the-missing-column-every-week\">Check the Missing column every week</h2>\n<p>Whichever method you pick, the dangerous failure is a quiet one: a row that arrives blank or half full. Sort the table by the Missing column once a week and open the emails behind any row that has an entry. A cluster of misses from one sender means its layout changed, and your flow or script needs a new label.</p>\n<p>If you are still deciding whether this is the task to automate first, our <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a> will tell you. When the emails carry invoices as attachments, <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments automatically</a> covers that half of the job. The <a href=\"https://28creation.com/articles/automate-weekly-report/\">weekly report guide</a> can turn the finished table into a report on a schedule, and <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">our checks for scheduled scripts</a> make a broken run speak up.</p>\n<p>If your mail is in Gmail and your table in Google Sheets, <a href=\"https://28creation.com/articles/gmail-to-google-sheets/\">our Gmail to Google Sheets guide</a> does the same job with a free Apps Script. More guides are under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Start with the three test emails from the flow section today: full, missing a phone, long message. They will show you which kind of failure your setup has before a real lead falls into it.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/conversionservice/\">Content Conversion connector reference</a> (Html to text behavior and limits)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\">Expression functions reference</a> (split, contains, last, decodeUriComponent; applies to Power Automate)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook connector reference</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Excel Online (Business) connector reference</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Types of Power Automate licenses</a> (updated September 22, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/ai-builder/credit-management\">Licensing and AI Builder credits</a> (updated October 8, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-query/connectors/microsoft-exchange-online\">Microsoft Exchange Online Power Query connector</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/powerquery-m/text-betweendelimiters\">Text.BetweenDelimiters</a></li>\n<li>Microsoft 365 message center MC1461158, Prepare for upcoming changes to the Excel Exchange connector (August 24, 2026), via the <a href=\"https://mc.merill.net/message/MC1461158\">merill.net mirror</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-US/Outlook/mail/save-an-outlook-message-as-a-eml-file-a-pdf-file-or-as-a-draft\">Save an Outlook message as a .eml file, a PDF file, or as a draft</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/import-or-export-text-txt-or-csv-files-5250ac4c-663c-47ce-937b-339e391393ba\">Import or export text (.txt or .csv) files</a></li>\n<li>28creation test, October 20, 2026: 20 sample .eml files built with Python’s email library; the label-split method emulated in Python from Microsoft’s documented rules; the script above run on Python 3.11; the AI prompt run once on a Claude model</li>\n</ul>\n",
      "summary": "The label-split recipe most tutorials teach got 8 of our 20 test emails right when we emulated it. A script and an AI prompt read all 100 fields.",
      "image": "https://28creation.com/og/email-data-into-excel.jpg",
      "date_published": "2026-10-20T09:18:38.000Z",
      "date_modified": "2026-10-20T09:18:38.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/cloudflare-clairic-invoice-extraction/",
      "url": "https://28creation.com/articles/cloudflare-clairic-invoice-extraction/",
      "title": "Cloudflare's Clairic reads invoices for $0.10 per million tokens",
      "content_html": "<h2>The short read</h2><p><strong>Cloudflare&apos;s new Clairic model charges only for input: $0.10 per million tokens, with no output fee.</strong></p><ul><li>Clairic, a preview on Workers AI, pulls typed fields such as totals and due dates out of text and images, each with a confidence score.</li></ul><h3>What you can do with this</h3><blockquote><p>Run 20 of your own invoices through the free tier and compare each field with what you typed by hand. Route anything below your confidence cutoff to a person before it touches payments.</p></blockquote><hr><p>Cloudflare released a preview of <a href=\"https://blog.cloudflare.com/clairic-extraction-model/\">Clairic</a> on October 19, a model that reads documents such as invoices and returns typed values instead of written answers. It costs <a href=\"https://developers.cloudflare.com/workers-ai/models/clairic/\">$0.10 per million input tokens</a>, with no charge for output. If you pay a chatbot-style model to pull totals out of documents, this is a cheaper tool built for that job. <a href=\"https://www.helpnetsecurity.com/2026/10/20/cloudflare-ai-clairic-extraction-model/\">Help Net Security</a> also reported the release.</p>\n<h2 id=\"it-fills-fields-instead-of-writing-text\">It fills fields instead of writing text</h2>\n<p>A request holds the text to read, plus up to four images, and the fields you want back, each with a type: text, number, date, duration, list, estimate or forecast. Clairic answers every field in one pass and generates no tokens, which is why Cloudflare <a href=\"https://blog.cloudflare.com/clairic-extraction-model/\">charges only for input</a> and says replies typically take under a second.</p>\n<p>Each answer carries a confidence score and <code>p_null</code>, the model’s estimate that the value is missing. Missing values come back as null rather than a plausible guess. In Cloudflare’s own example, an email with no tracking number returned null for that field, even though the model’s next best guess was the tax amount, “$52.10”, at 0.42 confidence.</p>\n<p>That matters because invoice extraction with a general language model fails quietly. The output looks right whether or not it is. A confidence number per field gives you a place to draw a line between “post it” and “check it”.</p>\n<h2 id=\"what-it-costs-worked-out-from-cloudflares-numbers\">What it costs, worked out from Cloudflare’s numbers</h2>\n<p>Cloudflare’s <a href=\"https://developers.cloudflare.com/workers-ai/models/clairic/\">model docs</a> run a short sample invoice through six fields. The response reports 655 input tokens: 61 for the invoice text and 472 for the field definitions. So the field list, not the document, drove most of that bill.</p>\n<p>At $0.10 per million tokens, a request like that costs $0.0000655, or about 66 cents for 10,000 of them. The <a href=\"https://developers.cloudflare.com/workers-ai/platform/pricing/\">Workers AI pricing page</a> counts Clairic at 9,091 neurons per million input tokens and gives every account 10,000 neurons a day free, on both the Free and Paid plans. By our arithmetic that is about 1.1 million tokens a day, or about 1,680 requests the size of Cloudflare’s sample. Real invoices are longer than 61 tokens, so expect fewer.</p>\n<p>In a test Cloudflare ran on a damaged-product claim, Clairic extracted the facts in 0.46 seconds against 16.1 seconds for the GLM-5.3 language model. These are the company’s own figures; we found no independent test.</p>\n<h2 id=\"the-limits-are-in-cloudflares-own-docs\">The limits are in Cloudflare’s own docs</h2>\n<ul>\n<li><strong>No PDFs.</strong> Images must be PNG, JPEG or WebP, at most four, 4 MiB each, and remote URLs are not accepted. A scanned PDF has to become text or images first.</li>\n<li><strong>Weak at long sums.</strong> Cloudflare says one- and two-step calculations look reliable, but longer chains and totals across many table rows do not. Its advice: “let code do the math.”</li>\n<li><strong>Well formed is not correct.</strong> Cloudflare’s <a href=\"https://developers.cloudflare.com/workers-ai/concepts/extraction-models/\">extraction guide</a> says typed results “do not guarantee correctness” and asks for human review before “consequential or irreversible actions.”</li>\n<li><strong>A preview with no guarantees.</strong> The model terms say Cloudflare “may cease providing it” without refund. Cloudflare says it intends to release open weights, which would ease that risk, but has not done so yet.</li>\n</ul>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li>If invoices or receipts arrive as text or photos, call <code>@cf/cloudflare/clairic</code> from a Worker or the REST API on the free allocation. An account and an API token are enough.</li>\n<li>Test on 20 invoices you have already keyed in by hand, and count wrong fields per invoice.</li>\n<li>Ask only for the values on the page, then compute totals and due dates in your own code.</li>\n<li>Set a confidence cutoff and send everything below it, and every null, to a person before it reaches payments.</li>\n</ol>\n<p>If you have not yet decided whether invoices are the job worth automating first, our <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally method</a> answers that before you write any code.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://blog.cloudflare.com/clairic-extraction-model/\">Introducing Clairic, a new System One class of extraction models</a>, Cloudflare blog, October 19, 2026</li>\n<li><a href=\"https://developers.cloudflare.com/workers-ai/models/clairic/\">Clairic model reference</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/workers-ai/concepts/extraction-models/\">Extraction models</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/changelog/post/2026-10-19-clairic-workers-ai/\">Clairic changelog entry</a>, Cloudflare, October 19, 2026</li>\n<li><a href=\"https://developers.cloudflare.com/workers-ai/platform/pricing/\">Workers AI pricing</a>, Cloudflare Docs</li>\n<li><a href=\"https://www.helpnetsecurity.com/2026/10/20/cloudflare-ai-clairic-extraction-model/\">Cloudflare bets you’re overpaying an LLM to read your invoices</a>, Help Net Security, October 20, 2026</li>\n</ul>\n",
      "summary": "A preview model returns invoice fields with confidence scores, not generated text. The free tier covers about 1,680 small requests a day; PDFs need converting.",
      "image": "https://28creation.com/og/cloudflare-clairic-invoice-extraction.jpg",
      "date_published": "2026-10-20T05:40:22.000Z",
      "date_modified": "2026-10-20T05:40:22.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "AI",
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/articles/save-outlook-attachments-automatically/",
      "url": "https://28creation.com/articles/save-outlook-attachments-automatically/",
      "title": "Outlook attachments saved automatically: three ways, no VBA",
      "content_html": "<h2>The short read</h2><p><strong>Saving every attachment under its own name lost two of three invoices in our 12-email test.</strong></p><ul><li>Outlook rules can&apos;t save attachments, and the new Outlook for Windows doesn&apos;t support VBA.</li><li>A free cloud flow, Power Automate for desktop or a 56-line Python script can, with an inline filter and unique file names.</li></ul><h3>What you can do with this</h3><blockquote><p>If your office uses Microsoft 365, build the cloud flow with an Is Inline filter and a date-plus-guid file name. New attachments then land in OneDrive without logos or overwrites.</p></blockquote><hr><p>Outlook’s rules can move, flag, forward or delete a message, but <a href=\"https://support.microsoft.com/en-us/office/manage-email-messages-by-using-rules-in-outlook-c24f5dea-9465-4df4-ad17-a50704d66c59\">none of their actions</a> saves an attachment to a folder. The old fix was a VBA macro, and the <a href=\"https://learn.microsoft.com/en-us/microsoft-365-apps/outlook/get-started/state-of-com-add-ins\">new Outlook for Windows doesn’t support VBA</a>. If you still drag invoices out of your inbox by hand, three tools without macros can do it for you.</p>\n<p>They are a Power Automate cloud flow, Power Automate for desktop and a short Python script. None costs extra for an office already on Microsoft 365, and each fits a different version of Outlook. Each also faces the same three traps: signature logos that pile up as junk, files with the same name that collide, and names Windows refuses.</p>\n<p>We built 12 sample emails with those traps in them and ran them through a naive save and through our script. The naive save quietly lost two of three invoices. Below are the results, the exact settings for each tool, and the limits Microsoft documents for them. All limits and license terms were read on October 19, 2026.</p>\n<h2 id=\"why-the-rule-and-the-macro-are-dead-ends\">Why the rule and the macro are dead ends</h2>\n<p>Microsoft’s rules page lists what a rule can do: move a message to a folder, delete it, change its importance, flag it or forward it. Saving an attachment is not on the list. Classic Outlook can run extra rule actions only through <a href=\"https://support.microsoft.com/en-us/office/manage-email-messages-by-using-rules-in-outlook-c24f5dea-9465-4df4-ad17-a50704d66c59\">third-party add-ins</a>, and the page says “Microsoft doesn’t provide custom action add-ins”.</p>\n<p>The answers you will find on old forums use a rule with the “Run a script” action, which calls a VBA macro. That action was switched off by a security update. An answer from Microsoft support staff on its Q&amp;A forum <a href=\"https://learn.microsoft.com/en-us/answers/questions/1332151/how-get-run-script-option-in-outlook-365-mail-rule\">explains how to bring it back</a> with a registry value called <code>EnableUnsafeClientMailRules</code>, and the name tells you what Microsoft thinks of it. It also works only in classic Outlook, because the new Outlook for Windows runs <a href=\"https://learn.microsoft.com/en-us/microsoft-365-apps/outlook/get-started/state-of-com-add-ins\">neither VBA nor COM add-ins</a>.</p>\n<p>Our view: don’t build anything new on that route. It needs a security setting turned off on every PC that runs it, and it dies the day your company moves to the new Outlook.</p>\n<h2 id=\"choose-the-way-that-fits-your-outlook\">Choose the way that fits your Outlook</h2>\n<p>The right tool depends on two facts: which Outlook you use, and whether your mailbox is a work account in Microsoft 365.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Cloud flow</th>\n<th>Power Automate for desktop</th>\n<th>Python script</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Works with</td>\n<td>Microsoft 365 work or school mailbox, any Outlook version</td>\n<td><a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/outlook\">Classic Outlook</a> on a Windows PC; not new Outlook</td>\n<td>Any Outlook that saves a message as an .eml file</td>\n</tr>\n<tr>\n<td>Cost</td>\n<td><a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Free license</a> for work and school accounts</td>\n<td>Free; personal Microsoft accounts <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/requirements\">can sign in</a></td>\n<td>Free; needs Python 3</td>\n</tr>\n<tr>\n<td>Runs</td>\n<td>By itself when mail arrives</td>\n<td>When you start it</td>\n<td>When you start it</td>\n</tr>\n<tr>\n<td>Saves to</td>\n<td>OneDrive</td>\n<td>A folder on the PC</td>\n<td>A folder on the PC</td>\n</tr>\n<tr>\n<td>Best for</td>\n<td>New mail, from now on</td>\n<td>Clearing a full folder in one run</td>\n<td>Mac, new Outlook, the web version</td>\n</tr>\n<tr>\n<td>Main limit</td>\n<td><a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Skips messages over 50 MB</a>, or your admin’s lower limit</td>\n<td>No ARM PCs; scheduling from the cloud needs a premium license</td>\n<td>You save each message by hand first</td>\n</tr>\n</tbody>\n</table>\n<p>Many offices will want two of these: the cloud flow for everything that arrives from today, and a one-off sweep for the backlog already sitting in the inbox.</p>\n<h2 id=\"learn-the-three-traps-from-12-test-emails\">Learn the three traps from 12 test emails</h2>\n<p>We built 12 sample emails in standard email format (.eml) with Python’s email library, covering what a real inbox throws at a “save every attachment” job.</p>\n<p>Three vendors each sent a file called <code>invoice.pdf</code>. Two messages carried a signature logo called <code>image001.png</code>, and one held a pasted screenshot. One file name, <code>Q3: \"final\" report?.pdf</code>, used characters Windows refuses. The set also had a forwarded email with a contract inside, a resent duplicate, a Swedish file name with å, ä and ö, and a file called <code>winmail.dat</code>.</p>\n<p>First we ran a naive script that saves every named attachment under its own name into one folder. It wrote 12 files, and 8 were left at the end.</p>\n<ul>\n<li><strong>Overwrites.</strong> Four writes of <code>invoice.pdf</code> left one file. The invoices from vendors B and C were gone, overwritten without a warning, and the survivor was vendor A’s resent copy.</li>\n<li><strong>Junk.</strong> Both signature logos were saved as <code>image001.png</code>, and the second overwrote the first. Logos with other names would each add a file.</li>\n<li><strong>Bad names.</strong> Our sandbox runs Linux, which accepted <code>Q3: \"final\" report?.pdf</code>. On Windows, the colon, the quotes and the question mark are all <a href=\"https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file\">reserved characters</a>, and the save would fail.</li>\n</ul>\n<p>Then we ran our script, shown in step 3. It saved all 9 real attachments, including the contract inside the forwarded email. It skipped 3 inline images and 1 exact duplicate, and gave every file a name like <code>2026-10-02_vendor-b.com_invoice.pdf</code>. A second run over the same emails saved nothing and reported 10 duplicates, so running it twice does no harm.</p>\n<p>The skipped images include the pasted screenshot. That is the trade-off in the cloud flow and the script below: a filter that drops signature logos also drops images people paste into the body. If your team mails screenshots that matter, keep inline images and delete the logos later.</p>\n<h2 id=\"1-build-a-cloud-flow-that-saves-each-new-attachment\">1. Build a cloud flow that saves each new attachment</h2>\n<p>This needs a Microsoft 365 work or school account. Power Automate’s <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">free license</a> covers it, because both the <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook</a> and <a href=\"https://learn.microsoft.com/en-us/connectors/onedriveforbusiness/\">OneDrive for Business</a> connectors are standard. The OneDrive connector <a href=\"https://learn.microsoft.com/en-us/connectors/onedriveforbusiness/\">needs a work or school account</a>, not a personal one. We had no Microsoft 365 tenant in our sandbox, so the steps below come from Microsoft’s connector reference, not from a flow we ran.</p>\n<ol>\n<li>In Power Automate, create an automated cloud flow with the trigger <strong>When a new email arrives (V3)</strong> from Office 365 Outlook.</li>\n<li>Set <strong>Only with Attachments</strong> to Yes and <strong>Include Attachments</strong> to Yes. Narrow it with <strong>Folder</strong>, <strong>From</strong> or <strong>Subject Filter</strong> if only some mail matters, for example <code>From: invoices@vendor.com</code>.</li>\n<li>Add an <strong>Apply to each</strong> loop over the trigger’s <strong>Attachments</strong>.</li>\n<li>Inside the loop, add a <strong>Condition</strong>: <strong>Is Inline</strong> is equal to <code>false</code>. It does the same job as the logo filter in our script.</li>\n<li>Under <strong>If yes</strong>, add OneDrive for Business <strong>Create file</strong>. Pick a <strong>Folder Path</strong> such as <code>/Email attachments</code>, set <strong>File Content</strong> to the attachment’s <strong>Content</strong>, and set <strong>File Name</strong> to this expression:</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>concat(formatDateTime(utcNow(), 'yyyy-MM-dd'), '_', guid(), '_', item()?['name'])</span></span></code></pre>\n<p>The date is the day the flow ran, in UTC, and it sorts the folder. The <a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\">guid()</a> adds a randomly generated 36-character code, so two vendors’ <code>invoice.pdf</code> files can never land on the same name. It makes names long; that is the price of never losing a file. You don’t need to clean the name yourself, because Create file <a href=\"https://learn.microsoft.com/en-us/connectors/onedriveforbusiness/\">replaces characters OneDrive disallows</a> with <code>_</code>.</p>\n<h2 id=\"watch-the-gaps-microsoft-documents-in-the-cloud-flow\">Watch the gaps Microsoft documents in the cloud flow</h2>\n<p>The <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook connector reference</a> lists these, and each one can leave a gap or a bad file in the folder:</p>\n<ul>\n<li>The trigger “will skip any email that has a total message size greater than the limit put by your Exchange Admin or 50 MB, whichever is less”. It may also skip protected emails.</li>\n<li>Forwarded emails attached to a message, calendar items and bounce reports are “not supported by the connector”. Attached .eml, .msg and .ics files appear only when they arrive inside a .zip. Our script handled the forwarded contract; this flow will not.</li>\n<li>For digitally signed emails, the trigger output “may contain incorrect attachment content”.</li>\n<li>With <strong>Include Attachments</strong> set to Yes, the trigger waits for every download and can time out when many emails with attachments arrive at once. Microsoft’s fix: set it to No and fetch each file with the <strong>Get Attachment (V2)</strong> action instead.</li>\n<li>If your company uses Safe Attachments with Dynamic Delivery, the trigger may run twice, the first time with no attachments. The loop in step 3 does nothing on the empty run, so this one costs you nothing.</li>\n<li>Each connection allows 300 calls a minute to Outlook and <a href=\"https://learn.microsoft.com/en-us/connectors/onedriveforbusiness/\">100 a minute to OneDrive</a>.</li>\n</ul>\n<p>Our view: this is the right default for a Microsoft 365 office. It runs without anyone’s PC switched on, costs nothing extra, and the folder fills as mail arrives. On the free license it <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">can’t be shared</a>, so it lives in one person’s account. Write down whose.</p>\n<h2 id=\"2-sweep-a-whole-folder-with-power-automate-for-desktop\">2. Sweep a whole folder with Power Automate for desktop</h2>\n<p>Power Automate for desktop drives the Outlook app on your PC, so it can work through a whole folder of old messages in one run.</p>\n<p>It needs Windows 10 or 11, Home editions included, and <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/requirements\">doesn’t run on ARM processors</a>. Its Outlook actions work only with <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/outlook\">classic Outlook</a>, not the new one. Installing from the Microsoft Store needs no admin rights; the MSI installer does. We had no Windows PC in this run, so we did not test these steps.</p>\n<ol>\n<li>Create a desktop flow and add <strong>Launch Outlook</strong>.</li>\n<li>Add <strong>Retrieve email messages from Outlook</strong>. Set <strong>Account</strong> to your mailbox name, <strong>Mail folder</strong> to the folder (for a subfolder, the full path, such as <code>Inbox\\Invoices</code>), and <strong>Retrieve</strong> to <code>Unread email messages only</code>. Leave <strong>Mark as read</strong> on.</li>\n<li>In the same action, set <strong>Attachments</strong> to <code>Save attachments</code> and <strong>Save attachments into</strong> to a folder that already exists. A missing folder <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/outlook\">stops the flow</a> with “Directory for saving attachments not found”.</li>\n<li>Add <strong>Process email messages in Outlook</strong> with the operation <code>Move email messages to mail folder</code>, pointing at a folder such as <code>Inbox\\Saved</code>. The next run then can’t pick up the same messages.</li>\n<li>Add <strong>Close Outlook</strong>, then run the flow.</li>\n</ol>\n<p>Microsoft’s reference does not say how this action names files, whether it skips inline images, or what it does when two attachments share a name. Before you point it at a full folder, send yourself two emails that each carry a file called <code>invoice.pdf</code>, plus one with a signature logo, and look at what lands in the folder.</p>\n<p>The other catch is the schedule. With a free personal or work account, the flow runs when you press Run. <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/requirements\">Triggering desktop flows from the cloud</a> needs a premium organization account and Windows Pro or Enterprise. That makes this tool best for the backlog, and the cloud flow better for what arrives next week.</p>\n<h2 id=\"3-run-a-script-on-messages-saved-as-eml-files\">3. Run a script on messages saved as .eml files</h2>\n<p>This route works for the new Outlook, Outlook on the web and any computer with Python 3. It is the one we ran end to end.</p>\n<p>Its weakness is the first step. Outlook on the web saves a message you select through <strong>More actions</strong> and then <strong>Download</strong>, as <a href=\"https://support.microsoft.com/en-US/Outlook/mail/save-an-outlook-message-as-a-eml-file-a-pdf-file-or-as-a-draft\">an EML or MSG file</a> in your Downloads folder. The new Outlook uses <strong>More actions</strong> and then <strong>Save as</strong>; Microsoft’s page doesn’t name the file type, so check that the files end in .eml. The script reads only .eml files.</p>\n<ol>\n<li>Save the messages as .eml files into one folder, for example <code>Downloads/mail</code>.</li>\n<li>Save the 56-line script below as <code>save_attachments.py</code>.</li>\n<li>Run <code>python3 save_attachments.py Downloads/mail Downloads/attachments</code>.</li>\n</ol>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"python\"><code><span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"Save the attachments from a folder of .eml files, skipping inline images.</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Usage: python3 save_attachments.py &lt;folder of .eml files&gt; &lt;output folder&gt;</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Names each file YYYY-MM-DD_sender-domain_original-name, never overwrites,</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">and skips exact duplicates, so it is safe to run again. Standard library only.</span></span>\n<span class=\"line\"><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> sys, re, hashlib, pathlib, email, email.policy</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> email.utils </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parsedate_to_datetime, parseaddr</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">BAD</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> r</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">[</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\\\</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">/:*?\"&lt;&gt;|</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\x00</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">-</span><span style=\"color:#22863A;--shiki-light-font-weight:bold;--shiki-dark:#8DDB8C;--shiki-dark-font-weight:bold\">\\x1f</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">]</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#6A737D;--shiki-dark:#768390\">        # characters Windows will not accept in a file name</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> clean</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(name):</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> re.sub(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">BAD</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"_\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, name).strip(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\" .\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[:</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">150</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"attachment\"</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> is_inline</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(part):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    disp </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_content_disposition()  </span><span style=\"color:#6A737D;--shiki-dark:#768390\"># 'inline', 'attachment' or None</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    return</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> disp </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!=</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"attachment\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (disp </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"inline\"</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Content-ID\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">is</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">def</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> main</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(src, out):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    src, out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> pathlib.Path(src), pathlib.Path(out)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    out.mkdir(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">parents</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">exist_ok</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    seen </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> {hashlib.sha256(p.read_bytes()).hexdigest() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> p </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out.iterdir() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> p.is_file()}</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    saved </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> inline </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> dupes </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">    for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> eml </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> sorted</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(src.glob(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"*.eml\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        msg </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> email.message_from_bytes(eml.read_bytes(), </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">policy</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">email.policy.default)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        day </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parsedate_to_datetime(msg[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]).strftime(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"%Y-%m-</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">%d</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Date\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">else</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"undated\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">        domain </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> parseaddr(msg[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"From\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">].rpartition(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"@\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"unknown\"</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">        for</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> msg.walk():</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.is_multipart():</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                continue</span><span style=\"color:#6A737D;--shiki-dark:#768390\">                        # containers, including forwarded emails</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_filename()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> not</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">and</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_content_maintype() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"text\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                continue</span><span style=\"color:#6A737D;--shiki-dark:#768390\">                        # the message body</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> is_inline(part):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                inline </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">                print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"  skip inline  </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">eml.name</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part[</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Content-ID'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">]</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            data </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> part.get_payload(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">decode</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">True</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> b</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            digest </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> hashlib.sha256(data).hexdigest()</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> digest </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">in</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> seen:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                dupes </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">                print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"  skip dupe    </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">eml.name</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">: </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">name</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">                continue</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            seen.add(digest)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            target </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> out </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> clean(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">day</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">_</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">domain</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">_</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">name </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">or</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'attachment'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            n </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 2</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">            while</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> target.exists():</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                target </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> target.with_name(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">target.stem</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">_</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">n</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">target.suffix</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">                n </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            target.write_bytes(data)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">            saved </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+=</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 1</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">            print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"  saved        </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">target.name</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">    print</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">f</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"saved </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">saved</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, inline skipped </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">inline</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">, duplicates skipped </span><span style=\"color:#005CC5;--shiki-dark:#F47067\">{</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">dupes</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> __name__</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> \"__main__\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">    main(sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">1</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">], sys.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">])</span></span></code></pre>\n<h3 id=\"what-it-printed-on-our-test-set\">What it printed on our test set</h3>\n<p>This is its real output on our 12 sample emails, run with Python 3.11 on Linux in under a tenth of a second (three runs took 0.080 to 0.091 seconds):</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"plaintext\"><code><span class=\"line\"><span>  skip inline  01.eml: image001.png</span></span>\n<span class=\"line\"><span>  saved        2026-10-01_vendor-a.com_invoice.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-02_vendor-b.com_invoice.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-03_vendor-c.com_invoice.pdf</span></span>\n<span class=\"line\"><span>  skip inline  04.eml: image001.png</span></span>\n<span class=\"line\"><span>  saved        2026-10-05_client.com_report.xlsx</span></span>\n<span class=\"line\"><span>  saved        2026-10-06_client.com_Q3_ _final_ report_.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-07_leverantor.se_Faktura_åäö.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-08_client.com_contract.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-12_builder.com_photo.jpg</span></span>\n<span class=\"line\"><span>  skip inline  10.eml: &lt;shot1&gt;</span></span>\n<span class=\"line\"><span>  skip dupe    11.eml: invoice.pdf</span></span>\n<span class=\"line\"><span>  saved        2026-10-15_client.com_winmail.dat</span></span>\n<span class=\"line\"><span>saved 9, inline skipped 3, duplicates skipped 1</span></span></code></pre>\n<h3 id=\"how-it-decides-what-to-skip\">How it decides what to skip</h3>\n<p>Python’s <a href=\"https://docs.python.org/3/library/email.message.html\"><code>get_content_disposition()</code></a> returns <code>inline</code>, <code>attachment</code> or nothing. The script treats a part as an inline image when it is marked inline, or when it carries a Content-ID and is not marked as an attachment, which is how our pasted screenshot arrived. A forwarded email counts as a container in Python’s library, so <a href=\"https://docs.python.org/3/library/email.message.html\"><code>walk()</code> goes inside it</a> and saves the contract it carries. Duplicates are caught by a fingerprint of the file’s content, so a resent invoice is skipped even under a new subject.</p>\n<p>Two limits, both from our run. The script saves a <code>winmail.dat</code> file as it is and does not unpack what it holds. And we ran the script on Linux only. The name cleaning follows Microsoft’s <a href=\"https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file\">list of reserved characters</a>, but we did not run it on a Windows PC.</p>\n<h2 id=\"check-the-folder-after-the-first-week\">Check the folder after the first week</h2>\n<p>Whichever way you pick, a saving job that breaks looks exactly like a quiet week with no invoices. After seven days, count the files in the folder against the emails with attachments in the inbox for the same days. A gap or a broken file points to one of the documented limits: a message over 50 MB, a signed email, an attached email the cloud flow can’t read.</p>\n<p>If you are still deciding whether attachments are the job worth automating first, a <a href=\"https://28creation.com/articles/noticing-what-to-automate/\">one-week tally</a> will tell you. Our guide to <a href=\"https://28creation.com/articles/tiny-automation-in-code/\">scheduled scripts that fail silently</a> has the checks that make a broken job speak up, and the <a href=\"https://28creation.com/articles/automate-weekly-report/\">weekly report guide</a> uses the same free Power Automate license for its schedule. When the data you need sits in the email text rather than an attachment, our <a href=\"https://28creation.com/articles/email-data-into-excel/\">email-to-Excel guide</a> tests three ways to pull it into a table. More guides like this one are under <a href=\"https://28creation.com/topics/automation/\">automation</a>.</p>\n<p>Then put the count in your calendar as a quarterly reminder today, before the folder fills up and nobody remembers what should be in it.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-us/office/manage-email-messages-by-using-rules-in-outlook-c24f5dea-9465-4df4-ad17-a50704d66c59\">Manage email messages by using rules in Outlook</a></li>\n<li>Microsoft Q&amp;A, <a href=\"https://learn.microsoft.com/en-us/answers/questions/1332151/how-get-run-script-option-in-outlook-365-mail-rule\">How to get the “Run a script” option in Outlook 365 mail rules</a> (July 2023)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/microsoft-365-apps/outlook/get-started/state-of-com-add-ins\">Identify COM add-ins in your organization</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook connector reference</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/connectors/onedriveforbusiness/\">OneDrive for Business connector reference</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Types of Power Automate licenses</a> (updated September 22, 2026)</li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/azure/logic-apps/expression-functions-reference\">Expression functions reference</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/outlook\">Outlook actions reference for Power Automate for desktop</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/power-automate/desktop-flows/requirements\">Power Automate for desktop prerequisites and limitations</a></li>\n<li>Microsoft Support, <a href=\"https://support.microsoft.com/en-US/Outlook/mail/save-an-outlook-message-as-a-eml-file-a-pdf-file-or-as-a-draft\">Save an Outlook message as a .eml file, a PDF file, or as a draft</a></li>\n<li>Microsoft Learn, <a href=\"https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file\">Naming files, paths, and namespaces</a></li>\n<li>Python documentation, <a href=\"https://docs.python.org/3/library/email.message.html\">email.message: representing an email message</a></li>\n<li>28creation test, October 19, 2026: 12 sample .eml files built with Python’s email library and run through a naive save and the script above; the output is reproduced in step 3</li>\n</ul>\n",
      "summary": "Outlook rules can't save attachments, and new Outlook has no VBA. Two free Microsoft tools and a 56-line script can, once you guard against three traps.",
      "image": "https://28creation.com/og/save-outlook-attachments-automatically.jpg",
      "date_published": "2026-10-19T12:38:27.000Z",
      "date_modified": "2026-10-19T12:38:27.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/anthropic-agents-wait-on-people/",
      "url": "https://28creation.com/articles/anthropic-agents-wait-on-people/",
      "title": "Anthropic's AI agents wait on people 80% of the time",
      "content_html": "<h2>The short read</h2><p><strong>At Anthropic, interactive Claude sessions spend about 80% of their time idle, waiting for a person to respond.</strong></p><ul><li>An October 16 Anthropic report finds its agents work six times the hours of the staff directing them; reading their output is the bottleneck.</li></ul><h3>What you can do with this</h3><blockquote><p>Before you add agents, estimate the reading load: output tokens times 0.75, divided by 250 words a minute. You see the review hours the agents will cost before you commit to them.</p></blockquote><hr><p>The company that makes Claude has measured its own staff, and the slow part of its AI work is now people. In interactive sessions, Anthropic’s agents spend <a href=\"https://www.anthropic.com/research/employment-at-anthropic\">about 80% of their time</a> waiting for a human to respond. If your team is adding agents, that number is the one to plan around. More agents will not help much if nobody has time to read what they produce.</p>\n<p>The figure comes from “How employment is changing at Anthropic”, a report published on October 16 by four researchers at the Anthropic Institute. It draws on the company’s internal data on hiring, AI use, output, surveys, calendars and Slack, plus 17 employee interviews. Most coverage, including <a href=\"https://openthemagazine.com/business/junior-tech-hires-share-at-anthropic-falls-from-27-to-9-as-ai-reshapes-work\">ANI’s</a>, led with the junior-hiring numbers.</p>\n<p>We think the more useful part for a smaller company is how the work itself changed. Below is what happened, why it happened and what to take from it.</p>\n<h2 id=\"agents-now-out-work-the-people-who-direct-them\">Agents now out-work the people who direct them</h2>\n<p>By August, agents directed by individual employees at Anthropic worked six times as many hours as the people handling them, according to <a href=\"https://www.anthropic.com/research/employment-at-anthropic\">the report</a>. That count covers only employee-directed interactive sessions, and it assumes each person works 40 hours a week.</p>\n<p>Output rose too. Code commits per engineer reached five times their pace of six months earlier.</p>\n<p>The gains reached sales too. One account executive said Claude automations took over around 20% of their previous tasks, such as drafting emails. Another said similar automations let them talk to 30% more of their customers.</p>\n<p>Quality is the obvious worry. The report compared code written by an agent with code written by hand by the same person, in the same month and the same directory. It found no difference in how often the lines were later removed or reverted.</p>\n<p>The researchers add a caution: AI errors may be harder to spot than human ones. So “no difference” here means no difference on the proxies they could measure.</p>\n<h2 id=\"the-bottleneck-moved-to-reading\">The bottleneck moved to reading</h2>\n<p>Faster agents create a new problem. Somebody has to read and answer them. One engineer described facing “an army of Claudes, with every one asking for my attention.” Another said people are “bottlenecking the agents.”</p>\n<p><a href=\"https://www.anthropic.com/research/employment-at-anthropic\">The report</a> puts a size on it. It converted each session’s output into reading time, at 0.75 words per token and 250 words a minute. Even for non-technical teams, reading everything their interactive sessions produced would take over half the work week.</p>\n<p>The load also keeps climbing. After new model releases, staff hand agents harder work: the share of tasks above the old model’s 90th percentile of difficulty rose from 10% to 13%.</p>\n<p>So the 80% idle figure is not a sign of lazy software. It is a queue. The agent has finished its part and is waiting for a person to look, decide and send it on.</p>\n<h2 id=\"managing-the-agents-became-a-job-of-its-own\">Managing the agents became a job of its own</h2>\n<p>People at Anthropic responded by building tools to manage their agents. The report calls it “meta-work”: writing specs for multi-agent runs, setting one agent to review another, and watching pipelines. It grew from 10% of the average employee’s interactive sessions in fall 2025 to 22% in spring 2026.</p>\n<p>Written standing instructions for agents grew from a few hundred in July 2025 to 8,500 a year later. One engineer set up an agent “chief of staff” to sort incoming agent requests. Another wrote a performance review for their team of Claudes.</p>\n<p>The trade-off is plain. That time comes out of the work itself. A fifth of a person’s agent sessions spent on tooling is time they are not spending on customers or code.</p>\n<h2 id=\"meetings-went-up-and-decisions-had-to-be-written-down\">Meetings went up, and decisions had to be written down</h2>\n<p>You might expect agents to cut meetings. The opposite happened. Based on <a href=\"https://www.anthropic.com/research/employment-at-anthropic\">calendar data</a>, the median Anthropic employee spends 20% more time in meetings than a year ago, and technical individual contributors 16% more. The report says part of that is company growth and a shift toward roles that meet more.</p>\n<p>Agents also joined the conversation. They now appear in nearly half of Anthropic’s active public Slack channels. Human threads kept the same number of human replies, with agent replies layered on top.</p>\n<p>That creates a new rule of thumb. Agents only know what is written where they can read it. One employee’s name stayed on a months-old document about company swag, so agents kept sending them requests for t-shirts and mugs long after the job had moved on. The report puts it bluntly: private work in unlogged meetings and direct messages “is invisible, confusing and discouraged in an AI-heavy workplace.”</p>\n<h2 id=\"hiring-shifted-toward-judgment-and-away-from-juniors\">Hiring shifted toward judgment, and away from juniors</h2>\n<p>The hiring data is the starkest part. The junior share of Anthropic’s technical hires <a href=\"https://www.anthropic.com/research/employment-at-anthropic\">fell from 27% to 9%</a>. The junior share of staff in technical departments fell from 21% to 14% since early 2025. In an April 2026 survey, around a third of machine learning managers and a fifth of software engineering managers said they were hiring less because of AI.</p>\n<p>What managers look for changed too. In 2024, interview scorecards mentioned a candidate’s record with AI tools in about 5% of technical assessments and 7% of non-technical ones. By 2026 those rates had tripled and quadrupled. For technical candidates, the biggest increases were comments on autonomy and judgment, while domain expertise shrank as a share.</p>\n<p>The authors name the gap in their own response. Anthropic tracks roles at risk and moves people between teams, but “employment protections for incumbents will not benefit those not hired to begin with.”</p>\n<h2 id=\"read-it-as-one-companys-evidence\">Read it as one company’s evidence</h2>\n<p>This is a vendor studying its own use of its own product, and that deserves weight. Anthropic tripled in size in the last year, so some of its output growth may have come from business growth alone. The authors say so, and add that “tool adoption is not random.” Engineers who took up fully autonomous agents were already on a steeply rising trajectory before they did.</p>\n<p>The report also says many of Anthropic’s practices, including wide stock ownership, “cannot be replicated across the economy.” A ten-person firm is unlikely to have 8,500 standing instructions or a research team counting idle minutes.</p>\n<p>What carries over is the mechanism. Agents produce faster than people can read, and the cost shows up as review time, tooling time and meetings.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision most teams face is whether to add more agents or make better use of the ones they have. Anthropic’s data says the limit will be your people’s attention, so plan for that first.</p>\n<ul>\n<li><strong>Estimate the reading load before you scale.</strong> Use the report’s own conversion: output tokens times 0.75 gives words, and 250 words a minute gives reading time. By that math, one million output tokens is 750,000 words, or 50 hours of reading.</li>\n<li><strong>Budget time for managing agents.</strong> At Anthropic it reached 22% of sessions. If nobody’s week has room for writing instructions and checking pipelines, the agents will mostly wait.</li>\n<li><strong>Write decisions down where agents can read them.</strong> Keep meeting notes and ownership documents current. A stale owner on a document will keep drawing requests, as the swag example shows.</li>\n<li><strong>Decide how you will train juniors.</strong> If agents take the entry-level tasks, write down which work new hires will still learn on before you cut those roles. The report’s authors admit their own measures do not solve this.</li>\n</ul>\n<p>The trade-off cuts both ways. More review keeps quality up but caps how much the agents deliver. Less review frees people but lets more errors through, and the report warns those errors may be harder to see.</p>\n<p>At Anthropic, the agents are already waiting on people 80% of the time. That is the number to check in your own team before you add the next one.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Anthropic Institute: <a href=\"https://www.anthropic.com/research/employment-at-anthropic\">How employment is changing at Anthropic</a> (Wilmers, Catipon, Massenkoff, McCrory, October 16, 2026)</li>\n<li>Open, with ANI: <a href=\"https://openthemagazine.com/business/junior-tech-hires-share-at-anthropic-falls-from-27-to-9-as-ai-reshapes-work\">Junior tech hires’ share at Anthropic falls from 27% to 9% as AI reshapes work</a></li>\n</ul>\n",
      "summary": "Anthropic's study of its own staff finds human attention, not the model, now limits what agents deliver. Plan review time before you add more agents.",
      "image": "https://28creation.com/og/anthropic-agents-wait-on-people.jpg",
      "date_published": "2026-10-19T07:09:55.000Z",
      "date_modified": "2026-10-19T07:09:55.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "AI",
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/notes/workers-static-assets/",
      "url": "https://28creation.com/notes/workers-static-assets/",
      "external_url": "https://developers.cloudflare.com/workers/static-assets/",
      "title": "Cloudflare's reference for serving static files from a Worker",
      "content_html": "<p>Read this next to our <a href=\"https://28creation.com/articles/cloudflare-workers-over-pages/\">news piece on Workers and Pages</a>. Static files upload with the Worker, a request that matches a file is served without running any Worker code, and the page covers single-page app fallbacks, custom 404 pages and <code>run_worker_first</code> for the paths where your code should go first.</p>\n<p>Link: <a href=\"https://developers.cloudflare.com/workers/static-assets/\">developers.cloudflare.com</a></p>",
      "summary": "Read this next to our news piece on Workers and Pages. Static files upload with the Worker, a request that matches a file is served without running any Worker code, and the page covers single-page app fallbacks, custom 404 pages and run_worker_first for the paths where your code…",
      "date_published": "2026-10-18T23:10:00.000Z",
      "date_modified": "2026-10-18T23:10:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/cloudflare-workers-over-pages/",
      "url": "https://28creation.com/articles/cloudflare-workers-over-pages/",
      "title": "Cloudflare sends new projects to Workers; static sites stay free",
      "content_html": "<h2>The short read</h2><p><strong>Cloudflare&apos;s own docs now tell new projects to start on Workers, not Pages.</strong></p><ul><li>Pages keeps running, but Cloudflare&apos;s feature work now goes to Workers.</li><li>Static files stay free on Workers, but a custom domain can only move if Cloudflare runs its DNS.</li></ul><h3>What you can do with this</h3><blockquote><p>If you start a site now, create it as a Worker, not a Pages project. Static hosting stays free, and you skip a migration later.</p></blockquote><hr><p>Cloudflare’s documentation for Pages, its hosting product for static sites and frontends, now tells readers to use something else. The <a href=\"https://developers.cloudflare.com/pages/\">Pages overview</a> says Workers “supports most Pages use cases and offers a broader feature set” and ends: “Start new projects with Workers.” A site on Pages keeps running. New features go to Workers, and moving does not change what static hosting costs.</p>\n<p>Cloudflare set this course in April 2025. It <a href=\"https://blog.cloudflare.com/full-stack-development-on-cloudflare-workers/\">said then</a> that Pages “will continue to be supported”, but “all of our investment, optimizations, and feature work will be dedicated to improving Workers.” Now the advice sits on the Pages docs themselves, and the <a href=\"https://developers.cloudflare.com/pages/migrations/\">migration guides page</a> opens it with “Are you sure you want to use Pages?” Neither page gives Pages an end date.</p>\n<h2 id=\"workers-now-does-what-pages-was-built-for\">Workers now does what Pages was built for</h2>\n<p>A Worker can host a static site on its own. The files deploy with the Worker, and a request that matches a file is <a href=\"https://developers.cloudflare.com/workers/static-assets/\">served without running any Worker code</a>.</p>\n<p>The price is the same. Cloudflare’s <a href=\"https://developers.cloudflare.com/workers/platform/pricing/\">pricing page</a> says requests to static assets “are free and unlimited”. Pages Functions “are charged at the same rate as Workers”, says the <a href=\"https://developers.cloudflare.com/workers/static-assets/migration-guides/migrate-from-pages/\">migration guide</a>. The limits match as well: 20,000 files on the free plan, 100,000 on paid plans and 25 MiB per file, for both <a href=\"https://developers.cloudflare.com/workers/platform/limits/\">Workers</a> and <a href=\"https://developers.cloudflare.com/pages/platform/limits/\">Pages</a>.</p>\n<h2 id=\"workers-has-more-features-and-one-real-gap\">Workers has more features and one real gap</h2>\n<p>The guide’s compatibility matrix marks Cron Triggers, Queue consumers, Workers Logs, rate limiting and gradual deployments as Workers-only. Durable Objects, which on Pages need a separate Worker to bind to, work directly.</p>\n<p>Some things get harder. Branch deploy controls are less configurable on Workers, and file-based routing from a <code>functions/</code> folder has to be compiled with Wrangler first.</p>\n<p>The real gap is DNS. “Unlike Pages, Workers does not support any domain whose nameservers are not managed by Cloudflare,” the guide says. If your domain’s DNS sits with your registrar, moving means moving DNS to Cloudflare first, or staying on Pages.</p>\n<h2 id=\"moving-is-mostly-configuration\">Moving is mostly configuration</h2>\n<p>Most of the work happens in the Wrangler config file. The Pages setting <code>pages_build_output_dir</code> becomes <code>assets.directory</code>. Pages guessed your 404 behavior from <code>index.html</code> and <code>404.html</code>; Workers wants <code>not_found_handling</code> set to <code>single-page-application</code> or <code>404-page</code>, “to prevent accidental misconfiguration”. Locally, <code>wrangler pages dev</code> becomes <code>wrangler dev</code>, and the port moves from 8788 to 8787.</p>\n<p>One change can catch you out. Pages ran Functions before static assets by default, while Workers “will default to serving static assets ahead of your Worker script.” If your code checks logins or logs requests before a file is served, set <code>assets.run_worker_first</code>.</p>\n<p>That setting has a cost. Requests that match it <a href=\"https://developers.cloudflare.com/workers/static-assets/billing-and-limitations/\">“will always invoke your Worker script”</a>, so they are billed. The free plan allows 100,000 requests a day, and past that those requests get a 429 error “instead of falling back to static asset serving.”</p>\n<h2 id=\"cloudflares-astro-post-talks-about-workers\">Cloudflare’s Astro post talks about Workers</h2>\n<p>On January 16, 2026 Cloudflare announced that <a href=\"https://blog.cloudflare.com/astro-joins-cloudflare/\">the company behind Astro is joining it</a>. The post describes local development running in “workerd, the open-source Cloudflare Workers runtime” and never mentions Pages. Cloudflare and <a href=\"https://astro.build/blog/joining-cloudflare/\">Astro</a> both say the framework stays MIT-licensed and keeps supporting other hosts. Disclosure: this site is built with Astro and hosted on Cloudflare.</p>\n<h2 id=\"what-to-do-now\">What to do now</h2>\n<ol>\n<li><strong>Start new projects on Workers.</strong> That is where Cloudflare says its feature work goes.</li>\n<li><strong>Check your nameservers before planning a move.</strong> If Cloudflare does not run your domain’s DNS, your custom domain cannot follow you to Workers.</li>\n<li><strong>Search the repository for <code>_routes.json</code> and middleware.</strong> The guide says to review the request order if you used either. Decide whether you need <code>run_worker_first</code>, and count the billed requests it adds.</li>\n<li><strong>Run the Worker beside the Pages site.</strong> Connect it to Workers Builds, set <code>preview_urls</code> to <code>true</code> and test on a preview link. Then turn off automatic deployments on the Pages project, add your domain to the Worker, and delete the Pages project last with <code>npx wrangler pages project delete</code>.</li>\n<li><strong>Update templates and internal docs</strong> that still say “deploy to Pages”.</li>\n</ol>\n<p>Nothing on either page forces an existing site to move. Move it when you need something on the Workers-only list, and start the next one on Workers.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://developers.cloudflare.com/pages/\">Cloudflare Pages overview</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/pages/migrations/\">Pages migration guides</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/workers/static-assets/migration-guides/migrate-from-pages/\">Migrate from Pages to Workers</a>, Cloudflare Docs</li>\n<li><a href=\"https://blog.cloudflare.com/full-stack-development-on-cloudflare-workers/\">Full-stack development on Workers</a>, Cloudflare blog, April 8, 2025</li>\n<li><a href=\"https://developers.cloudflare.com/workers/static-assets/\">Workers static assets</a> and <a href=\"https://developers.cloudflare.com/workers/static-assets/billing-and-limitations/\">billing and limitations</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/workers/platform/pricing/\">Workers pricing</a> and <a href=\"https://developers.cloudflare.com/workers/platform/limits/\">Workers limits</a>, Cloudflare Docs</li>\n<li><a href=\"https://developers.cloudflare.com/pages/platform/limits/\">Pages limits</a>, Cloudflare Docs</li>\n<li><a href=\"https://blog.cloudflare.com/astro-joins-cloudflare/\">Astro is joining Cloudflare</a>, Cloudflare blog, January 16, 2026</li>\n<li><a href=\"https://astro.build/blog/joining-cloudflare/\">The Astro Technology Company joins Cloudflare</a>, Astro blog, January 16, 2026</li>\n</ul>\n",
      "summary": "Pages stays supported while feature work goes to Workers; static hosting stays free after a move, but only domains on Cloudflare's nameservers can follow.",
      "image": "https://28creation.com/og/cloudflare-workers-over-pages.jpg",
      "date_published": "2026-10-18T23:00:00.000Z",
      "date_modified": "2026-10-18T23:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/boring-tools-win/",
      "url": "https://28creation.com/articles/boring-tools-win/",
      "title": "Boring tools win: five tests before a small team buys automation",
      "content_html": "<h2>The short read</h2><p><strong>Knight Capital lost more than $460 million in about 45 minutes, after 97 warning emails nobody acted on.</strong></p><ul><li>Knight Capital, the Post Office and CrowdStrike failed the same way: automation nobody could read or stop.</li><li>People stop checking systems that are usually right; in one study they caught only 33% of misses.</li></ul><h3>What you can do with this</h3><blockquote><p>Feed any workflow tool a missing field, a duplicate record and a wrong file format before you sign. If it does not stop and tell a named person, keep it away from money.</p></blockquote><hr><p>On August 1, 2012, a trading router at Knight Capital sent millions of unwanted orders into the US stock market in about 45 minutes, and the firm lost more than $460 million. The <a href=\"https://www.sec.gov/litigation/admin/2013/34-70694.pdf\">SEC’s order</a> traced it to nothing exotic: old code nobody had deleted, a manual update nobody checked, and 97 warning emails nobody acted on.</p>\n<p>That pattern runs through two other public failures, and it scales down to a ten-person office. So a small company buying workflow software should pick the predictable, legible tool over the clever one. Legible means a person can see what the tool did and why.</p>\n<p>The automation that does damage changes without asking, hides how it reached its answer, or is right so often that people stop checking. Below are the three cases, the research on how people supervise machines, and five tests to run on any tool before you sign.</p>\n<h2 id=\"knight-capital-lost-460-million-to-code-nobody-could-see\">Knight Capital lost $460 million to code nobody could see</h2>\n<p>Knight’s router, called SMARS, still carried a feature named Power Peg that the firm had <a href=\"https://www.sec.gov/litigation/admin/2013/34-70694.pdf\">stopped using in 2003</a>. The code stayed on the servers, “present and callable.” In 2012, Knight wrote new code for a New York Stock Exchange retail program and reused an old switch: a flag that used to turn Power Peg on.</p>\n<p>Starting July 27, Knight copied the new code onto the router’s eight servers in stages. A technician missed one of them. Knight had no second person review the work, and no written procedure required one.</p>\n<p>On August 1, orders carrying the reused flag reached the eighth server and woke Power Peg up. From 212 small customer orders, it produced more than 4 million executions in 154 stocks. By the time it stopped, Knight had taken on positions worth about $3.5 billion long and $3.15 billion short.</p>\n<p>The warnings were there. From 8:01 a.m., an internal system sent 97 emails flagging an error called “Power Peg disabled” to a group of staff before the market opened at 9:30. Knight had not designed them as alerts, and staff generally did not read them.</p>\n<p>There was no procedure to halt the router when it misbehaved, either. One attempted fix, removing the new code from the seven healthy servers, made things worse. The SEC later fined Knight $12 million on top of its losses.</p>\n<p>None of those failures called for a smarter system. They called for a change someone could verify, a warning that reached a person, and a way to stop. Swap the trading router for an invoice approval flow and the shape is familiar: an old rule nobody deleted, an edit made by one person, error emails landing in a shared inbox.</p>\n<h2 id=\"the-post-office-treated-a-system-nobody-could-inspect-as-proof\">The Post Office treated a system nobody could inspect as proof</h2>\n<p>The UK Post Office began rolling out its Horizon IT system to branches in about September 1999. Bugs in it could show losses in a branch’s accounts that never happened. The <a href=\"https://www.postofficehorizoninquiry.org.uk/volume-1-post-office-horizon-it-inquirys-final-report\">public inquiry</a> found that some Post Office staff “knew or, at the very least, should have known” this, yet the organization “maintained the fiction that its data was always accurate.”</p>\n<p>The people running the branches, called subpostmasters, had almost nothing to argue with. In 2019 the High Court found that their transaction logs were available for only 42 days, later 60, and could be printed on till rolls (<a href=\"https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-post-office-judgment.pdf\">Horizon Issues judgment</a>, paragraph 999). With so little data, they “simply could not identify apparent or alleged discrepancies and shortfalls, their causes,” the judge wrote. He called the Post Office’s stance “the 21st century equivalent of maintaining that the earth is flat.”</p>\n<p>The inquiry’s first report, published in July 2025, estimates that about 1,000 people were prosecuted and convicted on Horizon evidence. Many thousands of people were held responsible, wrongly, for losses that were illusory.</p>\n<p>The scale was national, but the mechanism is ordinary. When the people using a system cannot see why it says what it says, the system wins every argument by default. In a small company the argument is smaller: a customer disputing a late fee, an employee questioning a payroll figure. You still need to show what happened.</p>\n<h2 id=\"systems-that-are-usually-right-teach-people-to-stop-checking\">Systems that are usually right teach people to stop checking</h2>\n<p>Before the third case, a prediction. In a 1993 experiment, people juggled a tracking task and a fuel-management task by hand while an automated system watched engine gauges for them. Sometimes the automation missed a fault. Guess what share of those misses people caught when the automation had been steadily reliable all session.</p>\n<p>It was 33%. When the automation’s reliability varied over time, people caught 82%, according to a 2010 review by <a href=\"https://doi.org/10.1177/0018720810376055\">Parasuraman and Manzey</a>. When watching the automation was their only job, detection was near perfect. The trouble starts when other work competes for attention, which describes most small businesses on most days.</p>\n<p>The review has worse news for anyone counting on training. Complacency showed up in experts as well as novices and “cannot be overcome with simple practice.” Automation bias, following a machine’s advice against other evidence, “cannot be prevented by training or instructions.”</p>\n<p>In one simulator study the review cites, 75% of pilots followed an automated checklist that told them to shut down the wrong engine. With a paper checklist, 25% did.</p>\n<h2 id=\"crowdstrike-shipped-an-update-on-the-strength-of-past-successes\">CrowdStrike shipped an update on the strength of past successes</h2>\n<p>On July 19, 2024, at 04:09 UTC, CrowdStrike sent a configuration update to its Falcon security software for Windows. It pulled the update at 05:27 UTC, according to its <a href=\"https://www.crowdstrike.com/en-us/blog/falcon-content-update-preliminary-post-incident-report/\">preliminary review</a>. Windows machines running version 7.11 or later of the software that were online in those 78 minutes, and received the update, crashed. Microsoft <a href=\"https://blogs.microsoft.com/blog/2024/07/20/helping-our-customers-through-the-crowdstrike-outage/\">estimated</a> that 8.5 million Windows devices were hit, “less than one percent of all Windows machines.”</p>\n<p>The bug was small. A detection template defined 21 input fields, while the code feeding it supplied 20, and the new content was the first to read the 21st (<a href=\"https://www.crowdstrike.com/wp-content/uploads/2024/08/Channel-File-291-Incident-Root-Cause-Analysis-08.06.2024.pdf\">root cause analysis</a>). The tests had used a wildcard in that field, so they never touched it.</p>\n<p>Why did it ship? CrowdStrike’s review names earlier testing, “trust in the checks performed in the Content Validator,” and “previous successful” deployments of the same kind. That is the complacency result, at company scale. A process that had worked several times was trusted to work again.</p>\n<p>The fixes CrowdStrike announced are the boring ones. Its root cause analysis concludes that each such update “should be deployed in a staged rollout,” starting with a canary, meaning a small group gets it first. Customers can now choose “where and when” these updates land.</p>\n<h2 id=\"clever-steps-belong-inside-a-boring-frame\">Clever steps belong inside a boring frame</h2>\n<p>For this piece, clever mostly means an AI step, such as a model that reads incoming email or drafts a reply. Those steps are useful, and by design they carry both risks above. OpenAI’s own <a href=\"https://developers.openai.com/api/docs/guides/advanced-usage\">documentation</a> says its chat completions “are non-deterministic by default (which means model outputs may differ from request to request).” It also warns that determinism may change “due to necessary changes OpenAI makes to model configurations on our end.”</p>\n<p>Read that as a buyer. The same input can produce a different output, so explaining a past run gets harder, which was the Horizon problem. And the behavior can shift on the vendor’s schedule, which was the CrowdStrike problem.</p>\n<p>None of this argues for avoiding smart tools, and predictable has real costs. Rule-based tools need someone to write and maintain the rules. They also handle messy input badly: a customer who buries the order number in the third paragraph of a free-text email will beat most rules.</p>\n<p>Staged rollouts cost speed as well. CrowdStrike’s fast-track content exists so its software can respond to new threats “at operational speed,” and a canary stage slows that down. Some of the time, waiting is the risk.</p>\n<p>Our view is that the trade is worth making for anything that touches money, customers or staff records. Let the clever part do the reading, then hand its result to a plain, logged step a person can inspect. Require a human yes before anything leaves the building. Put the cleverness in a box with clear edges, and make the box easy to open.</p>\n<h2 id=\"what-this-means-for-you-five-tests-before-you-sign\">What this means for you: five tests before you sign</h2>\n<p>It is tempting to spend a trial clicking through features. Spend one afternoon of yours trying to break the tool instead. Each test below maps to one of the cases above.</p>\n<ol>\n<li><strong>Break it on purpose.</strong> Feed one real workflow a missing field, a duplicate record and a file in the wrong format. A pass means the tool stops and tells a named person, somewhere that person actually looks. Knight’s 97 emails show what a fail looks like.</li>\n<li><strong>Find the stop button.</strong> Ask how to pause every run in one action, what happens to runs already in flight, and whether resuming processes anything twice. Knight had no halt procedure, and its attempted fix made the damage worse.</li>\n<li><strong>Have a stranger explain a run.</strong> Pick a run from last month. Ask a colleague who was not part of the setup to say what came in, which rule fired and what went out, using only the tool’s own history. Then ask how long that history is kept. Horizon’s branch logs lasted 42 days at first.</li>\n<li><strong>Ask who changes it, and when.</strong> Find out what the vendor can change without you, whether you get release notes, and whether you can try an update on a copy before it reaches live work. For any AI step, ask which model version runs and whether you can pin it.</li>\n<li><strong>Clean out the old.</strong> Replace one rule and confirm the old one is deleted rather than switched off. Then ask for a list of every active automation. Power Peg sat unused for nine years before it woke up.</li>\n</ol>\n<p>A tool that passes all five has earned a look at its feature list. One that fails any of the first three should not touch money or customers, however good the demo was.</p>\n<p>Knight’s warnings arrived between 8:01 and 9:30 that morning, 97 of them, each naming the broken feature. The cheapest test on this list asks whether your tool would have put the first one in front of someone who reads.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>U.S. Securities and Exchange Commission, <a href=\"https://www.sec.gov/litigation/admin/2013/34-70694.pdf\">Order in the Matter of Knight Capital Americas LLC</a>, Release No. 70694, October 16, 2013.</li>\n<li>Post Office Horizon IT Inquiry, <a href=\"https://www.postofficehorizoninquiry.org.uk/volume-1-post-office-horizon-it-inquirys-final-report\">Volume 1 of the final report</a>, July 2025.</li>\n<li>High Court of England and Wales, <a href=\"https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-post-office-judgment.pdf\">Bates and Others v Post Office Ltd, Judgment No. 6, “Horizon Issues”</a>, [2019] EWHC 3408 (QB), December 16, 2019.</li>\n<li>R. Parasuraman and D. H. Manzey, <a href=\"https://doi.org/10.1177/0018720810376055\">Complacency and Bias in Human Use of Automation: An Attentional Integration</a>, Human Factors 52(3), 2010 (<a href=\"https://depositonce.tu-berlin.de/items/0c787d91-6021-480b-a333-009e0d1d0092/full\">free copy</a>).</li>\n<li>CrowdStrike, <a href=\"https://www.crowdstrike.com/en-us/blog/falcon-content-update-preliminary-post-incident-report/\">Preliminary Post Incident Review</a>, July 24, 2024.</li>\n<li>CrowdStrike, <a href=\"https://www.crowdstrike.com/wp-content/uploads/2024/08/Channel-File-291-Incident-Root-Cause-Analysis-08.06.2024.pdf\">External Technical Root Cause Analysis, Channel File 291</a>, August 6, 2024.</li>\n<li>David Weston, Microsoft, <a href=\"https://blogs.microsoft.com/blog/2024/07/20/helping-our-customers-through-the-crowdstrike-outage/\">Helping our customers through the CrowdStrike outage</a>, July 20, 2024.</li>\n<li>OpenAI, <a href=\"https://developers.openai.com/api/docs/guides/advanced-usage\">API documentation: Advanced usage, reproducible outputs</a>.</li>\n</ul>\n",
      "summary": "Knight Capital, the Post Office and CrowdStrike show the cost of automation nobody can read or stop, and five tests can expose it before a contract is signed.",
      "image": "https://28creation.com/og/boring-tools-win.jpg",
      "date_published": "2026-10-18T22:30:00.000Z",
      "date_modified": "2026-10-18T22:30:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Work software",
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/articles/choosing-ai-assistant/",
      "url": "https://28creation.com/articles/choosing-ai-assistant/",
      "title": "Choosing an AI assistant: 6 vendor questions and a 2-week trial",
      "content_html": "<h2>The short read</h2><p><strong>In Microsoft&apos;s survey, 78% of people who use AI at work bring their own tools; at smaller firms, 80%.</strong></p><ul><li>One vendor can write very different rules for personal and business accounts, from training on chats to how long they are kept.</li><li>The guide gives six questions to put to vendors in writing, and a two-week trial on real work.</li></ul><h3>What you can do with this</h3><blockquote><p>Send each shortlisted vendor the six questions in writing, starting with whether your data trains their models. You get answers tied to a contract or a setting, not a sales-call promise.</p></blockquote><hr><p>In Microsoft’s 2024 <a href=\"https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part\">Work Trend Index</a>, 78% of people who use AI at work said they bring their own AI tools. At small and medium-sized companies, the share was 80%. That matters because the same vendor can write very different rules for a personal account and a business one.</p>\n<p>Anthropic is a clear case. Its consumer plans <a href=\"https://www.anthropic.com/news/updates-to-our-consumer-terms\">keep chats for five years</a> if the user allows them to be used for training. Its business plans <a href=\"https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training\">don’t use inputs or outputs for training</a> by default.</p>\n<p>So the first decision is which terms your team’s words fall under. Features come second. This guide gives six questions to put to a vendor in writing. For each, it shows what a good answer looks like, quoting four vendors’ published policies.</p>\n<p>Then it lays out a two-week trial on real work and the five numbers worth measuring before anyone signs.</p>\n<h2 id=\"1-list-the-work-before-you-look-at-products\">1. List the work before you look at products</h2>\n<p>Write down the five to ten tasks your team actually wants help with, in tedious detail. “Writing” is useless. “First drafts of replies to support emails” is useful, and so is “a summary of a 40-message client thread before a call”.</p>\n<p>Rank the list by how often each task happens. The top three become your trial. A tool that is brilliant at something you do twice a year is a worse buy than one that is merely decent at something you do every morning.</p>\n<h2 id=\"2-ask-the-team-what-it-already-uses\">2. Ask the team what it already uses</h2>\n<p>Before you pick anything, find out which tools are already in use, and on which accounts. Ask without blame. Microsoft’s survey also found that 52% of people who use AI at work are <a href=\"https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part\">reluctant to admit using it</a> for their most important tasks. A survey that sounds like an audit will get polite answers.</p>\n<p>The answers tell you where your data sits today. Google’s <a href=\"https://support.google.com/gemini/answer/13594961\">privacy hub for the Gemini app</a> says human reviewers, including trained reviewers from its service providers, review some chats. Reviewed chats are kept for up to three years, and the page asks users not to enter confidential information they wouldn’t want a reviewer to see.</p>\n<p>The same page says work and school accounts may have different terms. That difference is what the next three steps check.</p>\n<h2 id=\"3-ask-whether-your-data-trains-the-model-and-who-reads-it\">3. Ask whether your data trains the model, and who reads it</h2>\n<p>Put the six questions below to every vendor in writing. Expect answers that point to a contract, a policy page or an admin setting. A promise on a sales call is worth what you paid for it.</p>\n<p><strong>Question 1: Is our input or output used to train or improve your models, and is that the default?</strong> A good answer is a plain no, in writing, with the exceptions named. As of October 2026, the large vendors’ business offerings put it like this:</p>\n<ul>\n<li><strong>Microsoft Copilot</strong> (the business product formerly called Microsoft 365 Copilot): prompts, responses and data accessed through Microsoft Graph <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">aren’t used to train foundation LLMs</a>.</li>\n<li><strong>Google Workspace:</strong> it <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">doesn’t use customer data for training models</a> without the customer’s prior permission or instruction.</li>\n<li><strong>Anthropic:</strong> by default it <a href=\"https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training\">won’t use inputs or outputs</a> from its commercial products to train its models.</li>\n<li><strong>OpenAI:</strong> data sent to its API <a href=\"https://developers.openai.com/api/docs/guides/your-data\">isn’t used to train its models</a> unless you opt in. For ChatGPT’s business plans, OpenAI publishes its commitments on its <a href=\"https://openai.com/business-data/\">business data page</a>; check that your plan is named there before you rely on it.</li>\n</ul>\n<p>Now look for the exception in the feedback buttons. Anthropic says a thumbs up or down <a href=\"https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training\">stores the entire related conversation</a> for up to five years. It may use that feedback to train its models. Owners of Team and Enterprise plans <a href=\"https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training\">can switch feedback off</a> for everyone, under Data and Privacy in the organization settings.</p>\n<p><strong>Question 2: Who at your company can read our content, and when?</strong> A good answer is specific. Microsoft says its Copilot services have <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">opted out of abuse monitoring</a> in Azure OpenAI, the process that includes human review of content.</p>\n<h2 id=\"4-ask-how-long-prompts-are-kept-and-where-they-are-processed\">4. Ask how long prompts are kept and where they are processed</h2>\n<p><strong>Question 3: How long do you keep prompts, files and outputs, and who sets that?</strong> A good answer gives a number and lets an admin change it. Google Workspace keeps Gemini conversations for <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">90 days to indefinitely, as set by admins</a>. On Anthropic’s business plans, deleted conversations leave its back end <a href=\"https://privacy.claude.com/en/articles/7996866-how-long-do-you-store-my-organization-s-data\">within 30 days</a>. Microsoft lets admins set Copilot retention in Purview, and users can <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">delete their own Copilot history</a>.</p>\n<p>Short retention has a cost of its own. People lose the history they search when they want last month’s draft, so agree on the number before the trial, not after.</p>\n<p><strong>Question 4: Where is our data processed, and which features are exceptions?</strong> Ask for the list of exceptions, because that is where the surprises sit. Microsoft keeps EU users’ Copilot traffic <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">inside its EU Data Boundary</a>, but Anthropic models used inside Copilot are currently excluded. Web searches are another carve-out: those <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/enterprise-data-protection\">queries go to Bing</a>, where Microsoft acts as an independent data controller and the EU Data Boundary doesn’t apply.</p>\n<p>OpenAI’s API offers <a href=\"https://developers.openai.com/api/docs/guides/your-data\">data storage in 10 regions</a> but in-region processing only in the US and Europe, plus the UAE for some models. One product can hold several sets of terms. An EU team that wants everything inside the boundary has more than one switch to check.</p>\n<h2 id=\"5-ask-what-it-can-open-and-which-contract-covers-it\">5. Ask what it can open, and which contract covers it</h2>\n<p><strong>Question 5: What can the assistant see once it is connected to our email and files?</strong> The standard answer is only what the user can already see. Microsoft Copilot <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">surfaces only data</a> the user has at least view permission for, and Gemini <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">won’t retrieve</a> a document or email the user can’t access.</p>\n<p>That sounds safe. It also moves the risk to your own sharing settings. Picture a hypothetical salary sheet shared with the whole company years ago and forgotten: anyone can now ask the assistant about it. Before you connect anything, search your drive for files shared with everyone and tighten them.</p>\n<p><strong>Question 6: Which contract covers our use, and are you our data processor?</strong> A good answer names the document. Microsoft says organizational use of Copilot is <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/enterprise-data-protection\">covered by its Data Protection Addendum</a>, with Microsoft acting as data processor. If a vendor can’t name the document, take that as the answer.</p>\n<h2 id=\"6-write-one-page-on-what-can-and-cant-be-pasted\">6. Write one page on what can and can’t be pasted</h2>\n<p>One page, two lists. The first says what may go in: internal drafts, public information, your own notes. The second says what may not, at least until the vendor’s answers check out: customer personal data; passwords and keys; unreleased figures; anything under a confidentiality clause.</p>\n<p>Name one person who answers “can I paste this?” People will ask at five to five with a deadline at five, and a quick answer from a named human beats a policy nobody has opened.</p>\n<p>In the EU, the page also does legal work. Article 4 of the AI Act, on AI literacy, has <a href=\"https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai\">applied since February 2, 2025</a>. The Commission’s own Q&amp;A takes the case of a company whose staff use ChatGPT to write ad copy or translate. Its answer: they <a href=\"https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers\">should be informed about the specific risks</a>, for example hallucination.</p>\n<p>The duty is lighter than it sounds. <a href=\"https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers\">No certificate is needed</a>; a company can keep its own internal records of training instead. The Digital Omnibus on AI, in force since July 2026, <a href=\"https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers\">keeps the obligation</a> but no longer sets a “sufficient” level. National market surveillance authorities <a href=\"https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers\">supervise it from August 2026</a>.</p>\n<p>A dated rules page and a 30-minute briefing on where the tool gets things wrong make a sensible record.</p>\n<h2 id=\"7-run-a-two-week-trial-on-real-work\">7. Run a two-week trial on real work</h2>\n<p>Test two candidates at most, or one against the way you work now. More than that and the trial turns into a hobby.</p>\n<ul>\n<li><strong>Day 1:</strong> a 30-minute kickoff. Everyone gets the task list and the house rules, then picks three tasks they will do that week anyway.</li>\n<li><strong>Days 2 to 5:</strong> people use the assistant on those tasks and log every use in a shared sheet.</li>\n<li><strong>Week 2:</strong> comparing two tools, swap them. Testing one, add the next two tasks from the list. Hold a 15-minute check-in halfway, while problems are still cheap to fix.</li>\n<li><strong>Last day:</strong> each person writes three lines: what worked, what didn’t, and whether they would pay for it from their own team’s budget.</li>\n</ul>\n<p>The log needs six columns:</p>\n<table>\n<thead>\n<tr>\n<th>Date</th>\n<th>Task</th>\n<th>Usual minutes</th>\n<th>Minutes with the assistant, fixes included</th>\n<th>Outcome: used, edited or binned</th>\n<th>Errors found</th>\n</tr>\n</thead>\n</table>\n<p>Self-reported minutes are rough, but they are good enough to compare two tools used by the same people on the same kinds of task.</p>\n<h2 id=\"8-measure-five-numbers-before-deciding\">8. Measure five numbers before deciding</h2>\n<ul>\n<li><strong>Unprompted use.</strong> On how many days in week 2 did people reach for the assistant without a nudge? We think this is the most telling number in the trial. Tools people need reminding about end up as a line on the card statement.</li>\n<li><strong>Edit rate.</strong> What share of outputs went out as written, and what share needed editing or went in the bin? In Stack Overflow’s 2025 survey, 66% of developers named <a href=\"https://survey.stackoverflow.co/2025/ai\">“AI solutions that are almost right, but not quite”</a> as their top frustration with AI tools. Near misses cost time in the fixing.</li>\n<li><strong>Minutes including corrections.</strong> Compare the usual time with the assisted time, counting every minute spent checking.</li>\n<li><strong>Errors that reached a draft.</strong> Count wrong facts and invented sources or numbers. Microsoft’s own documentation says generative AI responses <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">aren’t guaranteed to be 100% factual</a>, and we would treat one error that reached a customer as a serious finding.</li>\n<li><strong>Cost per active user.</strong> Divide the monthly bill by the people who used the tool in week 2. Seats don’t use anything.</li>\n</ul>\n<p>An assistant that saves one enthusiast ten minutes a day and nobody else anything is a personal tool. Buy it as one.</p>\n<h2 id=\"9-decide-on-one-page-with-a-review-date\">9. Decide on one page, with a review date</h2>\n<p>Write down the tool, the tasks it is for and the house rules it runs under. Add who administers it and who removes accounts when people leave. Set a review date three months out, when you look at usage again and decide whether to keep paying.</p>\n<p>Send the six questions to your shortlisted vendors today, and start the task list while you wait for their answers.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Microsoft: <a href=\"https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part\">2024 Work Trend Index, AI at work is here. Now comes the hard part</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/microsoft-365-copilot-privacy\">Data, privacy and security for Microsoft Copilot</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/microsoft-365/copilot/enterprise-data-protection\">Enterprise data protection in Microsoft Copilot and Copilot Chat</a></li>\n<li>Google: <a href=\"https://knowledge.workspace.google.com/admin/gemini/generative-ai-in-google-workspace-privacy-hub\">Generative AI in Google Workspace privacy hub</a></li>\n<li>Google: <a href=\"https://support.google.com/gemini/answer/13594961\">Gemini Apps privacy hub</a></li>\n<li>Anthropic: <a href=\"https://privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training\">Is my data used for model training?</a></li>\n<li>Anthropic: <a href=\"https://privacy.claude.com/en/articles/7996866-how-long-do-you-store-my-organization-s-data\">How long do you store my organization’s data?</a></li>\n<li>Anthropic: <a href=\"https://www.anthropic.com/news/updates-to-our-consumer-terms\">Updates to consumer terms and privacy policy</a></li>\n<li>OpenAI: <a href=\"https://developers.openai.com/api/docs/guides/your-data\">Data controls in the OpenAI platform</a></li>\n<li>OpenAI: <a href=\"https://openai.com/business-data/\">Business data privacy, security and compliance</a></li>\n<li>European Commission: <a href=\"https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai\">AI Act, regulatory framework and timeline</a></li>\n<li>European Commission: <a href=\"https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers\">AI literacy, questions and answers</a></li>\n<li>Stack Overflow: <a href=\"https://survey.stackoverflow.co/2025/ai\">2025 Developer Survey, AI</a></li>\n</ul>\n",
      "summary": "Most people who use AI at work bring their own tools; the six questions to ask a vendor, the answers to expect and a two-week trial that settles the choice.",
      "image": "https://28creation.com/og/choosing-ai-assistant.jpg",
      "date_published": "2026-10-18T22:00:00.000Z",
      "date_modified": "2026-10-18T22:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "AI",
        "Work software"
      ]
    },
    {
      "id": "https://28creation.com/articles/automate-weekly-report/",
      "url": "https://28creation.com/articles/automate-weekly-report/",
      "title": "Weekly reports can send themselves: 9 steps, little or no code",
      "content_html": "<h2>The short read</h2><p><strong>If your team uses Google Workspace or Microsoft 365, a scheduler that sends the weekly report already comes with it.</strong></p><ul><li>Setting the timer is easy; reports break later because a schedule runs on one person&apos;s login or fires at a random minute.</li><li>A broken automated report rarely looks broken. It looks like a quiet week.</li></ul><h3>What you can do with this</h3><blockquote><p>Put a check cell beside each key number that flags a week with no data, and include it in the email. A failed import then reaches everyone instead of passing as a quiet week.</p></blockquote><hr><p>If your team works in Google Workspace or Microsoft 365, a scheduler that can send the weekly report on its own already comes with it. <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">Data Studio</a>, the tool Google used to call Looker Studio, emails a report as a PDF on a timer. <a href=\"https://learn.microsoft.com/en-us/power-automate/run-scheduled-tasks\">Power Automate</a> runs a flow every week, and its <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">free license</a> covers work accounts at no extra cost.</p>\n<p>The timer is the easy part. What breaks these reports months later sits in the fine print. A trigger may <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">fire at a random minute</a> within the hour. A schedule may <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">run on one person’s login</a> and fail once they leave.</p>\n<p>A broken automated report rarely looks broken. It looks like a quiet week. The nine steps below set up the schedule and put a guard on each weak point. Most need no code at all; one option needs six lines.</p>\n<h2 id=\"1-write-down-how-the-report-gets-made-today\">1. Write down how the report gets made today</h2>\n<p>Build the report by hand one more time and write down every step. Each export counts, each copy and paste, each “then I check whether sales has sent its numbers”. That list is your specification. It will be longer than you expect.</p>\n<p>For every number, add two facts: where it comes from and when it is final. “Invoices sent: accounting export, final Friday 06:00” is a useful line. “Revenue: from the system” is not.</p>\n<p>Then ask who reads each section and what they decide because of it. Delete what nobody reads. A section with no reader is cheaper to remove today than to automate, schedule and maintain for the next several years. Our <a href=\"https://28creation.com/articles/automation-payback-formula/\">payback template</a> puts hours on that upkeep before you build.</p>\n<h2 id=\"2-choose-the-scheduler-that-comes-with-what-you-already-pay-for\">2. Choose the scheduler that comes with what you already pay for</h2>\n<p>All five options below can run once a week. They differ in what they can read and in how they fail.</p>\n<table>\n<thead>\n<tr>\n<th>Your setup</th>\n<th>Scheduler</th>\n<th>Code?</th>\n<th>The limit to know, as of October 2026</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Google, report is charts</td>\n<td><a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">Data Studio</a> scheduled delivery</td>\n<td>None</td>\n<td>50 recipients; without Pro, one schedule per report and at most daily</td>\n</tr>\n<tr>\n<td>Google, report is a sheet</td>\n<td><a href=\"https://support.google.com/workspace-studio/answer/16782648\">Workspace Studio</a> flow</td>\n<td>None</td>\n<td>Business Starter and up; an admin must turn on Gemini</td>\n</tr>\n<tr>\n<td>Google, custom logic</td>\n<td><a href=\"https://developers.google.com/apps-script/guides/services/quotas\">Apps Script</a> trigger</td>\n<td>6 lines</td>\n<td>90 trigger minutes a day on a personal account, 6 hours on Workspace</td>\n</tr>\n<tr>\n<td>Microsoft 365</td>\n<td><a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Power Automate</a> Recurrence</td>\n<td>None</td>\n<td>Free license: standard connectors only, and flows can’t be shared</td>\n</tr>\n<tr>\n<td>Mixed tools</td>\n<td><a href=\"https://zapier.com/pricing\">Zapier</a> Schedule</td>\n<td>None</td>\n<td>Free plan: 100 tasks a month, two-step Zaps</td>\n</tr>\n</tbody>\n</table>\n<p>Our view: start with the suite your team signs in to every morning. A second platform means another login and another place to look when something fails.</p>\n<p>Zapier earns its place when the numbers live in tools Google and Microsoft can’t reach. Its <a href=\"https://zapier.com/pricing\">free plan</a> allows only two-step Zaps, one trigger and one action.</p>\n<p>A Zap that reads a sheet and then emails it needs a trigger and two actions. That means Professional, at $19.99 a month billed annually for 750 tasks as of October 2026. The schedule trigger itself <a href=\"https://help.zapier.com/hc/en-us/articles/8496288648461\">doesn’t count toward task usage</a>. For what a longer workflow costs on Zapier, Make and n8n, see <a href=\"https://28creation.com/articles/zapier-make-n8n-cost-one-workflow/\">our pricing of one workflow on all three</a>.</p>\n<h2 id=\"3-build-the-report-from-formulas-in-one-spreadsheet\">3. Build the report from formulas in one spreadsheet</h2>\n<p>Give each source its own raw tab, filled only by an import or by someone pasting that source’s export. Then make one report tab that calculates everything the report shows: totals, change since last week, distance to target. No typed numbers on that tab. A typed number goes stale, and nobody knows when.</p>\n<p>Protect the report tab so a helpful colleague can’t type over a formula. In Google Sheets, open Data, then Protect sheets and ranges, and pick Restrict who can edit this range. Google says protection <a href=\"https://support.google.com/docs/answer/1218656\">shouldn’t be used as a security measure</a>, so treat it as a guard against accidents.</p>\n<p>On Microsoft 365, format each raw range as an Excel table, because Power Automate’s Excel actions work on tables. Its List rows present in a table action <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">returns up to 256 rows by default</a> unless you turn on pagination. Without that switch, a report that grows past 256 rows comes out short and says nothing.</p>\n<h2 id=\"4-put-a-check-cell-beside-every-number-that-matters\">4. Put a check cell beside every number that matters</h2>\n<p>Next to each key figure, add a cell that flags data you shouldn’t trust. Two checks cover the failures that look most like a normal week: nothing arrived, or something jumped.</p>\n<ul>\n<li><strong>Nothing arrived:</strong> <code>=IF(COUNTIFS(Raw!A:A,\"&gt;=\"&amp;B1,Raw!A:A,\"&lt;\"&amp;B1+7)=0,\"NO DATA\",\"OK\")</code>, where column A of the Raw tab holds dates and B1 holds the Monday of the reporting week.</li>\n<li><strong>Something jumped:</strong> <code>=IF(ABS(C5/D5-1)&gt;0.5,\"CHECK\",\"OK\")</code>, where C5 is this week and D5 last week. If last week was zero, the cell shows an error, which is also worth a look.</li>\n</ul>\n<p>Color the flags with Format, then Conditional formatting, then Custom formula is. Google notes that these formulas <a href=\"https://support.google.com/docs/answer/78413\">can only reference the same sheet</a> unless you use INDIRECT, so keep the check cells on the report tab. Then include them in the message itself. A failed import should land in front of everyone, including the people who never open the sheet.</p>\n<h2 id=\"5-set-the-schedule-in-the-tool-you-picked\">5. Set the schedule in the tool you picked</h2>\n<p>Each set of steps uses the vendor’s own menu names.</p>\n<ul>\n<li><strong>Data Studio:</strong> open the report, click Share, then <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">Schedule delivery</a>. Add up to 50 recipients, pick pages, set the start date and time, then use Repeat; Custom handles “every two weeks on Friday”. The email carries the PDF, a preview of page one and a link.</li>\n<li><strong>Workspace Studio:</strong> start a flow with the <a href=\"https://support.google.com/workspace-studio/answer/16765661\">On a schedule</a> starter, set a start date and time, choose Weekly and set the time zone. Add Get sheet contents, then Send an email, or Post in a space for Google Chat.</li>\n<li><strong>Power Automate:</strong> go to <a href=\"https://learn.microsoft.com/en-us/power-automate/run-scheduled-tasks\">My flows, Create from blank</a>, add the Recurrence trigger, and set Interval to 1 and Frequency to Week. Pick a Time zone, the day under On these days and the hour under At these hours. Add Excel’s List rows present in a table, then <a href=\"https://learn.microsoft.com/en-us/power-automate/data-operations\">Create HTML table</a> to turn the rows into a table, then Outlook’s <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Send an email (V2)</a>. Both connectors are standard.</li>\n<li><strong>Zapier:</strong> choose Schedule by Zapier, trigger event <a href=\"https://help.zapier.com/hc/en-us/articles/8496288648461\">Every Week</a>, then Day of the Week and Time of Day.</li>\n<li><strong>Apps Script:</strong> in the sheet, open <a href=\"https://developers.google.com/apps-script/guides/bound\">Extensions, then Apps Script</a>, paste the function below and change the tab, range and address. Then click <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">Triggers, then Add Trigger</a>, choose the function and a weekly time-driven timer, and save.</li>\n</ul>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"js\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">function</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> sendWeeklyReport</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() {</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> ss</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> SpreadsheetApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getActiveSpreadsheet</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> rows</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ss.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getSheetByName</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Report'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getRange</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'A1:B12'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getDisplayValues</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">  const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> body</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> rows.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> r.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">join</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">': '</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">join</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">+</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> +</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> ss.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">getUrl</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">();</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  MailApp.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">sendEmail</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'team@example.com'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'Weekly report'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, body);</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span></code></pre>\n<p>It mails the report tab’s cells as displayed, check cells included, with a link to the sheet. Run it once from the editor before you add the trigger, because <a href=\"https://developers.google.com/apps-script/guides/services/authorization\">triggers must be authorized</a> by the person who created them. <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">MailApp</a> can reach 100 recipients a day from a personal account and 1,500 from Workspace.</p>\n<h2 id=\"6-schedule-the-run-an-hour-after-the-data-is-final\">6. Schedule the run an hour after the data is final</h2>\n<p>Timers are looser than they look. Apps Script says a trigger’s time <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">might be slightly randomized</a>. One set for 9 a.m. runs at a time it picks between 9 and 10, then keeps that time day to day. Zapier says its schedule <a href=\"https://help.zapier.com/hc/en-us/articles/8496288648461\">should run within a few minutes</a> of the set time.</p>\n<p>Time zones are the other trap. Zapier <a href=\"https://help.zapier.com/hc/en-us/articles/8496288648461\">uses your Zapier account’s time zone</a> and shows run times in UTC in Zap history. Change the zone, and you must turn the Zap off and on again.</p>\n<p>Data Studio <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">follows your operating system’s time zone</a>, and you can’t change it inside Data Studio. Power Automate asks for a start time in a UTC format plus a separate <a href=\"https://learn.microsoft.com/en-us/power-automate/run-scheduled-tasks\">Time zone field</a>, so set that field.</p>\n<p>So if a source is final at 06:00, schedule for 07:30 rather than 06:05. Buy yourself the hour. Check the first run after the clocks change for daylight saving time.</p>\n<p>One more surprise for teams outside the US: Data Studio’s scheduled reports <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">use US English formatting</a> even if your account is set to another language. Read 10/03 as October 3.</p>\n<h2 id=\"7-send-the-draft-and-every-failure-to-one-owner-first\">7. Send the draft and every failure to one owner first</h2>\n<p>The automation can fetch and add up. It can’t write the two sentences that say revenue dipped because two large invoices slid into next week. So send the finished report to one owner at 07:30, and let them add that comment and forward it by 09:00.</p>\n<p>The owner is also where failures should land, and each tool raises them differently. When a trigger fails, Apps Script <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">shows nothing on screen and sends an email</a> with a link to reconfigure or switch it off. Power Automate <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">turns off a cloud flow</a> whose trigger or actions fail continuously for 14 days. If those alerts go to an inbox nobody reads, the report simply stops one Friday.</p>\n<h2 id=\"8-run-the-old-and-new-report-side-by-side-for-three-weeks\">8. Run the old and new report side by side for three weeks</h2>\n<p>Keep building the report by hand and compare the two, number by number. A comparison tab makes it quick: <code>=IF(Report!C5=Manual!C5,\"\",\"DIFF\")</code> in each cell that matters.</p>\n<p>Every difference is a bug in one of the two versions. Expect some in the manual one. Switch off the hand-built report only after three weeks in a row with no differences you can’t explain.</p>\n<h2 id=\"9-plan-for-the-day-the-owners-account-goes-away\">9. Plan for the day the owner’s account goes away</h2>\n<p>Most schedulers run as a person, not as the team. Apps Script triggers <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">always run under the account of the person who created them</a>. A flow built on Power Automate’s Free license <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">can’t be shared</a> at all.</p>\n<p>Data Studio schedules <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">run with the credentials of whoever last saved them</a>. When that account goes inactive, the schedule fails until an active editor opens it, clicks Edit and saves. Until then, nothing arrives.</p>\n<p>Write a short note at the top of the sheet. Say which account runs each schedule and who owns the report, then what to do when a check cell turns red. Add one line to your offboarding checklist: move or re-save every schedule the leaver owns before their account is closed.</p>\n<p>Start with step 1 this Friday. Build the report by hand one last time, with a blank document open beside it, and write down every click.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Google: <a href=\"https://docs.cloud.google.com/looker/docs/studio/schedule-automatic-report-delivery\">Schedule automatic report delivery, Data Studio</a></li>\n<li>Google: <a href=\"https://developers.google.com/apps-script/guides/triggers/installable\">Installable triggers, Apps Script</a></li>\n<li>Google: <a href=\"https://developers.google.com/apps-script/guides/services/quotas\">Quotas for Google services, Apps Script</a></li>\n<li>Google: <a href=\"https://developers.google.com/apps-script/guides/bound\">Scripts bound to Google Docs, Sheets, Slides and Forms</a></li>\n<li>Google: <a href=\"https://developers.google.com/apps-script/guides/services/authorization\">Authorization for Google services, Apps Script</a></li>\n<li>Google: <a href=\"https://support.google.com/workspace-studio/answer/16765661\">Guide to starters and steps, Workspace Studio</a></li>\n<li>Google: <a href=\"https://support.google.com/workspace-studio/answer/16782648\">Access to Workspace Studio</a></li>\n<li>Google: <a href=\"https://support.google.com/docs/answer/78413\">Use conditional formatting rules in Google Sheets</a></li>\n<li>Google: <a href=\"https://support.google.com/docs/answer/1218656\">Protect, hide and edit sheets</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/power-automate/run-scheduled-tasks\">Run a cloud flow on a schedule in Power Automate</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/power-automate/limits-and-config\">Limits of automated, scheduled and instant flows</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/power-platform/admin/power-automate-licensing/types\">Types of Power Automate licenses</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/connectors/excelonlinebusiness/\">Excel Online (Business) connector</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/connectors/office365/\">Office 365 Outlook connector</a></li>\n<li>Microsoft: <a href=\"https://learn.microsoft.com/en-us/power-automate/data-operations\">Use data operations in Power Automate</a></li>\n<li>Zapier: <a href=\"https://help.zapier.com/hc/en-us/articles/8496288648461\">Schedule by Zapier triggers</a></li>\n<li>Zapier: <a href=\"https://zapier.com/pricing\">Plans and pricing</a></li>\n</ul>\n",
      "summary": "Google, Microsoft and Zapier each include a scheduler that can send the report on time; how to set one up, and the documented limits that break it later.",
      "image": "https://28creation.com/og/automate-weekly-report.jpg",
      "date_published": "2026-10-18T21:30:00.000Z",
      "date_modified": "2026-10-18T21:30:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/articles/noticing-what-to-automate/",
      "url": "https://28creation.com/articles/noticing-what-to-automate/",
      "title": "Office tasks worth automating hide in habits; a tally finds them",
      "content_html": "<h2>The short read</h2><p><strong>People&apos;s estimates of their own screen time matched device logs within 5% in only 3 of 49 comparisons.</strong></p><ul><li>About 43% of what students logged in a 2002 diary study were habits, done with their minds more often elsewhere.</li><li>So asking a team what to automate tends to miss the small, frequent tasks that make the best first projects.</li></ul><h3>What you can do with this</h3><blockquote><p>Have one volunteer keep a tally sheet by the keyboard for five days, with a tick each time a task repeats. You get counts instead of guesses, and a clear first task to automate.</p></blockquote><hr><p>In a 2002 study, 209 students at Texas A&amp;M wore a wristwatch that chimed every hour and wrote down what they were doing and thinking. About 43% of what they recorded were habits, things done almost daily in the same place, and while doing them their minds were <a href=\"https://dornsife.usc.edu/wendy-wood/wp-content/uploads/sites/183/2023/10/Wood.Quinn_.Kashy_.2002_Habits_in_everyday_life.pdf\">more often somewhere else</a>.</p>\n<p>Office work runs on the same machinery, and that is why the hardest part of automating it is noticing what to automate. Asking a team what to automate relies on memory, and memory is least reliable for the small tasks people repeat all day. By the time a task is frequent enough to be worth automating, it has often become a habit.</p>\n<p>That gap matters for anyone who runs or builds software for a small company, because the opportunity sits in exactly those small tasks. This piece explains why people miss their own repetition, why the small tasks make the best first projects, and how to run a one-week tally that finds them.</p>\n<h2 id=\"habits-run-without-anyone-watching\">Habits run without anyone watching</h2>\n<p>The study, by psychologists Wendy Wood, Jeffrey Quinn and Deborah Kashy, defined a habit plainly: something done just about every day, usually in the same location. In the first study, 70 students kept the hourly diary for one day and 35% of the behaviors they listed were habits. In the second, 209 students kept it for two days and the figure was 43%.</p>\n<p>What happened in their heads was the striking part. In the first study, for 60% of the habits people recorded, they were thinking about something unrelated. For less routine actions, their thoughts matched what they were doing in 70% of cases.</p>\n<p>Overall, the authors estimate, their participants did almost 50% of everything without thinking about it. The sample was college students, and the authors note that the figure could be a lower bound, since students may spend more of the day on study and new activities than other people. Nobody was told the study was about habits until the debriefing.</p>\n<p>Now apply the definition to an office. Copying an order number from the web shop into the accounting system meets it. So do the same lookups before every customer call. Both are done almost daily in the same place, which by the study’s definition makes them habits.</p>\n<h2 id=\"asking-people-returns-the-official-process\">Asking people returns the official process</h2>\n<p>The usual way to find automation candidates is to ask: interviews, workshops, a process map on a whiteboard. All of them depend on people reporting their own behavior, and that is a weak instrument.</p>\n<p>Some of the best evidence comes from screen time, where self-reports can be checked against device logs. A 2021 meta-analysis led by Douglas Parry pooled <a href=\"https://www.bath.ac.uk/announcements/warnings-on-the-dangers-of-screen-time-are-ill-founded-new-study/\">47 studies</a> covering about 50,000 people. Guess how often people’s average estimate of their own use came within 5% of the log. It happened in 3 of 49 comparisons, about 6%.</p>\n<p>The errors ran both ways: people over-reported in 23 comparisons and under-reported in 23 (<a href=\"https://doi.org/10.1038/s41562-021-01117-5\">Parry et al.</a>). The overall correlation between what people said and what the logs showed was 0.38, which the authors call only medium in magnitude. They note that memory problems are most apparent for behavior that is frequent and woven into daily life. That is the profile of an office habit.</p>\n<p>User researchers learned this long ago. Jakob Nielsen’s 2001 essay on the <a href=\"https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/\">first rule of usability</a> tells designers to “pay attention to what users do, not what they say.” Watching people work in their own setting, the method called contextual inquiry, can “uncover low-level details that have become habitual and invisible,” as <a href=\"https://www.nngroup.com/articles/contextual-inquiry/\">Nielsen Norman Group</a> puts it.</p>\n<p>Ask how someone handles a supplier invoice and you hear the process. Watch the next one arrive and you see the work.</p>\n<h2 id=\"the-opportunity-sits-in-tasks-not-in-jobs\">The opportunity sits in tasks, not in jobs</h2>\n<p>The research on what can be automated points at the same small tasks. In 2017 the McKinsey Global Institute estimated that <a href=\"https://www.mckinsey.com/featured-insights/digital-disruption/harnessing-automation-for-a-future-that-works\">less than 5% of occupations</a> could be fully automated with technology already demonstrated. Yet about half of all the activities people are paid to do could be, worth almost $15 trillion in wages. The activities it rated most exposed included data collection and processing.</p>\n<p>McKinsey raised that figure in 2023. With generative AI, it estimated that work taking up <a href=\"https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier\">60 to 70% of employees’ time</a> could technically be automated. That is a theoretical ceiling: McKinsey’s adoption scenarios put the halfway point somewhere between 2030 and 2060. The same report recalls a 2012 MGI estimate that knowledge workers spent about a fifth of their time, one day a week, searching for and gathering information.</p>\n<p>Surveys of workers point the same way. Asana’s Anatomy of Work Index says people spend <a href=\"https://asana.com/resources/why-work-about-work-is-bad\">60% of their time</a> on “work about work,” such as searching for information, switching between apps and chasing the status of tasks. It puts duplicated work at 209 hours a year per knowledge worker.</p>\n<p>Two cautions apply. Asana sells work-management software, and its figures are self-reported, the kind of measurement the Parry study warns about. Read the shares as a direction rather than a measurement. Taken together, the sources point one way: the time worth winning back is scattered across small tasks inside jobs.</p>\n<h2 id=\"small-frequent-tasks-make-the-best-first-projects\">Small, frequent tasks make the best first projects</h2>\n<p>A small, frequent task usually follows a narrow rule, uses input that keeps its shape, and has one obvious owner. A mistake costs little and shows up fast. The big process everyone named in the workshop has the opposite profile: many people, many exceptions and rules nobody wrote down.</p>\n<p>The arithmetic also favors the small task. Randall Munroe’s chart <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a> works out how long you can spend making a routine task faster before you lose time over five years. Shave one minute off something done five times a day and you can spend six days on the automation. Shave 30 seconds off something done 50 times a day and the budget is four weeks.</p>\n<p>The chart counts 24-hour days and no upkeep; our <a href=\"https://28creation.com/articles/xkcd-worth-the-time-upkeep/\">recomputed xkcd table</a> uses a working year and subtracts the hours spent keeping an automation running.</p>\n<p>Small automations have costs too. Each one is another thing that can break, and ten of them without owners is a mess of its own. Some repetition should be deleted rather than automated, such as a status report nobody reads.</p>\n<p>Watching people work raises fair questions about consent, especially when software does the watching. A tally kept by the person themselves avoids most of that. Our explainer on <a href=\"https://28creation.com/articles/task-mining-vs-process-mining/\">task mining and process mining</a> sets out what regulators say about software that records screens.</p>\n<h2 id=\"what-this-means-for-you-run-a-one-week-tally\">What this means for you: run a one-week tally</h2>\n<p>The tally takes a pen, a sheet of paper and five working days. It replaces estimates with counts, which is the whole point.</p>\n<ol>\n<li><strong>Pick one person who volunteers.</strong> Ideally someone who does a lot of routine work and wants less of it. Say plainly that the tally measures tasks, not performance, and that the sheet stays with them.</li>\n<li><strong>Count, never estimate.</strong> Put the sheet next to the keyboard. Each time they copy something from one system to another, look up something they looked up before, or retype data that exists elsewhere, they write the task once and add a tick each time it recurs. Note the rough minutes for one instance.</li>\n<li><strong>Set an hourly prompt.</strong> Wood’s participants had a watch chime every hour to prompt their entries. A recurring reminder on the hour does the same job and catches what slipped past.</li>\n<li><strong>Add one hour of watching.</strong> Midweek, a colleague sits beside them for an hour and notes every switch between applications. Compare that list with the sheet. The gaps are the habits the tally missed.</li>\n<li><strong>Score the list on Friday.</strong> For each item, multiply ticks by minutes. Then ask whether the rule fits in one sentence, whether the input keeps the same shape, what a mistake would cost, and who would own the automation.</li>\n<li><strong>Pick one and build it small.</strong> Take the highest-scoring item with a clear rule, stable input and a cheap mistake. Check it against the xkcd chart. Before it runs, the person whose work changes should see exactly what it will do and say yes. If saving email attachments by hand tops the list, our guide to <a href=\"https://28creation.com/articles/save-outlook-attachments-automatically/\">saving Outlook attachments automatically</a> covers three ways to build it.</li>\n</ol>\n<p>Then wait a month and run the tally again. Once the loudest repetition is gone, the next one is easier to hear.</p>\n<p>The students in the 2002 study learned what the research was about only at the end. What they did on autopilot became visible because a watch chimed every hour and asked. A sheet of paper by the keyboard can do that job in an office.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li>Wendy Wood, Jeffrey M. Quinn and Deborah A. Kashy, <a href=\"https://dornsife.usc.edu/wendy-wood/wp-content/uploads/sites/183/2023/10/Wood.Quinn_.Kashy_.2002_Habits_in_everyday_life.pdf\">Habits in Everyday Life: Thought, Emotion, and Action</a>, Journal of Personality and Social Psychology 83(6), 2002.</li>\n<li>Douglas A. Parry and others, <a href=\"https://doi.org/10.1038/s41562-021-01117-5\">A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use</a>, Nature Human Behaviour, 2021; University of Bath <a href=\"https://www.bath.ac.uk/announcements/warnings-on-the-dangers-of-screen-time-are-ill-founded-new-study/\">summary</a>, May 17, 2021.</li>\n<li>Jakob Nielsen, <a href=\"https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/\">First Rule of Usability? Don’t Listen to Users</a>, Nielsen Norman Group, August 4, 2001.</li>\n<li>Kim Flaherty, <a href=\"https://www.nngroup.com/articles/contextual-inquiry/\">Contextual Inquiry: Inspire Design by Observing and Interviewing Users in Their Context</a>, Nielsen Norman Group, December 6, 2020.</li>\n<li>McKinsey Global Institute, <a href=\"https://www.mckinsey.com/featured-insights/digital-disruption/harnessing-automation-for-a-future-that-works\">Harnessing automation for a future that works</a>, January 12, 2017.</li>\n<li>McKinsey, <a href=\"https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier\">The economic potential of generative AI: The next productivity frontier</a>, June 14, 2023.</li>\n<li>Asana, <a href=\"https://asana.com/resources/why-work-about-work-is-bad\">How work about work gets in the way of real work</a>, updated April 17, 2026.</li>\n<li>Randall Munroe, <a href=\"https://xkcd.com/1205/\">Is It Worth the Time?</a>, xkcd 1205.</li>\n</ul>\n",
      "summary": "People misjudge the tasks they repeat most, so asking a team what to automate tends to miss the best candidates; counting for one week does not.",
      "image": "https://28creation.com/og/noticing-what-to-automate.jpg",
      "date_published": "2026-10-18T21:00:00.000Z",
      "date_modified": "2026-10-18T21:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Automation",
        "Startups"
      ]
    },
    {
      "id": "https://28creation.com/notes/astro-fonts-guide/",
      "url": "https://28creation.com/notes/astro-fonts-guide/",
      "external_url": "https://docs.astro.build/en/guides/fonts/",
      "title": "Astro's fonts guide walks through its built-in Fonts API",
      "content_html": "<p>Astro’s own walk-through of its built-in Fonts API, and a good one. Register a font in the project config with a provider like Fontsource or Google, or point it at local files, then put the <code>&lt;Font /&gt;</code> component in the page head, and the fonts get downloaded and served from your own site. Read the part on preloading and fallback settings if layout shift has ever annoyed you.</p>\n<p>Link: <a href=\"https://docs.astro.build/en/guides/fonts/\">docs.astro.build</a></p>",
      "summary": "Astro's own walk-through of its built-in Fonts API, and a good one. Register a font in the project config with a provider like Fontsource or Google, or point it at local files, then put the component in the page head, and the fonts get downloaded and served from your own site.…",
      "date_published": "2026-10-18T20:45:00.000Z",
      "date_modified": "2026-10-18T20:45:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/articles/tiny-automation-in-code/",
      "url": "https://28creation.com/articles/tiny-automation-in-code/",
      "title": "Scheduled scripts fail silently; seven checks make them speak up",
      "content_html": "<h2>The short read</h2><p><strong>In 2017 GitLab lost about six hours of data: its backups had been failing, and the alert emails bounced.</strong></p><ul><li>In a 34-line invoice script, two lines find the late invoices; the rest of the work is making sure a failure reaches a person.</li><li>A scheduled script runs where nobody watches, so a failure stays silent unless someone builds it a voice.</li></ul><h3>What you can do with this</h3><blockquote><p>Add a ping after each successful run to a dead man&apos;s switch on another machine, such as Healthchecks.io&apos;s free plan. If the script fails or stops running, you get an alert.</p></blockquote><hr><p>On January 31, 2017, an engineer at GitLab.com wiped a database directory on the primary server, <a href=\"https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/\">“errantly thinking”</a> it was the secondary. The storage bucket meant to hold the backups was empty, and GitLab lost about six hours of data. Its backup job, set to run every 24 hours, had been failing, and the alert emails it sent were rejected by the receiving server.</p>\n<p>GitLab’s <a href=\"https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/\">own summary</a>: “we were never aware of the backups failing, until it was too late.” That is the risk in every scheduled script. It runs where nobody watches, so a failure stays silent unless someone builds it a voice.</p>\n<p>We think that is where most of the effort in a small automation belongs, and a 34-line script that chases late invoices shows why. The checklist at the end works for any script you schedule.</p>\n<h2 id=\"two-lines-of-the-invoice-script-do-the-job\">Two lines of the invoice script do the job</h2>\n<p>A hypothetical small company’s accounting system drops a CSV export of invoices into a shared folder every night. The columns are <code>number</code>, <code>customer</code>, <code>amount</code>, <code>due</code> and <code>status</code>. Someone opens it each morning, sorts by due date and tells the team about anything late, except on busy days.</p>\n<p>The script below does that every weekday at 08:00 and posts to the team chat through an incoming webhook. It runs on Node.js 22 or newer, with no packages.</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"js\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// overdue.mjs: post overdue invoices from a CSV export to a chat webhook.</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// Expects columns number,customer,amount,due,status, with due as YYYY-MM-DD</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\">// and no commas inside fields.</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">import</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> { readFile } </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">from</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'node:fs/promises'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> file</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> process.argv[</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">2</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">??</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'invoices.csv'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> webhook</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> process.env.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">WEBHOOK_URL</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">webhook) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'WEBHOOK_URL is not set'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> today</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Date</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toISOString</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">slice</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">10</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">); </span><span style=\"color:#6A737D;--shiki-dark:#768390\">// UTC date</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">header</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">lines</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">] </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">await</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> readFile</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(file, </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'utf8'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">)).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">().</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">split</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\r</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">?</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(Boolean);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> cols</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> header.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">split</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">','</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> invoices</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> lines.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">line</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  Object.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">fromEntries</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(line.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">split</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">','</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">value</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#E36209;--shiki-dark:#F69D50\">i</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [cols[i], value.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">trim</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">()])),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> isDate</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">s</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">^</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{4}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">-</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">\\d</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">{2}$</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">/</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">test</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(s </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">??</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> open</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> invoices.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">inv</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (inv.status </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">??</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> ''</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">).</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">toLowerCase</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">() </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!==</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 'paid'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> overdue</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> open.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">inv</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> isDate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(inv.due) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> inv.due </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">&lt;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> today);</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> unclear</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> open.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">filter</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">((</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">inv</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> !</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">isDate</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(inv.due));</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (overdue.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ===</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> &amp;&amp;</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> unclear.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ===</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) process.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">exit</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">0</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> row</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#E36209;--shiki-dark:#F69D50\">inv</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">=&gt;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> `${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">inv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">number</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}  ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">inv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">customer</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}  ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">inv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">amount</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}  due ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">inv</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">due</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> ||</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> '?'}`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">;</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> parts</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> [];</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (overdue.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) parts.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">push</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">`Overdue (${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">overdue</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}):`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">overdue.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row));</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (unclear.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) parts.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">push</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">`Check by hand, no readable due date (${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">unclear</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">length</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}):`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">, </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">...</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">unclear.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">map</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(row));</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">const</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> res</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> =</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> await</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> fetch</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(webhook, {</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  method: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'POST'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">,</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  headers: { </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'content-type'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'application/json'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> },</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">  body: </span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\">JSON</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">stringify</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">({ text: parts.</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\">join</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">'</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">) }),</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">});</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F47067\">if</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> (</span><span style=\"color:#D73A49;--shiki-dark:#F47067\">!</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">res.ok) </span><span style=\"color:#D73A49;--shiki-dark:#F47067\">throw</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> new</span><span style=\"color:#6F42C1;--shiki-dark:#DCBDFB\"> Error</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">(</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">`Webhook answered ${</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">res</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">.</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">status</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">}`</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">);</span></span></code></pre>\n<p>Guess how many of its 28 non-blank lines decide which invoices are late. Two: the filters that build <code>open</code> and <code>overdue</code>.</p>\n<p>Seven lines exist only because something can go wrong, from the check for a missing secret to the check on the webhook’s answer. Two of those are comments that state assumptions. Five more do the job with a guard built in, so 12 of the 28 deal with failure, including both lines that do the job.</p>\n<h2 id=\"bad-input-looks-exactly-like-a-quiet-morning\">Bad input looks exactly like a quiet morning</h2>\n<p>Amazon’s <a href=\"https://aws.amazon.com/message/41926/\">S3 outage</a> on February 28, 2017 started with input. At 9:37AM PST an authorized team member following an established playbook ran a command, and “one of the inputs to the command was entered incorrectly.” It removed a larger set of servers than intended, and S3 in the region did not finish recovering until 1:54PM PST. AWS’s fix went into the tool, which now removes capacity more slowly and refuses to take a subsystem “below its minimum required capacity level.”</p>\n<p>A small script needs the same habit of refusing input that cannot be right. The CSV format’s specification, <a href=\"https://www.rfc-editor.org/rfc/rfc4180\">RFC 4180</a>, ends each record with a carriage return and a line feed. A file that follows it puts an invisible carriage return before every line break.</p>\n<p>We deleted the script’s handling for that character and ran a test file. A paid invoice showed up as overdue, because the <code>status</code> column’s name now ended in the stray character.</p>\n<p>Our tests also found a hole in the script itself. A file with only a header row made it exit cleanly and post nothing, which is exactly what it does on a good day. It never checks the file’s date either, so if the nightly export stops, the script keeps reading the last one. Two more refusals, for zero rows and for a file older than a day, would close both holes.</p>\n<h2 id=\"every-assumption-needs-a-place-to-fail-out-loud\">Every assumption needs a place to fail out loud</h2>\n<p>The <a href=\"https://www.rfc-editor.org/rfc/rfc4180\">same RFC</a> says fields containing commas “should be enclosed in double-quotes”, which the script’s plain split on commas would break. The comment at the top says the export has no such fields. That is fine as long as it stays true, and the written assumption tells the next person where to look when it stops being true.</p>\n<p>Dates get the same treatment. Any due date that is not <code>YYYY-MM-DD</code> goes to a “check by hand” list. Without that list, a row where someone typed <code>30/09</code> would never count as overdue, and the script would say nothing about it every morning.</p>\n<p>The date comparison is plain text. <a href=\"https://www.rfc-editor.org/rfc/rfc3339\">RFC 3339</a> notes that dates written this way, in a single time zone, “may be sorted as strings”, and <code>toISOString()</code> is <a href=\"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString\">always UTC</a>. For a run at 08:00 in a time zone an hour or two ahead of UTC, that is the same calendar day. Just after local midnight it would still be yesterday in UTC, and an invoice due yesterday would wait a day.</p>\n<h2 id=\"cron-has-failure-modes-of-its-own-and-its-manuals-admit-them\">Cron has failure modes of its own, and its manuals admit them</h2>\n<p>Put the webhook URL in a <code>.env</code> file next to the script, and run it once by hand before you schedule anything:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"bash\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># .env holds one line: WEBHOOK_URL=https://chat.example.com/hooks/your-hook</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">node</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> --env-file=.env</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> overdue.mjs</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> invoices.csv</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#768390\"># Then add it to cron with crontab -e: weekdays at 08:00</span></span>\n<span class=\"line\"><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">0</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> 8</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> *</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> *</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> 1-5</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> cd</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /srv/overdue</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\"> &amp;&amp; </span><span style=\"color:#6F42C1;--shiki-dark:#F69D50\">/usr/bin/node</span><span style=\"color:#005CC5;--shiki-dark:#6CB6FF\"> --env-file=.env</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> overdue.mjs</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> /data/exports/invoices.csv</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> &gt;&gt;</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\"> run.log</span><span style=\"color:#D73A49;--shiki-dark:#F47067\"> 2&gt;&amp;1</span></span></code></pre>\n<p>Each part of that cron line answers a failure. Cronie’s cron daemon sets its own <code>PATH</code> rather than inheriting one, and its <a href=\"https://man7.org/linux/man-pages/man8/cron.8.html\">manual</a> offers a <code>-P</code> flag for the opposite. So the line names <code>/usr/bin/node</code> in full instead of hoping <code>node</code> is found (<code>which node</code> shows where yours is).</p>\n<p>The <code>&gt;&gt; run.log 2&gt;&amp;1</code> keeps every error in a file. Without it, cron <a href=\"https://man7.org/linux/man-pages/man8/cron.8.html\">mails output</a> to the crontab’s owner, or writes it to the system log when sendmail is not installed. <a href=\"https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/\">GitLab’s alerts</a> went by mail too.</p>\n<p>Then there is the clock, because <code>0 8 * * 1-5</code> names a time without a time zone. <a href=\"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows\">GitHub Actions</a> runs schedules “in UTC” by default. <a href=\"https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/\">Kubernetes</a> reads them in the controller’s “local time zone” unless <code>.spec.timeZone</code> is set, and cronie lets a crontab pick one with <a href=\"https://man7.org/linux/man-pages/man5/crontab.5.html\"><code>CRON_TZ</code></a>. Run <code>date</code> on the server before you trust that 08:00 means your 08:00.</p>\n<p>Daylight saving time is worse, because cronie’s two manuals disagree. Its <a href=\"https://man7.org/linux/man-pages/man5/crontab.5.html\">crontab page</a> says jobs in the “missing hours” are “not to be run” and jobs in a repeated hour run twice. Its <a href=\"https://man7.org/linux/man-pages/man8/cron.8.html\">daemon page</a> says skipped jobs “will be run immediately” and double runs are avoided. Whichever is right, we would schedule nothing in the hour the clocks change.</p>\n<h2 id=\"a-scheduled-job-can-run-twice-or-not-at-all\">A scheduled job can run twice, or not at all</h2>\n<p>Cron assumes the machine is always on. That assumption is why anacron exists: its <a href=\"https://man7.org/linux/man-pages/man8/anacron.8.html\">manual</a> says that “unlike cron(8), it does not assume that the machine is running continuously.”</p>\n<p>Bigger schedulers say so outright. <a href=\"https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/\">Kubernetes</a> creates a job “approximately once” per scheduled time, because “two Jobs might be created, or no Job might be created.” Its advice is that jobs “should be idempotent”, which means safe to run twice.</p>\n<p><a href=\"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows\">GitHub Actions</a> warns that under heavy load “some queued jobs may be dropped”. In a public repository, it turns schedules off after 60 days without activity.</p>\n<p>For the invoice script, a double run posts the same list twice, which is annoying and harmless. A script that sends invoices or charges cards cannot shrug it off. It needs a record of what it already did. In our view, that record is the point where a script should grow into a service.</p>\n<h2 id=\"silence-needs-an-alarm-that-lives-somewhere-else\">Silence needs an alarm that lives somewhere else</h2>\n<p>On a bad morning the team sees this message. The body below is what our test run sent:</p>\n<pre class=\"astro-code astro-code-themes github-light github-dark-dimmed\" style=\"background-color:#fff;--shiki-dark-bg:#22272e;color:#24292e;--shiki-dark:#adbac7; overflow-x: auto;\" tabindex=\"0\" data-language=\"json\"><code><span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">{</span></span>\n<span class=\"line\"><span style=\"color:#005CC5;--shiki-dark:#8DDB8C\">  \"text\"</span><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">: </span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">\"Overdue (2):</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">1042  Acme Freight  12400  due 2026-09-24</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">1047  Example Studio  3150  due 2026-09-30</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">Check by hand, no readable due date (1):</span><span style=\"color:#005CC5;--shiki-dark:#F47067\">\\n</span><span style=\"color:#032F62;--shiki-dark:#96D0FF\">1051  Harbor Print  8900  due 30/09\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#ADBAC7\">}</span></span></code></pre>\n<p>On a good morning it sees nothing. It would see the same nothing if the server had died three weeks ago. Google’s <a href=\"https://sre.google/sre-book/monitoring-distributed-systems/\">SRE book</a> says monitoring should answer “what’s broken, and why?”, and that “it’s better to spend much more effort on catching symptoms than causes.” For a scheduled script the symptom is simple: it did not finish, and one alert on that covers every cause at once.</p>\n<p>That alert is a dead man’s switch: the job checks in after every success, and silence sets off the alarm. <a href=\"https://healthchecks.io/docs/\">Healthchecks.io</a>, one such service, says it “works as a dead man’s switch for processes that need to run continuously or on a regular, known schedule.” Its free <a href=\"https://healthchecks.io/pricing/\">Hobbyist plan</a> covers 20 jobs.</p>\n<p>Its <a href=\"https://healthchecks.io/docs/monitoring_cron_jobs/\">cron guide</a> adds one command after the job: <code>&amp;&amp; curl -fsS -m 10 --retry 5 -o /dev/null https://hc-ping.com/your-uuid-here</code>. The <code>&amp;&amp;</code> sends the ping only when the job exits with code 0. If no ping arrives within the grace time, the check goes down and the service sends its alerts.</p>\n<p>This is why the script throws instead of logging and carrying on. An uncaught error makes Node <a href=\"https://nodejs.org/docs/latest-v22.x/api/process.html#event-uncaughtexception\">exit with code 1</a>, so no ping goes out. We tested it with no <code>WEBHOOK_URL</code> set, and with a webhook that answered 410: both runs exited with 1. Slack’s webhooks <a href=\"https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/\">answer with errors</a> such as <code>channel_is_archived</code> when a channel is archived, which the <code>res.ok</code> check turns into that exit code.</p>\n<p>The alarm must not share the script’s fate. During the <a href=\"https://aws.amazon.com/message/41926/\">S3 outage</a>, AWS could not update its own status dashboard “because of a dependency the SHD administration console has on Amazon S3.” A check that runs on the same server as the script dies with it.</p>\n<h2 id=\"what-this-means-for-you\">What this means for you</h2>\n<p>The decision is how much failure handling a small script deserves. Too little and it fails like GitLab’s backups; too much and the alerts turn to noise. The <a href=\"https://sre.google/sre-book/monitoring-distributed-systems/\">SRE book</a> warns that when pages come too often, “employees second-guess, skim, or even ignore incoming alerts.” Its rule is that “every page should be actionable”, and we would hold a chat channel to the same rule.</p>\n<p>So aim for two kinds of message. One says the data needs a person, like the “check by hand” list; the other says the script did not run. Everything else goes to the log.</p>\n<p>Before you schedule a script, run it past this list:</p>\n<ol>\n<li><strong>The assumptions are written at the top:</strong> columns, formats and time zone.</li>\n<li><strong>Bad input stops the run or goes to a person:</strong> zero rows, a stale file, a missing column, a value that cannot be read.</li>\n<li><strong>Every failure exits with a non-zero code,</strong> including an error answer from any service the script calls.</li>\n<li><strong>Secrets live outside the file.</strong> <a href=\"https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/\">Slack</a> says a webhook URL “contains a secret”, and <a href=\"https://healthchecks.io/docs/\">Healthchecks.io</a> says to treat the IDs in its ping URLs as secrets too.</li>\n<li><strong>The clock is pinned.</strong> You have run <code>date</code> on the server, set the time zone where the scheduler allows it, and kept clear of the hour the clocks change.</li>\n<li><strong>A double run is harmless,</strong> or the script records what it already did.</li>\n<li><strong>A check-in runs on another machine,</strong> with a grace time longer than the job takes.</li>\n</ol>\n<p><a href=\"https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/\">GitLab’s response</a> in 2017 points the same way. Among its follow-up work it listed “Prometheus monitoring for backups”, and it started working on a public backup monitoring dashboard. The backup script was never the hard part. Knowing whether it had worked was.</p>\n<h2 id=\"sources\">Sources</h2>\n<ul>\n<li><a href=\"https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/\">Postmortem of database outage of January 31</a>, GitLab, 2017</li>\n<li><a href=\"https://aws.amazon.com/message/41926/\">Summary of the Amazon S3 Service Disruption in the Northern Virginia (US-EAST-1) Region</a>, AWS, 2017</li>\n<li><a href=\"https://sre.google/sre-book/monitoring-distributed-systems/\">Monitoring Distributed Systems</a>, Rob Ewaschuk, in Google’s Site Reliability Engineering</li>\n<li><a href=\"https://healthchecks.io/docs/\">Healthchecks.io documentation</a>, <a href=\"https://healthchecks.io/docs/monitoring_cron_jobs/\">cron guide</a> and <a href=\"https://healthchecks.io/pricing/\">pricing</a></li>\n<li>cronie manual pages: <a href=\"https://man7.org/linux/man-pages/man5/crontab.5.html\">crontab(5)</a> and <a href=\"https://man7.org/linux/man-pages/man8/cron.8.html\">cron(8)</a>; <a href=\"https://man7.org/linux/man-pages/man8/anacron.8.html\">anacron(8)</a></li>\n<li><a href=\"https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/\">CronJob</a>, Kubernetes documentation</li>\n<li><a href=\"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows\">Events that trigger workflows: schedule</a>, GitHub Docs</li>\n<li><a href=\"https://www.rfc-editor.org/rfc/rfc4180\">RFC 4180</a>, CSV format, and <a href=\"https://www.rfc-editor.org/rfc/rfc3339\">RFC 3339</a>, date and time on the internet</li>\n<li><a href=\"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString\">Date.prototype.toISOString()</a>, MDN</li>\n<li><a href=\"https://nodejs.org/docs/latest-v22.x/api/process.html#event-uncaughtexception\">process: uncaughtException</a>, Node.js documentation</li>\n<li><a href=\"https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/\">Sending messages using incoming webhooks</a>, Slack</li>\n</ul>\n",
      "summary": "In a 34-line invoice script, two lines find the late invoices; the rest of the work is making sure a failure reaches a person, as GitLab learned in 2017.",
      "image": "https://28creation.com/og/tiny-automation-in-code.jpg",
      "date_published": "2026-10-18T20:30:00.000Z",
      "date_modified": "2026-10-18T20:30:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools",
        "Automation"
      ]
    },
    {
      "id": "https://28creation.com/notes/core-web-vitals-thresholds/",
      "url": "https://28creation.com/notes/core-web-vitals-thresholds/",
      "external_url": "https://web.dev/articles/vitals",
      "title": "The Core Web Vitals thresholds, from the source",
      "content_html": "<p>The actual numbers, straight from Google’s web.dev: Largest Contentful Paint within 2.5 seconds, Interaction to Next Paint at 200 milliseconds or less, Cumulative Layout Shift at 0.1 or less. A page passes when it hits all three at the 75th percentile of page loads, measured separately for mobile and desktop. Keep it open for the next time someone wins a performance argument with one test run.</p>\n<p>Link: <a href=\"https://web.dev/articles/vitals\">web.dev</a></p>",
      "summary": "The actual numbers, straight from Google's web.dev: Largest Contentful Paint within 2.5 seconds, Interaction to Next Paint at 200 milliseconds or less, Cumulative Layout Shift at 0.1 or less. A page passes when it hits all three at the 75th percentile of page loads, measured…",
      "date_published": "2026-10-18T20:15:00.000Z",
      "date_modified": "2026-10-18T20:15:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    },
    {
      "id": "https://28creation.com/notes/pagefind-static-search/",
      "url": "https://28creation.com/notes/pagefind-static-search/",
      "external_url": "https://pagefind.app/",
      "title": "Pagefind adds search to a static site without a search server",
      "content_html": "<p>Pagefind indexes your built HTML after the build and drops a static search bundle into the output, so search runs in the visitor’s browser and there is no search server to keep alive. The index is split into chunks, so a search only downloads the pieces it needs. This site’s search runs on it.</p>\n<p>Link: <a href=\"https://pagefind.app/\">pagefind.app</a></p>",
      "summary": "Pagefind indexes your built HTML after the build and drops a static search bundle into the output, so search runs in the visitor's browser and there is no search server to keep alive. The index is split into chunks, so a search only downloads the pieces it needs. This site's…",
      "date_published": "2026-10-18T20:00:00.000Z",
      "date_modified": "2026-10-18T20:00:00.000Z",
      "authors": [
        {
          "name": "28creation Desk",
          "url": "https://28creation.com/authors/desk/"
        }
      ],
      "tags": [
        "Developer tools"
      ]
    }
  ]
}
