OneNote Whiteboard Scanner

The problem

A whiteboard is the most useful surface in a meeting and the least durable one. People take photos and the photos sit in a camera roll. The good products that try to fix this are either web services that send your meeting notes to a third party or expensive hardware tied to a specific room.

I wanted something that ran locally, respected the user’s data, and produced output you could actually edit afterwards — not a photograph of a whiteboard but the strokes from a whiteboard, as vectors, in a notebook.

What it does

A webcam captures the whiteboard. The pipeline normalises illumination, corrects perspective, detects strokes by colour, skeletonises and vectorises them, and emits SVG. The output goes to the clipboard and, if the user has connected a Microsoft account, into a specific OneNote notebook section through the Graph API. The whole process happens on the user’s machine. Nothing leaves it.

The pipeline

The image processing is a hybrid. Classical OpenCV stages handle illumination normalisation, perspective correction, and noise reduction — these problems are well-understood and don’t need a neural network. Stroke detection, separation, and vectorisation use neural models where they earn their place, and the pipeline can dispatch them to one of three backends:

• OpenVINO for Intel hardware acceleration
• ONNX with DirectML for general Windows GPUs
• CPU fallback when neither is available

The runtime picks the fastest available backend on startup. The result is that the same binary runs on an entry-level laptop and a workstation, and on the workstation it’s faster.

The desktop shell

The app is React inside Electron, with a Python backend running locally on port 5000. The Electron shell handles the camera access and the OneNote OAuth flow; the Python process handles the heavy image work. They talk over localhost. The packaging script produces a portable build with the Python backend frozen into a standalone executable so end users don’t need a Python installation.

OneNote integration

The Microsoft Graph integration is OAuth 2.0 with the Notes.ReadWrite, Notes.Create, and offline_access scopes. Credentials are stored per-user in the OS credential store, not in the app’s own files. The integration writes Ink Strokes into a notebook section the user selects, which means the output appears as native OneNote ink that can be edited, tagged, and exported through OneNote’s own pipeline.

Why this lives on the Dynamic Reasoning site

This project is the working example of the local-first computer-vision work I do. The interesting choices were architectural: deciding which stages benefit from classical CV and which from a neural model, dispatching across hardware backends, keeping the entire pipeline on the user’s machine because the data shouldn’t leave it. If you have a workflow that needs to capture, clean up, and route physical-world input — documents, whiteboards, forms, handwriting — this is the shape of the engagement.