On the Design of Decision Tree Visiting Kernels at the Edge

As the adoption of Machine Learning (ML) surges, Internet of Things (IoT) applications increasingly require models to execute directly on edge nodes while still delivering efficient performance. Indeed, on-target inference enables low-latency execution and avoids costly data transmissions to high-end computing systems that potentially expose applications to security attacks. However, this comes at the cost of (i) executing inference algorithms within the tight computational and memory budgets of microcontroller-based IoT devices while (ii) delivering performance comparable to ML models running on full-fledged systems. To address these challenges, this paper investigates the design of Decision Tree (DT)-based inference algorithms for modern microcontrollers, systematically exploiting novel architectural features such as vector extensions and Tightly Coupled Memory (TCM). Through formal analysis and extensive evaluation on both synthetic models and publicly available datasets, conducted on a real target platform, namely an STM32N6, we prove that leveraging these hardware enhancements enables efficient execution of DT ensembles, achieving up to 52.6% latency reduction and 17.7% energy savings compared to state-of-the-art baselines, while preserving full model accuracy.