Comparative Evaluation Methodologies for Algorithm Acceleration on Embedded Platforms: A Structured Review and Reproducible Assessment Framework
DOI:
https://doi.org/10.59992/IJCI.2026.v5n11p3الكلمات المفتاحية:
Algorithm Acceleration، Embedded Computing، FPGA، Mpsoc، Tinyml، Energy Efficiency، Memory Utilization، Performance Benchmarking، Reproducibility، Hardware Evaluationالملخص
Background: Performance results for algorithms accelerated on microcontrollers, field-programmable gate arrays (FPGAs), multiprocessor systems-on-chip (MPSoCs), embedded GPUs, and application-specific accelerators are often reported under different workloads, accuracy requirements, and measurement conditions. Objective: This review examines how published studies measure execution time, throughput, energy, memory use, and hardware resources, and proposes a consistent approach to interpreting comparisons across platforms. Methods: A targeted review of publisher records, research repositories, and benchmarking standards identified 15 representative implementations published between 2015 and 2026. The analysis draws on 52 references, including primary studies, surveys, and methodological sources. Study outcomes were recorded with their associated tasks, platforms, measurement conditions, and level of source verification. Because the search was not an exhaustive database-level systematic search, the article is presented as a structured review rather than a PRISMA-compliant systematic review. Results: Reported outcomes vary substantially in both units and experimental scope. One FPGA CNN implementation reported 61.62 GFLOPS at 100 MHz; FINN reported 21,906 image classifications per second under a stated system power of less than 25 W; and a 2026 microcontroller study reported 38 ms execution time, 387 µJ per inference, and 174 kB of SRAM in a specified configuration. These results cannot be ranked directly without matching workload and measurement conditions. Discussion: Differences in power boundaries, memory definitions, hardware configurations, and baseline optimization limit cross-study interpretation. The proposed Cross-Platform Accelerator Evaluation Protocol (CAEP) provides a sequence of checks for workload equivalence, measurement boundaries, and reporting completeness before numerical comparisons are made. Conclusion: A defensible comparison begins by establishing whether results measure equivalent tasks under sufficiently similar conditions. The proposed protocol requires prospective validation, and a comprehensive systematic review would require a fully documented database search and independent full-text assessment.
المراجع
Page, M. J., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, n71. https://doi.org/10.1136/bmj.n71
[2] Rethlefsen, M. L., et al. (2021). PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Systematic Reviews 10, 39. https://doi.org/10.1186/s13643-020-01542-z
[3] Williams, S., Waterman, A., and Patterson, D. (2009). Roofline: an insightful visual performance model for multicore architectures. Communications of the ACM 52(4), 65-76. https://doi.org/10.1145/1498765.1498785
[4] Reddi, V. J., et al. (2020). MLPerf Inference Benchmark. Proceedings of ISCA 2020. https://doi.org/10.1109/ISCA45697.2020.00045
[5] Banbury, C., et al. (2021). MLPerf Tiny Benchmark. NeurIPS Datasets and Benchmarks Track. https://arxiv.org/abs/2106.07597
[6] MLCommons (2026). MLPerf Tiny v1.4 results and benchmark rules. MLCommons, 7 July 2026. https://mlcommons.org/2026/07/mlperf-tiny-v1-4-results/
[7] Tschand, A., Idgunji, S., Rajan, A. T. R., Kanter, D., and Reddi, V. J. (2026). MLPerf Power: Addressing the AI Power Wall. IEEE Micro, early access. https://doi.org/10.1109/MM.2026.3700126
[8] Sze, V., Chen, Y.-H., Yang, T.-J., and Emer, J. S. (2017). Efficient processing of deep neural networks: a tutorial and survey. Proceedings of the IEEE 105(12), 2295-2329.
https://doi.org/10.1109/JPROC.2017.2761740
[9] Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., and Cong, J. (2015). Optimizing FPGA-based accelerator design for deep convolutional neural networks. ACM/SIGDA International Symposium on FPGAs. https://www.microsoft.com/en-us/research/publication/optimizing-fpga-based-accelerator-design-for-deep-convolutional-neural-networks/
[10] Qiu, J., et al. (2016). Going deeper with embedded FPGA platform for convolutional neural network. ACM/SIGDA International Symposium on FPGAs, 26-35.
https://doi.org/10.1145/2847263.2847265
[11] Ma, Y., Cao, Y., Vrudhula, S., and Seo, J.-S. (2017). Optimizing loop operation and dataflow in FPGA acceleration of deep convolutional neural networks. ACM/SIGDA International Symposium on FPGAs, 45-54. https://doi.org/10.1145/3020078.3021736
[12] Umuroglu, Y., et al. (2017). FINN: a framework for fast, scalable binarized neural network inference. ACM/SIGDA International Symposium on FPGAs. https://arxiv.org/abs/1612.07119
[13] Blott, M., Preusser, T. B., Fraser, N. J., Gambardella, G., O’Brien, K., and Umuroglu, Y. (2018). FINN-R: an end-to-end deep-learning framework for fast exploration of quantized neural networks. ACM Transactions on Reconfigurable Technology and Systems / preprint.
https://arxiv.org/abs/1809.04570
[14] Duarte, J., et al. (2018). Fast inference of deep neural networks in FPGAs for particle physics. Journal of Instrumentation 13, P07027. https://doi.org/10.1088/1748-0221/13/07/P07027
[15] Zhang, X., et al. (2018). DNNBuilder: an automated tool for building high-performance DNN hardware accelerators for FPGAs. ICCAD 2018, Article 56.
https://doi.org/10.1145/3240765.3240801
[16] Sharma, H., Park, J., Mahajan, D., Amaro, E., Kim, J. K., Shao, C., Mishra, A., and Esmaeilzadeh, H. (2016). From high-level deep neural models to FPGAs. 49th IEEE/ACM International Symposium on Microarchitecture. https://doi.org/10.1109/MICRO.2016.7783720
[17] Chen, Y.-H., Krishna, T., Emer, J. S., and Sze, V. (2017). Eyeriss: an energy-efficient reconfigurable accelerator for deep convolutional neural networks. IEEE Journal of Solid-State Circuits 52(1), 127-138. https://doi.org/10.1109/JSSC.2016.2616357
[18] Chen, Y.-H., Yang, T.-J., Emer, J. S., and Sze, V. (2019). Eyeriss v2: a flexible accelerator for emerging deep neural networks on mobile devices. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9(2), 292-308. https://doi.org/10.1109/JETCAS.2019.2910232
[19] Jouppi, N. P., et al. (2017). In-datacenter performance analysis of a tensor processing unit. ISCA 2017, 1-12. https://doi.org/10.1145/3140659.3080246
[20] Lin, J., Chen, W.-M., Lin, Y., Cohn, J., Gan, C., and Han, S. (2020). MCUNet: tiny deep learning on IoT devices. Advances in Neural Information Processing Systems 33. https://papers.neurips.cc/paper_files/paper/2020/hash/86c51678350f656dcc7f490a43946ee5-Abstract.html
[21] Lin, J., Chen, W.-M., Cai, H., Gan, C., and Han, S. (2021). MCUNetV2: memory-efficient patch-based inference for tiny deep learning. Advances in Neural Information Processing Systems 34. https://papers.nips.cc/paper_files/paper/2021/hash/1371bccec2447b5aa6d96d2a540fb401-Abstract.html
[22] Banbury, C., et al. (2021). MicroNets: neural network architectures for deploying TinyML applications on commodity microcontrollers. Proceedings of Machine Learning and Systems 3. https://proceedings.mlsys.org/paper_files/paper/2021/hash/c4d41d9619462c534b7b61d1f772385e-Abstract.html
[23] Lai, L., Suda, N., and Chandra, V. (2018). CMSIS-NN: efficient neural network kernels for Arm Cortex-M CPUs. arXiv:1801.06601 / Arm technical work. https://arxiv.org/abs/1801.06601
[24] David, R., et al. (2021). TensorFlow Lite Micro: embedded machine learning for TinyML systems. Proceedings of Machine Learning and Systems 3.
[25] Chen, T., et al. (2018). TVM: an automated end-to-end optimizing compiler for deep learning. 13th USENIX OSDI. https://www.usenix.org/conference/osdi18/presentation/chen
[26] Zheng, L., et al. (2020). Ansor: generating high-performance tensor programs for deep learning. 14th USENIX OSDI. https://www.usenix.org/system/files/osdi20-zheng.pdf
[27] Novickis, R., Lielāmurs, E., Justs, D. J., Cvetkovs, A., and Ozols, K. (2024). Spatial transformation accelerator with parallel data access scheme for sample reconstruction. Electronics 13(5), 922. https://doi.org/10.3390/electronics13050922
[28] Li, J., Li, T., Chen, R., Shen, G., Zhao, D., Zhang, Q., and Zeng, Y. (2025). Hummingbird: a smaller and faster large language model accelerator on embedded FPGA. IEEE/ACM ICCAD 2025. https://doi.org/10.1109/ICCAD66269.2025.11241002
[29] Huang, M., Shen, A., Li, K., Peng, H., Li, B., and Yu, H. (2025). EdgeLLM: a highly efficient CPU-FPGA heterogeneous edge accelerator for large language models. IEEE Transactions on Circuits and Systems I 72(7), 3352-3365. https://doi.org/10.1109/TCSI.2025.3546256
[30] Alanazi, A. G., Alanazi, H. A., and Albalawi, N. S. (2026). Cross-layer resource optimization for ultra-low-power TinyML inference on ARM Cortex-M microcontrollers. Electronics 15(13), 2918. https://doi.org/10.3390/electronics15132918
[31] Hussein, Y. M., Hassan, R. F., and Chisab, R. F. (2026). FPGA-based reconfigurable SoCs for safety-critical AI inference: a systematic literature review. Electronics 15(12), 2695.
https://doi.org/10.3390/electronics15122695
[32] Cordova-Cardenas, R., Amor, D., and Gutiérrez, Á. (2025). Edge AI in practice: a survey and deployment framework for neural networks on embedded systems. Electronics 14(24), 4877. https://doi.org/10.3390/electronics14244877
[33] Alam, S., Yakopcic, C., Wu, Q., Barnell, M., Khan, S., and Taha, T. M. (2024). Survey of deep learning accelerators for edge and emerging computing. Electronics 13(15), 2988. https://doi.org/10.3390/electronics13152988
[34] Liu, F., Li, H., Hu, W., and He, Y. (2024). Review of neural network model acceleration techniques based on FPGA platforms. Neurocomputing 610, 128511.
https://doi.org/10.1016/j.neucom.2024.128511
[35] Zhang, Z. (2026). A review of embedded artificial intelligence research (2023-2026): technological advancements, representative advances, and future prospects. Micromachines 17(5), 586. https://doi.org/10.3390/mi17050586
[36] Blankestijn, A., Odyurt, U., and Yousefzadeh, A. (2026). Recent developments in Transformer inference deployment on FPGA platforms: a survey. Journal of Systems Architecture 177, 103841. https://doi.org/10.1016/j.sysarc.2026.103841
[37] Silvano, C., et al. (2025). A survey on deep learning hardware accelerators for heterogeneous HPC platforms. ACM Computing Surveys 57(11), Article 286. https://doi.org/10.1145/3729215
[38] Morsy, M. M. (2026). AI-on-chip systems: a cross-layer review of architectures, interconnects, design automation, and embedded intelligence. Electronics 15(12), 2645.
https://doi.org/10.3390/electronics15122645
[39] Huang, Y., Ando, T., Sebastian, A., Chang, M.-F., Yang, J. J., and Xia, Q. (2024). Memristor-based hardware accelerators for artificial intelligence. Nature Reviews Electrical Engineering 1, 286-299. https://doi.org/10.1038/s44287-024-00037-6
[40] Bieszczad, G. (2016). SoC-FPGA embedded system for real-time thermal image processing. MIXDES 2016, 469-473. https://doi.org/10.1109/MIXDES.2016.7529788
[41] Sedeeq, M. Q., Salih, M. H., Yousif, O. F., and Mohammed, N. Q. (2016). The vantage of utilizing FPGAs in the design of an embedded multiprocessor. ARPN Journal of Engineering and Applied Sciences 11(23), 13815-13827.
https://www.arpnjournals.com/jeas/volume_23_2016.htm
[42] Sedeeq, M. Q., Salih, M. H., Yousif, O. F., and Mohammed, N. Q. (2017). Insight on MPSoC environment and its performance related to load balance. ARPN Journal of Engineering and Applied Sciences 12(1). https://www.arpnjournals.com/jeas/volume_01_2017.htm
[43] Salih, M. H., Yousif, O. F., Seddeq, M. Q., and Isam, H. M. (2015). Enhance implementation of FPGA based laser missile frequency jamming system using spatial parallelism. ARPN Journal of Engineering and Applied Sciences 10(20), 9626-9632.
https://www.arpnjournals.org/jeas/research_papers/rp_2015/jeas_1115_2918.pdf
[44] Yousif, O. F., Salih, M. H., Hassnawi, L. A., Albreem, M. A. M., Seddeq, M. Q., and Isam, H. M. (2015). Design and implementation computing unit for laser jamming system using spatial parallelism on FPGA. IEEE ICSIPA 2015, 38-43. https://doi.org/10.1109/ICSIPA.2015.7412160
[45] Ducasse, Q., Cotret, P., Lagadec, L., and Stewart, R. (2021). Benchmarking quantized neural networks on FPGAs with FINN. arXiv:2102.01341. https://arxiv.org/abs/2102.01341
[46] Tan, M., and Le, Q. (2019). EfficientNet: rethinking model scaling for convolutional neural networks. Proceedings of ICML 2019. https://proceedings.mlr.press/v97/tan19a.html
[47] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018). MobileNetV2: inverted residuals and linear bottlenecks. CVPR 2018.
[48] Han, S., Mao, H., and Dally, W. J. (2016). Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1510.00149
[49] Howard, A. G., et al. (2017). MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. https://arxiv.org/abs/1704.04861
[50] MLCommons Tiny Working Group (2026). MLPerf Tiny: benchmarking AI at the edge, v1.4 release commentary. MLCommons official technical communication, 7 July 2026.
https://mlcommons.org/2026/07/mlperf-tiny-v1-4-results/
[51] Grant, M. J., and Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal 26(2), 91-108.
https://doi.org/10.1111/j.1471-1842.2009.00848.x
[52] Amdahl, G. M. (1967). Validity of the single processor approach to achieving large scale computing capabilities. AFIPS Spring Joint Computer Conference.
