Evidence-Constrained Visual Question Answering: A Diagnostic Study of Grounding and Selective Prediction

المؤلفون

  • Hamsa M. Ahmed المؤلف
  • Shokhan M. Al-Barzinji المؤلف

DOI:

https://doi.org/10.59992/IJCI.2026.v5n11p6

الكلمات المفتاحية:

Visual Question Answering، Multimodal Artificial Intelligence، Visual Grounding، Object Hallucination، Selective Prediction، Evidence Verification، Reproducibility

الملخص

Inspectable visual evidence does not necessarily indicate that an evidence-based acceptance rule is more reliable in prediction. The aim of this study is to measure that difference in binary object-presence visual question answering. It is accompanied with an independent object detector providing localized evidence for positive answers, and an open multimodal generator, SmolVLM-256M-Instruct. The number of positive candidates accepted depends on a development-selected threshold, while negative candidates abstain because lack of detection cannot establish absence. The exploratory pilot resulted in a new replication with 60 new COCO val2017 images and 360 new adversarial POPE questions, where the images were split into development/evaluation sets. 68.75% reference-label agreement on 240 evaluation questions was obtained using direct generation. The evidence gate accepted 39 answers at 16.25% coverage with 100.00% reference agreement, while positive-only confidence selection achieved 97.44% agreement at the same accepted count. The gate had a higher accepted-answer accuracy point estimate than positive-only confidence selection, but the exploratory difference interval included zero; superiority was not established. Also reported in this study are threshold sensitivity, positive-answer retention, image-cluster bootstrap intervals and checks on component training provenance. The results apply to just one backbone and a limited object-presence task, and do not establish semantic evidence sufficiency, causal visual dependence or general hallucination mitigation. The contribution is a reproducible diagnostic case study that demonstrates how answer-class composition and component training provenance should be controlled when assessing the evidence-constrained visual answering problem.

السير الشخصية للمؤلفين

  • Hamsa M. Ahmed

    Department of Islamic Finance and Banking Sciences, College of Islamic Sciences, University of Anbar, Iraq

  • Shokhan M. Al-Barzinji

    Department Theory, College of Physical Education and Sports Sciences, University of Anbar, Ramadi, Iraq

المراجع

1. Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015). VQA: Visual Question Answering. Proceedings of ICCV. Primary paper.

2. Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017). Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. Proceedings of CVPR, 6904–6913. Primary paper.

3. Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P. (2018). VizWiz Grand Challenge: Answering Visual Questions from Blind People. Proceedings of CVPR, 3608–3617. Primary paper.

4. Hudson, D. A., & Manning, C. D. (2019). GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. Proceedings of CVPR. Primary paper.

5. Radford, A., et al. (2021). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of ICML, PMLR 139, 8748–8763. Primary paper.

6. Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., & Wen, J.-R. (2023). Evaluating Object Hallucination in Large Vision-Language Models. Proceedings of EMNLP, 292–305. Primary paper and DOI.

7. Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023). Visual Instruction Tuning. Advances in Neural Information Processing Systems, 36. Primary paper.

8. Guan, T., et al. (2024). Hallusion Bench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models. Proceedings of CVPR, 14375–14385. Primary paper.

9. Yin, H., Si, G., & Wang, Z. (2025). Clear Sight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large Language Models. Proceedings of CVPR, 14625–14634. Primary paper.

10. Jo, Y., Bae, S., & Kim, T. (2026). Attention-Space Contrastive Guidance for Efficient Hallucination Mitigation in LVLMs. Proceedings of CVPR Findings, 9706–9715. Primary paper.

11. Marafioti, A., et al. (2025). SmolVLM: Redefining small and efficient multimodal models. arXiv:2504.05299. Technical report. Model card.

12. PyTorch Contributors. Faster R-CNN MobileNet V3 Large 320 FPN. Torchvision documentation. Model and weight specification.

13. LMMs-Lab-Encoder. POPE dataset mirror. Hugging Face Datasets. Original benchmark: Li et al. [6]. Dataset source.

14. COCO Organizers. (2017). COCO 2017 Detection Overview. Official workshop presentation. Train and validation split reorganization.

15. Whitehead, S., Petryk, S., Shakib, V., Gonzalez, J., Darrell, T., Rohrbach, A., & Rohrbach, M. (2022). Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly. Proceedings of ECCV. Primary paper.

16. Eisenschlos, J., Maina, H., Ivetta, G., & Benotti, L. (2024). Selectively Answering Visual Questions. Findings of ACL, 4219–4229. Primary paper and DOI.

17. Srinivasan, T., Hessel, J., Gupta, T., Lin, B. Y., Choi, Y., Thomason, J., & Chandu, K. (2024). Selective “Selective Prediction”: Reducing Unnecessary Abstention in Vision-Language Reasoning. Findings of ACL, 12935–12948. Primary paper and DOI.

18. Kotte, V. (2026). EVICT: Evidence-Sufficiency Verification via Counterfactual Dropout for Visually-Grounded Selective Question Answering. Proceedings of CVPR Workshops, 11251–11259. Primary paper.

التنزيلات

منشور

2026-11-16

إصدار

القسم

المقالات

كيفية الاقتباس

Hamsa M. Ahmed, & Shokhan M. Al-Barzinji. (2026). Evidence-Constrained Visual Question Answering: A Diagnostic Study of Grounding and Selective Prediction. المجلة الدولية للحاسبات والمعلوماتية, 5(11). https://doi.org/10.59992/IJCI.2026.v5n11p6