Hi everyone, Petya here, Senior Tech Partnership and Engagement Lead. I am happy to bring to you this third edition of the “HOT Open source Contributor Journeys”. While in the previous two editions - here and here - we focus on highlighting the individual stories, this time I wanted to bring to you our collaboration with a small student- led organisation at the University of California, Berkeley.

Code for Good Berkeley reached to me via email at the start of the year with the opportunity to collaborate and I was so impressed how professional and passionate the team was. We started collaborating with them for three months on two separate projects: OAM Uploader and Street Level Imagery.

Below is the story of the team, including Project Managers Aniket Gupta and Shayla He, along with team members Riya Khasnis, Aryan Das, Michelle Bao, Bryan Huang, Aadhi Balasubramanian, and Takenosuke Nagata, who worked on the AI-assisted street-level imagery project supervised by Sam Woodcock, Senior Tech Lead at HOT. Instead of me telling you more about it, let’s hear it directly from them!

What is the Code for Good program?

Code for Good Berkeley is a student-run organization at the University of California, Berkeley that partners with nonprofits and mission-driven companies around the world. Each semester, interdisciplinary teams of students work closely with a partner to develop technical solutions that address real-world challenges. Our goal is not only to build impactful technology, but also to give our members hands-on experience applying software engineering, AI, and product thinking to meaningful social impact problems.

Can you tell us why you choose HOT for your project? What interested you?

We chose the Humanitarian OpenStreetMap Team (HOT) because of its mission of making geospatial data more accessible for humanitarian response and community development. The opportunity to contribute to an open-source project with real-world impact was very exciting.

Street-level imagery is a rich but also underutilized source of information, and we saw an opportunity to use recent advances in AI to help extract useful insights more efficiently and reduce the amount of manual image review while still keeping humans involved in the decision-making process.

Why street level imagery analysis can be useful?

Street-level imagery gives details that are in many cases difficult or impossible to discern from just satellite imagery. It can provide information about road conditions, accessibility, building characteristics, signage, infrastructure, and other environmental features that are very valuable for mapping and humanitarian disaster response efforts.

What we achieved using Large Language Models

The Vision-Language Model (VLM) team, led by Shayla He, designed and evaluated an end-to-end AI pipeline for automatically generating OpenStreetMap (OSM) road-surface tags from street-level imagery.

Rather than relying on traditional object detection alone, the team benchmarked multiple multimodal foundation models, including Qwen2.5-VL 7B, LLaVA 7B, and GPT-4o-mini, across accuracy, latency, hallucination rate, cost, hardware requirements, and data privacy. The evaluation covered more than 100 images from 35+ countries across 5 continents, including Global South, nighttime, and ambiguous road conditions.

Qwen2.5-VL 7B was selected as the primary recommendation, as it can run locally while delivering the strongest overall performance.

Beyond model selection, the team engineered a structured inference pipeline to make VLM outputs reliable enough for humanitarian mapping:

1. Pre-inference quality gate: Brightness and Laplacian blur filters screened out roughly 11% of low-quality images before inference, reducing unnecessary GPU/API costs.
2. Structured output parsing: The VLM generated standardized road-surface classifications, parsed into a structured JSON schema with fallback validation.
3. Confidence-based routing:
   • Above 0.75: automatically approved for OSM
   • 0.50–0.75: escalated for human review
   • Below 0.50: discarded

1. Hallucination guard: The model first verified that a road was actually present before classifying, preventing false tags on non-road imagery (wilderness scenes, building interiors, satellite images).

This human-in-the-loop architecture reduced erroneous annotations while preserving automation for high-confidence cases. By combining quality filtering, structured outputs, confidence calibration, and human review, the team delivered a scalable workflow that converts unstructured Panoramax imagery into standardized OSM tags suitable for large-scale humanitarian mapping.

What we achieved using Computer Vision?

The Computer Vision team, led by Aniket Gupta, focused on a more traditional object detection approach for street-level imagery: identifying electrical poles and towers to support power infrastructure mapping. Since these features are often visible from the street but not always easy to extract from satellite imagery alone, the team wanted to see how far a lightweight computer vision pipeline could go in turning raw Panoramax images into useful mapping signals.

A big part of the work was building the dataset from the ground up. The team manually labeled a 5,000-image dataset from Panoramax, including 1,777 pole images, 1,075 tower images, and 2,148 negative examples where neither a pole nor tower was present. Bounding boxes around poles and towers were drawn using Roboflow to generate YOLO-style annotation files, and additional imagery was pulled through the Mapillary API to make the dataset more varied.

After the dataset was prepared, the team fine-tuned YOLOv8 models to detect poles and towers in street-level images. The first training setup used a Google Colab T4 GPU, a 70/20/10 train-test-validation split, a batch size of 16, and 512×512 image inputs. The model was fast enough to handle large batches of imagery, and early results showed that tower detection worked especially well, while pole detection proved harder, since poles are usually thinner, smaller, and easier to confuse with trees, wires, signs, or other background objects.

The best-performing model, YOLOv8s, was trained for 50 epochs with rotation and resizing augmentations, reaching an overall mAP50 of about 0.678. On the test set, the model reached an overall mAP50 of 0.721, with tower detection performing strongest at 0.87 mAP50, while pole detection reached 0.572 mAP50. The confusion matrix reflected the same pattern: towers were detected more reliably, while many poles were still missed or confused with the background.

Although the model was not perfect, it gave a clear sense of what computer vision can already do well for humanitarian mapping and where more work is needed. The biggest takeaway was that object detection can help reduce manual review by quickly flagging likely infrastructure features, but it still requires careful dataset design, human review, and targeted improvements for difficult cases such as occluded poles, crowded street scenes, shadows, trees, and utility structures. As part of the handoff, the model weights and the dataset compiled by the team are intended to be open-sourced and made publicly available, so HOT and the broader mapping community can reuse, evaluate, and build on the work beyond this semester.

What did you enjoy the most about collaboration with HOT?

One of the most rewarding parts was working with an organization that has such a strong open-source and humanitarian mission. The HOT team gave us lots of freedom to explore different technical approaches while also helping us understand the practical needs of the mapping community. It was super valuable to receive feedback throughout the project and see how our technical work connected directly to real users and real-world applications.

What is next for you? What advice would you give to other students?

We all plan to continue exploring AI, machine learning, and software engineering, especially in areas where technology can create social impact.

This project showed us how research and open-source collaboration can translate into practical tools that benefit communities around the world. Our advice to other students is to not be intimidated by open-source projects.

You don’t need to know everything before contributing. Find a mission you care about, stay curious, ask questions, and be willing to learn from the community. We have found that open-source projects offer a great opportunity to build technical skills while also creating something that has a lasting impact beyond the classroom.

Ready to get started?

I hope you enjoyed the 3rd edition of the HOT Open Source Contributor Journeym maybe it inspired your student society to contribute to open source?