Labs · Experiment 4 of 7 · Street capture, The game, Open data

Measured storefronts from street captures

Started: 8 Nov 2026
Updated: 8 Nov 2026
Status: shipped
Verdict: Supported

Can a few phone photos of a building be turned into a measured facade spec, accurate to centimetres, that the game draws in its own style without publishing a single pixel of the photos?

Play Slopcouver The measured storefront is in Mount Pleasant.

Why

Slopcouver's buildings come from City footprints and heights, and their walls are procedural: plausible, not real. Real facades usually come from street-level photography, which brings people, faces, licence plates and private property into the picture. We wanted a route where photos are only a measuring instrument: they never ship, and what ships is a list of numbers in metres that our own renderer draws.

Hypothesis

From a handful of phone captures with the phone's own pose data, we can (1) solve the camera to well under 1 m of error, (2) measure openings, pilasters, awning and sign band to about ±5 cm relative to each other, and (3) draw them in the game so the wall is recognisable, while (4) shipping no image derived from the photos.

Method

Wall: a single-storey storefront, 27.0 m long and 4.85 m high, in Mount Pleasant. The City's address points fall inside the building's footprint and match the numbers on the wall.
Captures: three phone photos taken 3 s apart from one spot about 25 m east of the wall, with the phone's AR pose (ARKit camera transform) and fused GPS heading saved alongside. The rule for choosing frames is no people and no readable plates.
Pose: rotation from ARKit, turned to match the fused heading; position from the fused latitude and longitude in the game frame. By hand: the wall's corner columns plus two points on the pavement line and two on the parapet line in each photo (about 10 points per photo). Then a joint least-squares fit of camera east, south, heading, eye height and wall height.
Rectify: sample the wall plane at 64 px per metre, each texel from the photo that sees it best. This image is a private working file.
Measure: column and row darkness profiles find the black window frames (heads, sills, mullions); colour profiles find the lighter pilasters and the green awning and sign. Each extent was checked on zoomed crops and converted to wall metres; colours are medians over each element.
Check: draw the spec back as a schematic under the rectified photo, privately, and compare.
Ship: one JSON file, the facade spec, with no images. The game draws it with its procedural facade style.
Tools: Python, NumPy, SciPy least squares, Pillow. Time: about 15 minutes to the rectified wall and 25 more for the spec and the check. Cost: none.

Results

Pose: 2.0 m to 0.43 m

Raw phone pose2.0 m

After the solve0.43 m

RMS fit error of the picked points, in metres along the wall (242 px to 53 px in the photos), worst at the south corner. The solve moved the camera 2.1 m south and 0.2 m east of the phone's own position, turned it 3.3°, and put the eye at 1.66 m. The solved wall height is 4.85 m against 5 m in OpenStreetMap.

The spec

The facade spec contains three wall zones (painted block at each end, textured stucco between), five pilasters, eight openings (six gridded shopfronts each about 2.3–2.8 m wide and 2.3 m high, a recessed glazed door, one 2 × 2 window) with mullion and muntin grids, frame colour, glass tint, sill, head and recess depth; one green shed awning; one sign band; a dark metal parapet coping at 4.72–4.85 m; and details (two bike racks, a soffit light, a hedge, a utility box). Lettering on the sign and awning is marked as present and deliberately not transcribed.

Accuracy	Figure	Basis
Sizes and spacings within the wall	±0.05 m	about 3 px at 64 px/m, plus the flat-wall assumption
Placement of the whole pattern along the wall	±0.4 m	from the 0.43 m RMS pose solve
Scale	about 3%	x pinned to the 27.0 m footprint edge; y from the solved height
Colours	about ±10% luminance	medians from an overcast late-afternoon photo, no white-balance correction (expect dim and warm)
Estimated by eye	—	muntin size, awning projection, recess depths, the utility box

The wall in the game

The storefront in the game at midday, seen from across the street — Midday, from across the street.

The storefront in the game at twilight with the shopfronts lit — Blue hour: the glass tint switches to the measured lit-interior colour.

The storefront from its north end, showing the sign band block — From the north end.

Close view of the gridded shopfronts at twilight — Close up: mullions and the fine muntin grid.

Limitations and threats to validity

One wall. A flat, single-storey, frontal wall is the easy case. Deep recesses, balconies, curved walls and tall buildings will be harder.
Assumptions: one shared camera position for the three photos; the wall is a flat vertical plane; the saved images keep the phone's intrinsics.
The picks are manual. About ten points per photo by eye; the accuracy depends on them.
Colour from an uncorrected overcast photo is a weak measurement.
"Recognisable" is our judgement, not a survey of people who know the building.

Verdict

Supported The pose solve brought the error from 2.0 m to 0.43 m RMS; the spec, drawn back, lines up with the rectified photo within a few pixels for every window, the door, the awning, the sign band and the pilasters; the game draws it, and no image derived from the photos is published. Shipped in the game's Release 142.

Practical application

For the game: real storefronts, street by street, in the game's own look.
For open data: a facade spec is a compact, privacy-safe record of a street wall (openings, materials, signage present) that could sit alongside footprints and heights. Heritage and main-street inventories collect similar facts by hand.
For other cities: any phone with AR pose data can collect the input.
What it would take to scale to a block (about 40 walls): replace the manual picks with detected corner edges and pavement and roof lines (or have the capture app save AR plane anchors and depth); a window and door detector on the rectified wall with results snapped to a grid; colour clustering with white-balance normalisation; sign detection that marks lettering without transcribing it. A person reviews each wall's drawn-back schematic privately, and walls with a pose error over about 0.5 m, or a height that disagrees with LiDAR, are rejected automatically.

Reproducibility

Scripts (Slopcouver repository, private for now): capture_camera.py, solve_pose.py, rectify.py, make_spec.py. The photos themselves stay private and are not part of any release.

Data and credits: building footprints and address points: City of Vancouver Open Data, OGL–Vancouver; building height comparison: OpenStreetMap contributors, ODbL. Game renders: Slopcouver.