Map-free Visual Relocalization:
Metric Pose Relative to a Single Image

ECCV 2024 Workshop & Challenge

The Map-free Visual Relocalization workshop investigates topics related to metric visual relocalization relative to a single reference image instead of relative to a map. This problem is of major importance to many higher level applications, such as Augmented/Mixed Reality, SLAM and 3D reconstruction. It is important now, because both industry and academia are debating whether and how to build HD-maps of the world for those tasks. Our community is working to reduce the need for such maps in the first place.

We host the first Map-free Visual Relocalization Challenge 2024 competition with two tracks: map-free metric relative pose from a single image to a single image (proposed by Arnold et al. in ECCV 2022) and from a query sequence to a single image (new). While the former is a more challenging and thus interesting research topic, the latter represents a more realistic relocalization scenario, where the system making the queries may fuse information from query images and tracking poses over a short amount of time and baseline. We invite papers to be submitted to the workshop.

About the Challenge πŸ”—


We are extending the Map-free benchmark for the challenge with a sequence-based scenario, based on feedback from senior community members. Therefore, the challenge will consist of two tracks:
   1. The original, single query frame to single map frame task published with the ECCV 2022 paper;
   2. A new task with multiple query frames (9) and their mobile device provided, metric tracking poses.

1. Single query frame to a single "map" frame

To recap, the task in the first track consists of from a single query image predict the metric relative pose to a single map image without any further auxiliary information.

2. Multiple query frames (9) to a single "map" frame

The second track is motivated by the observation that a burst of images, capturing small motion, can be recorded while staying true to the map-free scenario: No significant data capture or exploration of the environment.
At the same time, the burst of images allows the application of multi-frame depth estimation and contains strong hints about the scene scale from the IMU sensor on device.
We are creating a second version of the test set and leaderboard for this track.

Stay up to date! πŸ”—


Please register your interest here, so we can keep you notified about news and updates!

Important Dates πŸ”—


Challenge start
  • 21st May, 2024
Workshop paper and extended abstracts submission deadline
  •  2nd August, 2024
Workshop paper and extended abstracts camera-ready deadline
  • 15th August, 2024
Challenge end
  • 23rd August, 2024
Challenge submission method description deadline
  • 27th August, 2024
Challenge winners announcement
  • 30th August, 2024
Workshop date (ECCV'24)
  • 30th September, 2024, AM

Prizes πŸ”—


$6000 in prizes will be divided between the top submissions of the two tracks.
Niantic is also seeking partners from the growing community to co-fund and co-judge the prizes.

Update Naver Labs has kindly agreed to co-sponsor the challenge with $2000!

Call for Papers: ECCV Map-free Visual Relocalization Workshop & Challenge 2024 πŸ”—


We invite submissions of workshop papers and extended abstracts to the ECCV Map-free Visual Relocalization Workshop & Challenge 2024. This workshop aims to advance the field of visual relocalization without relying on pre-built maps. The following topics and related areas are of interest:

  • visual relocalization,
  • feature-matching,
  • pose regression (absolute and relative),
  • depth estimation (monocular and multi-frame),
  • scale estimation,
  • confidence and uncertainty,
  • structure-from-motion

Workshop paper and extended abstract submission deadline: 2nd August, 2024. See also Important Dates.
Sign up through the contact form to stay up to date with future announcements.

Submission Guidelines πŸ”—


1. Workshop paper πŸ”—
2. Extended abstract πŸ”—
  • Deadline: 2nd August, 2024.
  • Extended abstracts are not subject to the ECCV rules, so they can be in any template (e.g., ECCV or CVPR).
  • The maximum length of extended abstracts excluding references is the equivalent of 4 pages in the official CVPR 2024 format.
  • Extended abstracts can include already published or accepted works, as they are meant to provide authors with the possibility to showcase their ongoing research in areas relevant to map-free relocalization.
  • Extended abstracts will not be included in the conference proceedings, do not count as paper publication.
  • However, the authors of accepted extended abstracts will get the opportunity to present their work at the workshop poster session.
  • Examples of extended abstracts: Accepted Extended Abstracts from HANDS @ ECCV 2022 . E.g.,
    Scalable High-fidelity 3D Hand Shape Reconstruction via Graph Frequency Decomposition. Tianyu Luan, Jingjing Meng, Junsong Yuan. [pdf]
  • Please submit at the workshop CMT submission portal.
    Use the "Extended abstract track" category under "Subject areas".
3. Method description (~technical report) πŸ”—
  • Challenge leaderboard entry deadline: 23rd August, 2024.
  • Method description publication deadline: 27th August, 2024.
  • The minimum required for valid competition entries.
  • Accepted workshop papers or accepted extended abstracts also qualify as valid method descriptions, no need to submit more.
  • Must be publicly available on ArXiv or equivalent.
    If there are delays with an ArXiv publication, please send us proof of pending submission.
  • This can be a non-peer reviewed paper in any format (e.g., ECCV or CVPR). The method description has to contain sufficient detail for replication of the leaderboard entry's results.

Competition Requirements πŸ”—


  • Entries must be submitted within the competition time limits: 21st May, 2024 - 23rd August, 2024.
  • Entries in the leaderboard that were submitted before the 21st May, 2024 will not be considered as participating in the 2024 Map-free Visual Relocalization challenge.
  • Entries in the leaderboard must point to a valid method description (as outlined above under "3. Method description").

FAQ πŸ”—


Q:
I would like to participate in the challenge, and I don't have a paper yet for my method. What should I do?
A:
You should submit a workshop paper or an extended abstract to the workshop, or make a method description available on ArXiv.
Upon acceptance, your leaderboard entry should refer to the accepted submission.
If rejected, make sure to have a method description available on ArXiv to keep your leaderboard entry valid.
Make sure your leaderboard entry dates between the competition start and end dates (21st May, 2024 - 23rd August, 2024).

Q:
I would like to participate in the challenge with a method I have already published. What should I do?
A:
Make sure your leaderboard entry dates between the competition start and end dates (21st May, 2024 - 23rd August, 2024).
Your leaderboard entry should point to a valid method description, e.g., the link to your published paper.

Q:
I would like to participate in the challenge with a method I have already published, but I've made some changes to the method specific to the challenge. What should I do?
A:
You probably should submit an extended abstract to the workshop or make a method description available on ArXiv, citing your original work.
Resubmitting your original work without sufficient amount of differences compared to the already submitted or published version might violate the ECCV policy regarding dual submissions.
Make sure your leaderboard entry dates between the competition start and end dates (21st May, 2024 - 23rd August, 2024).
Your leaderboard entry should refer to a valid method description, e.g., the title of the extended abstract or the link to your method description on ArXiv.

Q:
I don't want to participate in the challenge, but I have a paper that has not been published anywhere else and is not currently under review anywhere.
Can I submit it to the workshop?
A:
Yes! Submit it as a workshop paper on the workshop CMT.
We encourage you to update the paper with evaluation on the Niantic Map-free Relocalization Dataset, but it is not required.
If your method can solve the task in the challenge, you should consider also submitting it to the leaderboard.

We look forward to your contributions advancing the field of map-free visual relocalization.
For any questions or clarifications, please contact the workshop organizers.

Speakers πŸ”—


Speaker Image

Jakob Engel, Meta

Talk title: TBC

Jakob Engel joined the Surreal Vision team at Oculus Research in Redmond in 2016, working on the future of 3D-enabled Machine Perception. He did his Bachelor and Master at TU Munich (2009 and 2012), followed up with a PhD at the Computer Vision Group there, headed by Professor Daniel Cremers. He spent 6 months as Intern at Intel Research with Vladlen Koltun, was a Google PhD fellow, and worked with SIEMENS within the Software Campus Initiative. His PhD was about direct visual SLAM, 3D reconstruction, sensor fusion and some cool flying quadrocopter stuff - mostly he developed LSD-SLAM and DSO.

Speaker Image

Simon Lynen, Google (or a member of the VPS team, TBC)

Talk title: TBC

Simon Lynen is a tech lead manager at Google Zurich. His group focuses on providing high precision mobile-phone localization as part of the Visual Positioning Service (VPS). Devices with Google’s augmented reality capabilities can leverage VPS to enable global scale location aware experiences such as ARCore CloudAnchors and GoogleMaps LiveView. Simon earned a doctorate at the Autonomous Systems Lab at ETH Zurich with a focus on visual navigation and localization algorithms for robotics, mobile devices, and autonomous cars. As visiting researcher, Simon contributed central pieces to core algorithms of Google’s ARCore. In talks at TEDx Zurich, ThinkingDigital, Zurich Minds, and scientific conferences, Simon has provided a behind-the-scenes view of the ARCore technology and its latest capabilities.

Speaker Image

Torsten Sattler, CTU Prague

Talk title: TBC

Torsten Sattler is a Senior Researcher at CTU. Before, he was a tenured associate professor at Chalmers University of Technology. He received a PhD in Computer Science from RWTH Aachen University, Germany, in 2014. From Dec. 2013 to Dec. 2018, he was a post-doctoral and senior researcher at ETH Zurich. Torsten has worked on feature-based localization methods [PAMI’17], long-term localization [CVPR’18, ICCV’19, ECCV’20, CVPR’21] (see also the benchmarks at visuallocalization.net), localization on mobile devices [ECCV’14, IJRR’20], and using semantic scene understanding for localization [CVPR’18, ECCV’18, ICCV’19]. Torsten has co-organized tutorials and workshops at CVPR (’14, ’15, ’17-’20), ECCV (’18, ’20), and ICCV (’17, ’19), and was / is an area chair for CVPR (’18, ’22, ’23), ICCV (’21, ’23), 3DV (’18-’21), GCPR (’19, ’21), ICRA (’19, ’20), and ECCV (’20). He was a program chair for DAGM GCPR’20, a general chair for 3DV’22, and will be a program chair for ECCV’24.

Speaker Image

Shubham Tulsiani, Carnegie Mellon University

Talk title: TBC

Shubham Tulsiani is an Assistant Professor at Carnegie Mellon University in the Robotics Institute. Prior to this, he was a research scientist at Facebook AI Research (FAIR). He received a PhD. in Computer Science from UC Berkeley in 2018 where his work was supported by the Berkeley Fellowship. He is interested in building perception systems that can infer the spatial and physical structure of the world they observe. He was the recipient of the Best Student Paper Award in CVPR 2015.

Speaker Image

Victor Adrian Prisacariu, Niantic and Oxford University

Talk title: TBC (opening remarks)

Professor Victor Adrian Prisacariu received the Graduate degree (with first class hons.) in computer engineering from Gheorghe Asachi Technical University, Iasi, Romania, in 2008, and the D.Phil. degree in engineering science from the University of Oxford in 2012.
He continued here first as an EPSRC prize Postdoctoral Researcher, and then as a Dyson Senior Research Fellow, before being appointed an Associate Professor in 2017.
He also co-founded 6D.ai, where he built APIs to help developers augment reality in ways that users would find meaningful, useful and exciting. The 6D.ai SDK used a standard built-in smartphone camera to build a cloud-based, crowdsourced three-dimensional semantic map of the world all in real-time, in the background. 6D.ai was acquired by Niantic in March 2020. He is now Chief Scientist with Niantic.
Victor's research interests include semantic visual tracking, 3-D reconstruction, and SLAM.

Preliminary Schedule πŸ”—


Time Event
09:00-09:10 Welcome introduction by Victor Adrian Prisacariu (Niantic and Oxford University)
09:10-09:40 Invited Talk (Talk 1)
09:40-10:10 Invited Talk (Talk 2)
10:10-10:40 Invited Talk (Talk 3)
10:40-10:55 Coffee break
10:55-11:10 Winner Talk 1 (Track 1)
11:10-11:25 Winner Talk 2 (Track 2)
11:25-11:55 Invited Talk (Talk 4)
11:55-12:10 Roundtable discussion
12:10-12:15 Closing Remarks