Key recommendations: designing and developing AI systems responsibly and ethically for vulnerable and marginalized groups 


In the third mentoring session of the Digital Helpdesk for Cities platform, Antoine Congost (IVADO) and Samuel Kohn (Open North) presented the methodology and recommendations from the pilot project in Montréal’s Côte-des-Neiges neighbourhood, to investigate if and how artificial intelligence (AI) can benefit vulnerable and marginalized communities. 

The project demonstrates how the AI risk framework presented in the UN-Habitat’s white paper AI & Cities (2022) can be applied in practice, and provides a valuable example of a people-centered approach to AI and community engagement.

This insightful mentoring session took place on April 24th 2024, with the participation of over 35 participants, representing various stakeholders from around the world.

Recording and materials from the session are available at the end of the post.

Participating experts

Antoine Congost 

Antoine holds a Bachelor’s degree in International Studies and a Master’s degree in Political Science from the University of Montreal. He has been working on AI governance and responsible AI implementation projects for several years.

At IVADO, Canada’s leading consortium for AI research, training and knowledge mobilization, Antoine develops projects aimed at promoting the responsible development and deployment of AI in our societies, in collaboration with the research community and various partners from the public and private sectors.

Samuel Kohn

A graduate of McGill University, Samuel holds a Bachelor’s degree in international development and a Master’s degree in urban planning.

With over 6 years of experience in economic development, specializing in foreign direct investment and consultancy for government organizations globally, Samuel has transitioned his focus to human-centered smart cities and urban innovation processes.

At Open North, he supervises programs and projects focused on open smart cities, digital data partnerships, data governance strategies, and digital literacy and capacity-building.

Key considerations

Urban AI, known as IA Urbaine in French, is led by the Université de Montréal in partnership with the Canadian non-profit organization Open North. Its goal is to investigate if and how artificial intelligence (AI) can benefit vulnerable and marginalized communities in Montréal.

In the first part, Antoine and Samuel shared the methodology and lessons learnt from the pilot project in Montréal’s Côte-des-Neiges neighbourhood during the discovery stage, providing an overview of the needs-assessment approach and methodology, the results of the assessment and links with potential fit-for-purpose urban AI systems to be developed and implemented.

In this second part, they  explored further the results of the neighborhood needs-assessment conducted in 2023 by Open North and the associated opportunities and risks with urban AI systems.

The third and final part, described in this post, provide an overview of the project’s roadmap that was designed by Open North in collaboration with Université de Montréal to ensure the responsible and ethical design of the AI tool envisioned in the project’s action plan: a virtual neighbourhood assistant trained on local datasets to solve local use cases.

Proof of concept, neighborhood virtual assistant (Credits: Samuel Kohn, Jérémy Diaz, Open North)

The action plan - key recommendations

The action plan is meant to be replicable and we believe these key considerations can be useful for any stakeholder that wishes to design and develop urban AI solutions destined to meet the local needs of vulnerable and marginalized groups.

1. Getting started

Before starting the project, make sure you assess the resources needed:

  • Human resources: Assess your team’s skills, time requirements, and training needs.
  • Material resources: Include premises, computer equipment and any other infrastructure required.
  • Non-material resources: Define a clear mission, vision and values to mobilize stakeholders.
  • Financial resources: Make sure you have the necessary budget for the project throughout its lifecycle, from design to implementation, monitoring and maintenance.

Some recommendations:

  • Conduct an early assessment of your resources
  • Secure the necessary funding
  • Build a multidisciplinary team
  • Apply fair hiring principles and encourage local hiring

Next, it may be useful to define a theoretical framework that you can use as an ethical compass throughout the project. This tool should guide decisions and the organization responsible for carrying out the project. For the Urban AI pilot project, we have used the principles of the Montreal Declaration on Responsible AI. But it is, of course, possible to use other references. For example, the Sustainable Development Goals (SDGs), or the 10 ethical principles of UNESCO’s Recommendation on the Ethics of AI.

2. Establish a transparent, inclusive and accountable governance

Transparent, inclusive and accountable governance is essential to building stakeholder trust. This involves communicating transparently about the objectives, data use and responsibilities of all stakeholders in the design and implementation of solutions, and ensuring accountability via monitoring and evaluation mechanisms. For this, we recommend setting up governance committees such as a steering committee, a data governance committee, a technology committee and a community mobilization committee.

A few recommendations:

  • Promote communication between the committees
  • Ensure the diversity of its members
  • Document and justify the decisions made throughout the project
  • Clearly communicate the project’s values and principles to the entire team

3. Formalize strategic partnerships, focusing on local partnerships

To meet the specific needs of neighbourhood residents, it is crucial to identify partnership needs, prospect for potential partners, and formalize agreements that define roles and responsibilities.

In the context of an urban AI project, local partnerships offer benefits such as filling resource gaps and bringing in new skills and perspectives. There are generally two key types of partnerships: technical partners, responsible for developing the AI system or application, and community partners, who strengthen links with the community and can mobilize residents.

A few recommendations:

  • Favour local partners wherever possible
  • Involve partners already committed to the project
  • Regularly evaluate partnerships to improve collaboration

4. Involve the local communities

To ensure the success of the project, it is necessary to establish sustainable structures to involve residents, invest in their training and integrate them into feedback mechanisms. The aim is to adopt a truly collaborative approach, respecting community autonomy and adhering to the principle of “nothing about us without us”. This enables solutions to be tested and validated, cultural sensitivities to be considered, and the evolving needs of populations to be monitored.

It means enhancing access to information, promoting informed consent, investing in digital literacy, and ensuring equitable representation of voices. In neighbourhoods where access to cultural communities might be challenging, additional efforts and strategic partnerships will be required to mobilize these groups.

Some recommendations:

  • Invite non-experts and local residents to contribute to the project
  • Overcome barriers to participation, such as finding translators or interpreters to engage linguistically diverse communities facing language barriers in user testing
  • Identify and support champions of change in the community by ensuring representation of different groups
  • Integrate the project with existing consultation initiatives at the neighbourhood scale

5. Work in an agile, iterative and scalable way

The aim here is to prioritize useful features and use cases adapted to the local context, to develop and test minimum viable products (MVPs) in collaboration with target users, and to apply the objectives of the other themes to reinforce this approach.

In this context, an agile methodology is crucial to adapt to frequent changes in requirements, data and technologies, as well as to meet the evolving needs of populations. Agility means remaining open to change and continually adjusting to user feedback. Rather than committing to the complete design of a solution, it is advisable to follow an iterative approach, developing MVPs to quickly test with users and then adjust according to feedback.

A few recommendations:

  • Prioritize essential needs
  • Establish specific objectives and performance indicators
  • Co-design and prototype MVPs, plan regular testing sessions with users
  • Adapt the solutions according to user feedback

6. Establish a sound data governance and management system

Strong and effective data governance is a crucial aspect of any project involving AI technologies. This involves designing a robust data governance framework, implementing a data management strategy and ensuring compliance with current legal frameworks to protect personal data.

Data quality and relevance are essential in the design of AI systems. For this, once again, an agile approach is recommended to adapt to changes and user needs. Data governance involves making transparent, inclusive and accountable decisions about the production, use and management of data throughout its life cycle.

A data governance framework should include mechanisms for defining decision-making bodies, policies and data management practices, considering best practices adapted to the local context.

A few recommendations:

  • Inform stakeholders of their data rights and responsibilities through a privacy policy, consent process, and complaint mechanism.
  • Protect personal and confidential data through controlled access management, data sharing agreements, cybersecurity standards, and a data retention policy.
  • Ensure data is actionable by maintaining a data registry, standardizing metadata and data, and establishing data quality standards.

7. Promote transparency and open communication

It is important to provide the right information to the right stakeholders in a comprehensible manner. This involves mapping communication and transparency needs, developing a sustainable communication plan and implementing an algorithmic transparency standard.

Transparency and communication efforts must be tailored to the various stakeholders, offering clear explanations of the algorithms used. It is crucial to provide information about the data, the system’s objectives and its limitations, while considering the different needs of designers, end users and regulators.

A few recommendations:

  • Develop a stakeholder map,
  • Embrace open source solutions to promote transparency,
  • Invest in educational initiatives on algorithms,
  • Adopt the algorithmic transparency standard to document project decisions and choices.

8. Ensure universal accessibility and ease of use

Universal accessibility and ease of use are key to reducing digital inequalities and facilitating the adoption of Business to Consumer (B2C) urban AI applications. This means designing interfaces that are user-friendly, intuitive and adaptable to diverse formats, while complying with accessibility standards.

A few recommendations:

  • Improve the visual presentation of digital interfaces
  • Ensure forms are compatible with screen readers
  • Add subtitles and image descriptions, and consider mouse-free navigation options
  • Simplify content and provide user guides for assistance
  • Conduct usability testing with diverse users to ensure optimal accessibility

9. Ensure project monitoring and evaluation

The aim here is to establish a strategy to measure performance and success indicators and regularly communicate results.

Monitoring and evaluation are essential to measure the project’s progress, identify obstacles and ensure that the actions undertaken align with the established needs and objectives. It’s an ongoing process that fosters stakeholder commitment, ensures transparency and identifies areas for improvement.

It is crucial to clearly specify and communicate the measures of success specific to the AI system, ensuring that they reflect the true objectives of the project. This fosters stakeholder confidence and enables actions to be adjusted according to the results obtained.

A few recommendations:

  • Define precise project objectives
  • Regularly publish results
  • Integrate quality assurance and reporting requirements
  • Assess the alignment of practices with ethical principles such as the ones from UNESCO Recommendation on AI, or the Montreal Declaration on Responsible AI

10. Other considerations

Finally, there are three cross-cutting considerations to keep in mind throughout the project lifecycle:

  • Risk management: Carefully identifying and documenting risks, including those specific to digital inequalities and data management, is crucial to taking appropriate action.
  • Reducing digital inequalities: Digital inequalities are a major risk for urban AI projects. The objectives set out in the various themes of the project’s roadmap contribute to reducing these inequalities. It is therefore essential to effectively identify, prevent and mitigate the risks associated with digital inequalities.
  • Business model: To ensure the sustainability of the project, it is necessary to consider a long-term financing plan. While commercialization may be difficult to justify for an ethical project aiming to remain accessible to neighbourhood residents, it is possible to explore alternative business models aligned with the digital common principle, such as providing training or coaching to other organizations or local municipalities.


Jump to

Register and be part of the community of Cities and Digital Rights


Practical guides for digital transformation in Latin America and the world.

A community garden in Côte-des-Neiges (Credits- Jérémy Diaz, Open North)
Montreal - part 2: Can urban AI systems help address local needs of vulnerable groups at the neighborhood-scale?
Montreal - part 1: Exploring urban AI solutions for and with vulnerable communities: a needs-centred approach
Waste Collectors Map
Citizen’s Engagement in Urban Data Initiatives: A typology outlining citizens' engagement in local data initiatives


This is a new platform. Leave your comments here to improve it.