Armilla AI is a rapidly scaling, venture backed AI startup on a mission to enable enterprises to safely deploy cutting edge AI. Our products sit at the intersection of Trustworthy AI, predictive and generative AI model evaluation and insurance. Armilla empowers enterprises to detect, mitigate and transfer AI risk through industry-leading third party verification and warranties for AI-powered products. We work with AI scale ups and large enterprises in financial services, insurance, healthcare, human resources, legal services, manufacturing and beyond.We’ve graduated from some of the top AI accelerators in the world, including Creative Destruction Lab, Y-Combinator (W22), Lloyd’s Lab, Betaworks’ AI Camp and are backed by leading reinsurers Swiss Re, Greenlight Re and Chaucer. We were recently named to FinTech Global's Insurtech100 as well as InsureTechConnect's Forward50, alongside the most exciting insurtech startups in the world. If you’re passionate about responsible AI and building a platform for rapid evaluation of AI models then come join a fast paced environment with seasoned entrepreneurs.

‍Working at Armilla AI 
‍Armilla AI is located at the Carpet Factory in Liberty Village in Toronto with a hybrid work environment. Our core market is North America so we primarily operate in the Eastern Time Zone. You will work in a highly collaborative environment with AI/ML scientists, engineers and product development professionals with opportunities to attend industry or academic conferences.

‍About the role 
‍AI assessments at Armilla are both a qualitative and quantitative exercise. Leveraging global frameworks and best practice, we combine the assessment of an organization’s Responsible AI policies and practices with state-of-the-art model testing and validation practices, to measure AI risk and compliance against business and legal requirements. In addition, trustworthy AI is an active area of academic research and the senior machine learning scientist will keep abreast of the domain and if possible actively participate in building Armilla intellectual property via patents and publications. The senior machine learning scientist will also actively work with the product and engineering team to inform the build of a scalable platform for underwriting AI risks. We are seeking a machine learning scientist experienced in conducting socio-technical evaluations of companies’ AI governance frameworks and products, who is excited about driving the evolution of Armilla’s qualitative and quantitative AI assessment capabilities, product and participation in AI research and development.

‍Role Responsibilities 
> Support client model assessment projects from start to finish, including advising on the selection and development of test plans and evaluation metrics suited to the business use case and regulatory requirements.
> Play a lead role in drafting model assessment reports for clients, including addressing the broader compliance and/or business impacts of assessment technical findings.
> Develop and document assessment team standard operating procedures with a view to improving project efficiency and customer satisfaction Serve as one of the main points of contact for technical questions related to model assessment approaches and findings.
> Work with the Chief Technology and Product Officer and assessment team to identify state-of-the-art techniques, methods and tools for model testing, evaluation and validation of AI/ML systems, including generative AI.
> Participate in active research and development of trustworthy AI.
> Work with engineering and product team to inform development of a scalable platform for underwriting AI risks.
> Help translate emerging requirements in AI regulations and standards into technical assessment capabilities Research and draft blogs, articles, reports, and/or research papers relating the findings of model assessments in collaboration with Armilla colleagues.
> Other related assignments, upon request. 

‍What we’re looking for 
> Phd or Master’s level degree in computer science, machine learning, engineering or related fields.
> 2-5 years of experience working as a data/AI scientist developing AI models and evaluation of AI/LLM models. Experience on issues related to AI regulations, policy and standards with a focus on data quality, performance, fairness, interpretability, robustness and security will be beneficial. 
> Experience applying both technical and non-technical (policy, ethical, social) approaches to model risk and impact assessments.
> Familiarity with requirements contained in AI regulations as well as leading standards and frameworks (EU AI Act, NIST AI Risk Management Framework, ISO Standards, etc).
> Excellent organizational and project management skills, including the ability to lead a small team on client-facing projects.
> Strong technical, analytical and conceptual problem-solving skills.
> Demonstrated experience drafting reports and academic publications.
> Team player.
> Comfortable in a flexible and evolving environment.
> Ability to multitask.

‍What’s in it for you
> Small team offers exposure to all aspects of a growing startup.
> Ability to work alongside serial entrepreneurs.
> Competitive salary.
> Generous health, dental, and vision insurance.
> Flexible working hours.
> Flexible vacation time.
> Hybrid/ Remote friendly.
> Regular knowledge sharing session with the team.
> Training and capability in building opportunities.

‍Salary & Benefits
> Our salary ranges are based on paying competitively for our size and industry and are one part of our total compensation package that also includes benefits and other perks.
> We also include stock options in all compensation packages and believe all Armilla employees should have the opportunity to become owners in the company.
> The base pay range provided is subject to change and may be modified in the future and individual decisions are based on a number of factors, including qualifications for the role, experience level, skill set, location, and business need.

Armilla AI is a rapidly scaling, venture backed AI startup on a mission to enable enterprises to safely deploy cutting edge AI. Our products sit at the intersection of Trustworthy AI, predictive and generative AI model evaluation and insurance. Armilla empowers enterprises to detect, mitigate and transfer AI risk through industry-leading third party verification and warranties for AI-powered products. We work with AI scale ups and large enterprises in financial services, insurance, healthcare, human resources, legal services, manufacturing and beyond.We’ve graduated from some of the top AI accelerators in the world, including Creative Destruction Lab, Y-Combinator (W22), Lloyd’s Lab, Betaworks’ AI Camp and are backed by leading reinsurers Swiss Re, Greenlight Re and Chaucer. We were recently named to FinTech Global's Insurtech100 as well as InsureTechConnect's Forward50, alongside the most exciting insurtech startups in the world. If you’re passionate about responsible AI and building a platform for rapid evaluation of AI models then come join a fast paced environment with seasoned entrepreneurs.

‍Working at Armilla AI 
‍Armilla AI is located at the Carpet Factory in Liberty Village in Toronto with a hybrid work environment. Our core market is North America so we primarily operate in the Eastern Time Zone. You will work in a highly collaborative environment with AI/ML scientists, engineers and product development professionals with opportunities to attend industry or academic conferences.

‍About the role 
‍AI assessments at Armilla are both a qualitative and quantitative exercise. Leveraging global frameworks and best practice, we combine the assessment of an organization’s Responsible AI policies and practices with state-of-the-art model testing and validation practices, to measure AI risk and compliance against business and legal requirements. In addition, trustworthy AI is an active area of academic research and the senior machine learning scientist will keep abreast of the domain and if possible actively participate in building Armilla intellectual property via patents and publications. The senior machine learning scientist will also actively work with the product and engineering team to inform the build of a scalable platform for underwriting AI risks. We are seeking a machine learning scientist experienced in conducting socio-technical evaluations of companies’ AI governance frameworks and products, who is excited about driving the evolution of Armilla’s qualitative and quantitative AI assessment capabilities, product and participation in AI research and development.

‍Role Responsibilities 
> Support client model assessment projects from start to finish, including advising on the selection and development of test plans and evaluation metrics suited to the business use case and regulatory requirements.
> Play a lead role in drafting model assessment reports for clients, including addressing the broader compliance and/or business impacts of assessment technical findings.
> Develop and document assessment team standard operating procedures with a view to improving project efficiency and customer satisfaction Serve as one of the main points of contact for technical questions related to model assessment approaches and findings.
> Work with the Chief Technology and Product Officer and assessment team to identify state-of-the-art techniques, methods and tools for model testing, evaluation and validation of AI/ML systems, including generative AI.
> Participate in active research and development of trustworthy AI.
> Work with engineering and product team to inform development of a scalable platform for underwriting AI risks.
> Help translate emerging requirements in AI regulations and standards into technical assessment capabilities Research and draft blogs, articles, reports, and/or research papers relating the findings of model assessments in collaboration with Armilla colleagues.
> Other related assignments, upon request. 

‍What we’re looking for 
> Phd or Master’s level degree in computer science, machine learning, engineering or related fields.
> 2-5 years of experience working as a data/AI scientist developing AI models and evaluation of AI/LLM models. Experience on issues related to AI regulations, policy and standards with a focus on data quality, performance, fairness, interpretability, robustness and security will be beneficial. 
> Experience applying both technical and non-technical (policy, ethical, social) approaches to model risk and impact assessments.
> Familiarity with requirements contained in AI regulations as well as leading standards and frameworks (EU AI Act, NIST AI Risk Management Framework, ISO Standards, etc).
> Excellent organizational and project management skills, including the ability to lead a small team on client-facing projects.
> Strong technical, analytical and conceptual problem-solving skills.
> Demonstrated experience drafting reports and academic publications.
> Team player.
> Comfortable in a flexible and evolving environment.
> Ability to multitask.

‍What’s in it for you
> Small team offers exposure to all aspects of a growing startup.
> Ability to work alongside serial entrepreneurs.
> Competitive salary.
> Generous health, dental, and vision insurance.
> Flexible working hours.
> Flexible vacation time.
> Hybrid/ Remote friendly.
> Regular knowledge sharing session with the team.
> Training and capability in building opportunities.

‍Salary & Benefits
> Our salary ranges are based on paying competitively for our size and industry and are one part of our total compensation package that also includes benefits and other perks.
> We also include stock options in all compensation packages and believe all Armilla employees should have the opportunity to become owners in the company.
> The base pay range provided is subject to change and may be modified in the future and individual decisions are based on a number of factors, including qualifications for the role, experience level, skill set, location, and business need.