Tags
access to information
AI
AIDA
AI governance
AI regulation
Ambush Marketing
artificial intelligence
big data
bill c11
Bill c27
copyright
data governance
data protection
Electronic Commerce
freedom of expression
Geospatial
geospatial data
intellectual property
Internet
internet law
IP
open courts
open data
open government
personal information
pipeda
Privacy
smart cities
trademarks
transparency
|
Teresa Scassa
Friday, 22 November 2024 08:50
Crafting a More Nuanced Right to an Explanation in Administrative Law: A comment on Ali v. Minister of Public Safety and Emergency PreparednessA recent decision of the Federal Court of Canada (Ali v. Minister of Public Safety and Emergency Preparedness) highlights the role of judicial review in addressing automated decision-making. It also prompts reflection on the limits of emerging codified rights to an explanation. In July 2024, Justice Battista overturned a decision of the Refugee Protection Division (RPD) which had vacated the refugee status of the applicant, Mr. Ali. The decision of the RPD was based largely on a photo comparison that the RPD to conclude that Mr. Ali was not a Somali refugee as he had claimed. Rather, they concluded that he was a Kenyan student who had entered Canada on a student visa in 2016, a few months prior to Mr. Ali’s refugee protection claim. Throughout the proceedings the applicant had sought information about how photos of the Kenyan student had been found and matched with his own. He was concerned that facial recognition technology (FRT) – which has had notorious deficiencies when used to identify persons of colour – had been used. In response, the Minister denied the use of FRT, maintaining instead that the photographs had been found and analyzed through a ‘manual process’. A Canadian Border Services agent subsequently provided an affidavit to the effect that “a confidential manual investigative technique was used” (at para 15). The RPD was satisfied with this assurance. It considered that how the photographs had been gathered was irrelevant to their own capacity as a tribunal to decide based on the photographs before them. They concluded that Mr. Ali had misrepresented his identity. On judicial review, Justice Battista found that the importance of the decision to Mr. Ali and the quasi-judicial nature of the proceedings meant that he was owed a high level of procedural fairness. Because a decision of the RPD cannot be appealed, and because the consequences of revocation of refugee status are very serious (including loss of permanent resident status and possible removal from the country), Justice Battista found that “it is difficult to find a process under [the Immigration and Refugee Protection Act] with a greater imbalance between severe consequences and limited recourse” (at para 23). He found that the RPD had breached Mr. Ali’s right to procedural fairness “when it denied his request for further information about the source and methodology used by the Minister in obtaining and comparing the photographs” (at para 28). Justice Battista ruled that, given the potential consequences for the applicant, disclosure of the methods used to gather the evidence against him “had to be meaningful” (at para 33). He concluded that it was unfair for the RPD “to consider the photographic evidence probative enough for revoking the Applicant’s statuses and at the same time allow that evidence to be shielded from examination for reliability” (at para 37). In addition to finding a breach of procedural fairness, Justice Battista also found that the RPD’s decision was unreasonable. He noted that there had been sufficiently credible evidence before the original RPD refugee determination panel to find that Mr. Ali was a Somali national entitled to refugee protection. None of this evidence had been assessed in the decision of the panel that vacated Mr. Ali’s refugee status. Justice Battista noted that “[t]he credibility of this evidence cannot co-exist with the validity of the RPD vacation panel’s decision” (at para 40). He also noted that the applicant had provided an affidavit describing differences between his photo and that of the Kenyan student; this evidence had not been considered in the RPD’s decision, contributing to its unreasonableness. The RPD also dismissed evidence from a Kenyan official that, based on biometric records analysis, there was no evidence that Mr. Ali was Kenyan. Justice Battista noted that this dismissal of the applicant’s evidence was in “stark contrast to its treatment of the Minister’s photographic evidence” (at para 44). The Ali decision and the right to an explanation Ali is interesting to consider in the context of the emerging right to an explanation of automated decision-making. Such a right is codified for the private sector context in the moribund Bill C-27, and Quebec has enacted a right to an explanation for both public and private sector contexts. Such rights would apply in cases where an automated decision system (ADS) has been used (and in the case of Quebec, the decision must be based “exclusively on an automated processing” of personal information. Yet in Ali there is no proof that the decision was made or assisted by an AI technology – in part because the Minister refused to explain their ‘confidential’ process. Further, the ultimate decision was made by humans. It is unclear how a codified right to an explanation would apply if the threshold for the exercise of the right is based on the obvious and/or exclusive use of an ADS. It is also interesting to consider the outcome here in light of the federal Directive on Automated Decision Making (DADM). The DADM, which largely addresses the requirements for design and development of ADS in the federal public sector, incorporates principles of fairness. It applies to “any system, tool, or statistical model used to make an administrative decision or a related assessment about a client”. It defines an “automated decision system” as “[a]ny technology that either assists or replaces the judgment of human decision-makers […].” In theory, this would include the use of automated systems such as FRT that assist in human decision-making. Where and ADS is developed and used, the DADM imposes transparency obligations, which include an explanation in plain language of:
The catch, of course, is that it might be impossible for an affected person to know whether a decision has been made with the assistance of an AI technology, as was the case here. Further, the DADM is not effective at capturing informal or ‘off-the-books’ uses of AI tools. The decision in Ali therefore does two important things in the administrative law context. First, it confirms that – in the case of a high impact decision – the right of the individual to an explanation of how the decision was reached as a matter of procedural fairness. Judicial review thus provides recourse for affected individuals – something that the more prophylactic DADM does not. Second, this right includes an obligation to provide details that could either explain or rule out the use of an automated system in the decisional process. In other words, procedural fairness includes a right to know whether and how AI technologies were used in reaching the contested decision. Mere assertions that no algorithms were used in gathering evidence or in making the decision are insufficient – if an automated system might have played a role, the affected individual is entitled to know the details of the process by which the evidence was gathered and the decision reached. Ultimately, what Justice Battista crafts in Ali is not simply a right to an explanation of automated decision-making; rather, it is a right to the explanation of administrative decision-making processes that account for an AI era. In a context in which powerful computing tools are available for both general and personal use, and are not limited to purpose-specific, carefully governed and auditable in-house systems, the ability to demand an explanation of the decisional process in order to rule out the non-transparent use of AI systems seems increasingly important. Note: The Directive on Automated Decision-Making is currently undergoing its fourth review. You may participate in consultations here.
Published in
Privacy
Tuesday, 20 August 2024 17:36
Investigation into use of personal health information for research leaves core issue unresolvedA 2024 investigation report from the Office of the Ontario Information and Privacy Commissioner (OIPC) highlights the tension between the desire of researchers to access health data on the one hand, and the need to protect patient privacy on the other. The protection of personal health information (PHI) is of great practical importance as misuse of such information can have serious consequences for individuals. Yet there is also a significant autonomy and dignity dimension as well. As patients, we are required to share very personal health information with physicians in order to be treated. The understanding is that when we provide that information, it will be appropriately cared for, and that it will not be used for other purposes without our express consent unless it falls within carefully constrained legislative exceptions. In Ontario, the Personal Health Information Protection Act (PHIPA) provides the basic framework for the protection of PHI. Under PHIPA, those who collect PHI from patients are custodians of that information and have significant legal duties. Custodians must obtain appropriate consent for the collection of PHI; they are obliged to use it only for consented purposes; and they must keep it secure. Because of the strong public interest in medical research – for which good data is essential – PHIPA provides several avenues to support medical research. The first is consent. For research studies that require identifiable individuals to participate and to share their data, researchers can recruit participants and seek their informed consent to the collection and use of their data. Consent is not required if researchers use de-identified data, but they must request access to such data, and must complete a research ethics protocol, which is evaluated by a hospital or university research ethics board (REB). The Ontario government has also created “prescribed entities” under PHIPA. A prescribed entity has authority under the legislation to collect health administrative data as well as other data, to secure and administer it, and to use it for analytic purposes. Pursuant to PHIPA, they have the lawful authority to disclose the PHI to researchers under conditions that also involve research protocols and ethics review. ICES is the leading example of a prescribed entity for analytics and research using and disclosing Ontarian’s PHI. Prescribed entities amass significant quantities of PHI but do so under strict regulatory control. Their privacy and security practices are reviewed every three years by the OIPC, and they must comply with any recommendations made by the OIPC. However, prescribed entities do not meet all needs for health data for research, in spite of their growing patient chart datasets. In addition, there have been concerns raised that access to data in the hands of prescribed entities is cumbersome, although this is in part due to the added requirement for a privacy impact assessment (in addition to a research ethics protocol) as mandated by the OIPC. . It is within this context that the complaint that fueled this investigation into the University of Toronto’s Practice-Based Research Network (UTOPIAN) must be understood. Created and overseen through the Department of Family & Community Medicine at the University of Toronto, UTOPIAN was essentially framed as a research project. The “research” described by the University in its REB application involved the creation of a database of “anonymized patient data from EMRs of primary health care providers”, providing “accessible data options for research and public health surveillance”, and devising “algorithms or other processes to enable automated EMR data collection, data de-identification, and other data processes” (at para 150). The University of Toronto (the University) sought and received ethics approval from its Research Ethics Board (REB). It then collected PHI on a regular and ongoing basis from clinicians in primary care practices in Ontario affiliated with the University of Toronto to create a pool of health data. To obtain the data, UTOPIAN sought the agreement of individual physicians to provide regular downloads of their patient electronic medical record (EMR) data to UTOPIAN. It then provided access to this data for health research to members of the broader U of T health research community. Not just the volume, but also the type of information collected by UTOPIAN increased over time. The investigation report notes that in 2020 the University “significantly increased the extent of the information it uploaded from physicians’ electronic medical records (EMR) systems” (para 5). The information collected included full chart and identifying information although the identifying information was stored separately. An initial complaint to the OIPC was filed by doctors who were aware of but uncomfortable with UTOPIAN. They raised several concerns with the OIPC but sought to remain anonymous out of fear of retribution within the university health network. The OIPC therefore proceeded with the investigation as if it were a commissioner-initiated complaint. The issues for investigation were whether UTOPIAN was properly “research” within the meaning of PHIPA; and if it was, whether it complied with the requirements for research under s. 44 of PHIPA. The investigator began by considering whether, assuming UTOPIAN was research, it had complied with PHIPA requirements. Research projects that use patient data without consent must have a research plan approved by a Research Ethics Board (REB). They must also enter into a research agreement with the REB, and they must comply with all conditions set by the REB. A copy of the research plan and the REB decision approving the plan must be shared with the custodian who is asked to provide data for the project. PHI obtained under such a research plan must only be used for the specified purposes approved by the REB. Researchers must also notify the custodians who provide the data if there has been any breach of the research agreement. The investigator found that the University was in breach of several of its obligations. First, it did not share its research plan with data custodians, nor did it provide copies of updated research plans as the project progressed. Instead, it provided a letter that summarized the project, and that the custodian could sign to indicate agreement. Although the University maintained that copies of the other documentation was available on request, the letter did not specify this. The investigator found that the letter lacked important details, including the end date of the project. While she found the idea of providing a high-level summary commendable, she also found that the other documents should have been appended to the letter, and it should have been clear to custodians that these documents contained additional information. The investigator also found that the UTOPIAN project was changed over time, and while new custodians were asked to sign an updated version of the Provider Letter, there was no new letter sent to existing participant custodians. Instead, they received email notices about changes to the project. Some of these, such as the extraction of the full patient chart, were significant. The investigator found that email notices did not suffice – there had to be express agreement with the new changes. Further, she found that notice was only provided of what the University considered to be the most significant changes. The investigator found that it was not reasonable to consider that sending out emails and assuming consent if no objections were raised was sufficient to constitute agreement. She noted that emails can be overlooked by busy physicians or can even be lost in spam filters. She also disagreed with the University’s characterization of some of the changes as ‘minor’. She found that the University need to ensure that custodians “clearly, unambiguously and unequivocally communicated their acceptance of the proposed amendment to the Provider Agreement rather than relying on silence.” (at para 101). REB approvals for research projects are time-limited and can be renewed. In this case, the REB approval expired in November 2022, but the University continued to collect PHI after that date (a date which had not been provided in the letter to custodians). This collection of PHI was therefore not authorized under PHIPA. The University sent a letter in January 2023 to custodians informing them that there had been an inadvertent uploading of patient data after the expiry of the agreement. Although it destroyed this data, the investigator nevertheless found that this was a significant breach of PHIPA. The investigator also found that there had been an earlier period where the REB approval had been allowed to expire and where data had been collected during the two-month period between its expiry and a new REB approval. That too was a breach of PHIPA. The investigator declined to characterize these breaches as administrative oversights, noting instead that they were “deeply concerning from both a legal and ethical perspective.” (at para 80). She also found that although the University had provided notice of the breach caused by collection of data after the expiry of the agreement in 2022, it had failed to provide notice of the breach that occurred when the agreement lapsed for two months in 2018. This failure to provide notice violated s. 44(6) of PHIPA. The REB had required the University to de-link collected data from identifying information. The investigator reviewed the University’s deidentification practices and found no evidence to suggest there were problems with it. However, she nonetheless recommended that, considering the volume and sensitivity of the data collected, the University should conduct a re-identification study of its UTOPIAN database. The REB had also required the University to conduct site visits to custodians’ offices to ensure that notices were properly provided to patients of the custodians. The investigator found that although site visits had been constrained by the COVID-19 pandemic, the University had not resumed these visits post-pandemic. The REB had required “regular” site visits, and she found that this failure to resume visits did not meet this requirement. Further, she raised concerns about the adequacy of notices posted in physician waiting rooms in a context in which doctors used virtual technologies with many patients. This shift in practice should have prompted a variation to the research plan. The complainants had also raised concerns that deidentified patient data was being sold. The investigator was satisfied that this was not the case. However, she found that these concerns – raised by doctors who had been invited to participate in the project – highlighted a lack of adequate transparency. She noted that the abbreviated form of notice provided by the University “may have contributed to the suspicion and distrust on the part of at least some of the custodians” (at para 135). At the time of the investigation and report, the University had put on hold all its activities in relation to UTOPIAN. Although it had no plans to collect new data, it was developing an REB application in relation to the use of the existing data in the database. The investigator made a series of recommendations to the University to correct its practices in the event that it sought to use the archived UTOPIAN database for research purposes. Up to this point, the investigator’s report raises serious concerns about a project that operated on a large scale. UTOPIAN was a substantial pool of data – the investigator noted that it contained the health data of almost 600,000 Ontarians. However, the most significant issue from a public policy point of view is whether this type of project – which essentially creates a “data safe haven” to use the University’s own words – qualifies as “research” under PHIPA. In other words, the fundamental issue was whether this was an appropriate statutory basis to leverage so as to engage in this type of data sharing. In addition to its research exceptions, PHIPA contains provisions allowing for the creation of “prescribed entities” who are empowered under the legislation to pool data from different sources and to make it available for analytics. Prescribed entities are also permitted to disclosure these data to researchers for research purposes. However, prescribed entities must meet the requirements of s. 45(3) of PHIPA, which mandate close supervision by the OIPC. The investigator noted that UTOPIAN performed functions similar to ICES, a prescribed entity for health data in Ontario, but did so without the same levels of oversight. She observed that using the research provisions of PHIPA “to authorize large-scale research platforms that operate as an ongoing concern, such as UTOPIAN, can lead to many practical difficulties given the awkward fit” (at para 155). Since UTOPIAN was no longer operating at the time of the decision, the investigator ultimately reached no conclusion as to whether it constituted “research”, and she declined to send the matter to adjudication. This is unfortunate given the University’s plans, acknowledged in the report, to seek REB approval to use the data already collected for further research studies. The investigator also noted in a postscript to her report that Queen’s University had applied for and received REB approval to create a similar project Ontario-wide project, called the Primary Care Ontario Practice-based Learning and Research Network (POPLAR). She noted that she had forwarded her decision on the UTOPIAN file to Queen’s University, highlighting for them her reservations about whether this type of project qualified for PHIPA’s research exception. She noted that the OIPC was open to consultation by Queen’s on this issue. The conclusion of the UTOPIAN investigation is thus rather inconclusive. If UTOPIAN was ‘research’, it clearly breached several PHIPA requirements. What is less clear is whether it was ‘research’. If it was not, then there was no legal basis for the collection, hosting and sharing of this data. The investigator avoided making a call on the fundamental legitimacy of this data pooling project because it had ended at the time of the report, even though there appeared to be plans to make use of the already-collected data, and even though the concept had been embraced by another institution with plans for an even larger data pool. As a result, serious issues regarding the pooling of health data for research in Ontario remain unresolved. Seen one way, the OIPC’s invitation to Queen’s University to consult with them regarding POPLAR signals the OIPC’s willingness to explore whether and how complex new proposals designed to enhance health research in Ontario can be reconciled with existing legislation. Seen another way, leveraging the research exception in this way seems to create a clearly inadequate framework for data sharing on this scale. This is evident when compared with the considerable safeguards for privacy and security protection in the case of prescribed entities. If prescribed entities are not meeting the needs of researchers, then perhaps the solution lies in law reform rather than privacy law hacks. What the decision lacks (and could not have been expected to provide) is an analysis of the landscape for health data research in Ontario, an assessment of the existing frameworks and any shortcomings they might have, and proposals to address any issues in a manner that both furthers research goals and protects privacy. This should be the role of government. The investigation report into UTOPIAN – situated within this public policy vacuum – leaves Ontarians with ongoing uncertainty and no clear path forward.
Published in
Privacy
Friday, 07 June 2024 12:58
Submission to Consultation on Ontario's Bill 194: Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024On May 13, 2024, the Ontario government introduced Bill 194. The bill addresses a catalogue of digital issues for the public sector. These include: cybersecurity, artificial intelligence governance, the protection of the digital information of children and youth, and data breach notification requirements. Consultation on the Bill closes on June 11, 2024. Below is my submission to the consultation. The legislature has now risen for the summer, so debate on the bill will not be moving forward now until the fall.
Submission to the Ministry of Public and Business Service Delivery on the Consultation on proposed legislation: Strengthening Cyber Security and Building Trust in the Public Sector Act, 2024 Teresa Scassa, Canada Research Chair in Information Law and Policy, University of Ottawa June 4, 2024 I am a law professor at the University of Ottawa, where I hold the Canada Research Chair in Information Law and Policy. I research and write about legal issues relating to artificial intelligence and privacy. My comments on Bill 194 are made on my own behalf. The Enhancing Digital Security and Trust Act, 2024 has two schedules. Schedule 1 has three parts. The first relates to cybersecurity, the second to the use of AI in the broader public service, and the third to the use of digital technology affecting individuals under 18 years of age in the context of Children’s Aid Societies and School Boards. Schedule 2 contains a series of amendments to the Freedom of Information and Protection of Privacy Act (FIPPA). My comments are addressed to each of the Schedules. Please note that all examples provided as illustrations are my own. Summary Overall, I consider this to be a timely Bill that addresses important digital technology issues facing Ontario’s public sector. My main concerns relate to the sections on artificial intelligence (AI) systems and on digital technologies affecting children and youth. I recommend the addition of key principles to the AI portion of the Bill in both a reworked preamble and a purpose section. In the portion dealing with digital technologies and children and youth, I note the overlap created with existing privacy laws, and recommend reworking certain provisions so that they enhance the powers and oversight of the Privacy Commissioner rather than creating a parallel and potentially conflicting regime. I also recommend shifting the authority to prohibit or limit the use of certain technologies in schools to the Minister of Education and to consider the role of public engagement in such decision-making. A summary of recommendations is found at the end of this document. Schedule 1 - Cybersecurity The first section of the Enhancing Digital Security and Trust Act (EDSTA) creates a framework for cybersecurity obligations that is largely left to be filled by regulations. Those regulations may also provide for the adoption of standards. The Minister will be empowered to issue mandatory Directives to one or more public sector entities. There is little detail provided as to what any specific obligations might be, although section 2(1)(a) refers to a requirement to develop and implement “programs for ensuring cybersecurity” and s. 2(1)(c) anticipates requirements on public sector entities to submit reports to the minister regarding cyber security incidents. Beyond this, details are left to regulations. These details may relate to roles and responsibilities, reporting requirements, education and awareness measures, response and recovery measures, and oversight. The broad definition of a “public sector entity” to which these obligations apply includes hospitals, school boards, government ministries, and a wide range of agencies, boards and commissions at the provincial and municipal level. This scope is important, given the significance of cybersecurity concerns. Although there is scant detail in Bill 194 regarding actual cyber security requirements, this manner of proceeding seems reasonable given the very dynamic cybersecurity landscape. A combination of regulations and standards will likely provide greater flexibility in a changeable context. Cybersecurity is clearly in the public interest and requires setting rules and requirements with appropriate training and oversight. This portion of Bill 194 would create a framework for doing this. This seems like a reasonable way to address public sector cybersecurity, although, of course, the effectiveness will depend upon the timeliness and the content of any regulations. Schedule 1 – Use of Artificial Intelligence Systems Schedule 1 of Bill 194 also contains a series of provisions that address the use of AI systems in the public sector. These will apply to AI systems that meet a definition that maps onto the Organization for Economic Co-operation and Development (OECD) definition. Since this definition is one to which many others are being harmonized (including a proposed amendment to the federal AI and Data Act, and the EU AI Act), this seems appropriate. The Bill goes on to indicate that the use of an AI system in the public sector includes the use of a system that is publicly available, that is developed or procured by the public sector, or that is developed by a third party on behalf of the public sector. This is an important clarification. It means, for example, that the obligations under the Act could apply to the use of general-purpose AI that is embedded within workplace software, as well as purpose-built systems. Although the AI provisions in Bill 194 will apply to “public service entities” – defined broadly in the Bill to include hospitals and school boards as well as both federal and municipal boards, agencies and commissions – the AI provisions will only apply to a public sector entity that is “prescribed for the purposes of this section if they use or intend to use an artificial intelligence system in prescribed circumstances” (s. 5(1)). The regulations also might apply to some systems (e.g., general purpose AI) only when they are being used for a particular purpose (e.g., summarizing or preparing materials used to support decision-making). Thus, while potentially quite broad in scope, the actual impact will depend on which public sector entities – and which circumstances – are prescribed in the regulations. Section 5(2) of Bill 194 will require a public sector entity to which the legislation applies to provide information to the public about the use of an AI system, but the details of that information are left to regulations. Similarly, there is a requirement in s. 5(3) to develop and implement an accountability framework, but the necessary elements of the framework are left to regulations. Under s. 5(4) a public sector entity to which the Act applies will have to take steps to manage risks in accordance with regulations. It may be that the regulations will be tailored to different types of systems posing different levels of risk, so some of this detail would be overwhelming and inflexible if included in the law itself. However, it is important to underline just how much of the normative weight of this law depends on regulations. Bill 194 will also make it possible for the government, through regulations, to prohibit certain uses of AI systems (s. 5(6) and s. 7(f) and (g)). Interestingly, what is contemplated is not a ban on particular AI systems (e.g., facial recognition technologies (FRT)); rather, it is potential ban on particular uses of those technologies (e.g., FRT in public spaces). Since the same technology can have uses that are beneficial in some contexts but rights-infringing in others, this flexibility is important. Further, the ability to ban certain uses of FRT on a province-wide basis, including at the municipal level, allows for consistency across the province when it comes to issues of fundamental rights. Section 6 of the bill provides for human oversight of AI systems. Such a requirement would exist only when a public entity uses an AI system in circumstances set out in the regulations. The obligation will require oversight in accordance with the regulations and may include additional transparency obligations. Essentially, the regulations will be used to customize obligations relating to specific systems or uses of AI for particular purposes. Like the cybersecurity measures, the AI provisions in Bill 194 leave almost all details to regulations. Although I have indicated that this is an appropriate way to address cybersecurity concerns, it may be less appropriate for AI systems. Cybersecurity is a highly technical area where measures must adapt to a rapidly evolving security landscape. In the cybersecurity context, the public interest is in the protection of personal information and government digital and data infrastructures. Risks are either internal (having to do with properly training and managing personnel) or adversarial (where the need is for good security measures to be in place). The goal is to put in place measures that will ensure that the government’s digital systems are robust and secure. This can be done via regulations and standards. By contrast, the risks with AI systems will flow from decisions to deploy them, their choice and design, the data used to train the systems, and their ongoing assessment and monitoring. Flaws at any of these stages can lead to errors or poor functioning that can adversely impact a broad range of individuals and organizations who may interact with government via these systems. For example, an AI chatbot that provides information to the public about benefits or services, or an automated decision-making system for applications by individuals or businesses for benefits or services, interacts with and impacts the public in a very direct way. Some flaws may lead to discriminatory outcomes that violate human rights legislation or the Charter. Others may adversely impact privacy. Errors in output can lead to improperly denied (or allocated) benefits or services, or to confusion and frustration. There is therefore a much more direct impact on the public, with effects on both groups and individuals. There are also important issues of transparency and trust. This web of considerations makes it less appropriate to leave the governance of AI systems entirely to regulations. The legislation should, at the very least, set out the principles that will guide and shape those regulations. The Ministry of Public and Business Service Delivery has already put considerable work into developing a Trustworthy AI Framework and a set of (beta) principles. This work could be used to inform guiding principles in the statute. Currently, the guiding principles for the whole of Bill 194 are found in the preamble. Only one of these directly relates to the AI portion of the bill, and it states that “artificial intelligence systems in the public sector should be used in a responsible, transparent, accountable and secure manner that benefits the people of Ontario while protecting privacy”. Interestingly, this statement only partly aligns with the province’s own beta Principles for Ethical Use of AI. Perhaps most importantly, the second of these principles, “good and fair”, refers to the need to develop systems that respect the “rule of law, human rights, civil liberties, and democratic values”. Currently, Bill 194 is entirely silent with respect to issues of bias and discrimination (which are widely recognized as profoundly important concerns with AI systems, and which have been identified by Ontario’s privacy and human rights commissioners as a concern). At the very least, the preamble to Bill 194 should address these specific concerns. Privacy is clearly not the only human rights consideration at play when it comes to AI systems. The preamble to the federal government’s Bill C-27, which contains the proposed Artificial Intelligence and Data Act, states: “that artificial intelligence systems and other emerging technologies should uphold Canadian norms and values in line with the principles of international human rights law”. The preamble to Bill 194 should similarly address the importance of human rights values in the development and deployment of AI systems for the broader public sector. In addition, the bill would benefit from a new provision setting out the purpose of the part dealing with public sector AI. Such a clause would shape the interpretation of the scope of delegated regulation-making power and would provide additional support for a principled approach. This is particularly important where legislation only provides the barest outline of a governance framework. In this regard, this bill is similar to the original version of the federal AI and Data Act, which was roundly criticized for leaving the bulk of its normative content to the regulation-making process. The provincial government’s justification is likely to be similar to that of the federal government – it is necessary to remain “agile”, and not to bake too much detail into the law regarding such a rapidly evolving technology. Nevertheless, it is still possible to establish principle-based parameters for regulation-making. To do so, this bill should more clearly articulate the principles that guide the adoption and use of AI in the broader public service. A purpose provision could read: The purpose of this Part is to ensure that artificial intelligence systems adopted and used by public sector entities are developed, adopted, operated and maintained in manner that is transparent and accountable and that respects the privacy and human rights of Ontarians. Unlike AIDA, the federal statute which will apply to the private sector, Bill 194 is meant to apply to the operations of the broader public service. The flexibility in the framework is a recognition of both the diversity of AI systems, and the diversity of services and activities carried out in this context. It should be noted, however, that this bill does not contemplate any bespoke oversight for public sector AI. There is no provision for a reporting or complaints mechanism for members of the public who have concerns with an AI system. Presumably they will have to complain to the department or agency that operates the AI system. Even then, there is no obvious requirement for the public sector entity to record complaints or to report them for oversight purposes. All of this may be provided for in s. 5(3)’s requirement for an accountability framework, but the details of this have been left to regulation. It is therefore entirely unclear from the text of Bill 194 or what recourse – if any – the public will have when they have problematic encounters with AI systems in the broader public service. Section 5(3) could be amended to read: 5(3) A public sector entity to which this section applies, shall, in accordance with the regulations, develop and implement an accountability framework respecting their use of the artificial intelligence system. At a minimum, such a framework will include: a) The specification of reporting channels for internal or external complaints or concerns about the operation of the artificial intelligence system; b) Record-keeping requirements for complaints and concerns raised under subparagraph 5(3)(a), as well as for responses thereto. Again, although a flexible framework for public sector AI governance may be an important goal, key elements of that framework should be articulated in the legislation. Schedule 1 – Digital Technology Affecting Individuals Under Age 18 The third part of Schedule 1 addresses digital technology affecting individuals under age 18. This part of Bill 194 applies to children’s aid societies and school boards. Section 9 enables the Lieutenant Governor in Council to make regulations regarding “prescribed digital information relating to individuals under age 18 that is collected, used, retained or disclosed in a prescribed manner”. Significantly, “digital information” is not defined in the Bill. The references to digital information are puzzling, as it seems to be nothing more than a subset of personal information – which is already governed under both the Municipal Freedom of Information and Protection of Privacy Act (MFIPPA) and FIPPA. Personal information is defined in both these statutes as “recorded information about an identifiable individual”. It is hard to see how “digital information relating to individuals under age 18” is not also personal information (which has received an expansive interpretation). If it is meant to be broader, it is not clear how. Further, the activities to which this part of Bill 194 will apply are the “collection, use, retention or disclosure” of such information. These are activities already governed by MFIPPA and FIPPA – which apply to school boards and children’s aid societies respectively. What Bill 194 seems to add is a requirement (in s. 9(b)) to submit reports to the Minister regarding the collection, use, retention and disclosure of such information, as well as the enablement of regulations in s. 9(c) to prohibit collection, use, retention or disclosure of prescribed digital information in prescribed circumstances, for prescribed purposes, or subject to certain conditions. Nonetheless, the overlap with FIPPA and MFIPPA is potentially substantial – so much so, that s. 14 provides that in case of conflict between this Act and any other, the other Act would prevail. What this seems to mean is that FIPPA and MFIPPA will trump the provisions of Bill 194 in case of conflict. Where there is no conflict, the bill seems to create an unnecessary parallel system for governing the personal information of children. The need for more to be done to protect the personal information of children and youth in the public school system is clear. In fact, this is a strategic priority of the current Information and Privacy Commissioner (IPC), whose office has recently released a Digital Charter for public schools setting out voluntary commitments that would improve children’s privacy. The IPC is already engaged in this area. Not only does the IPC have the necessary expertise in the area of privacy law, the IPC is also able to provide guidance, accountability and independent oversight. In any event, since the IPC will still have oversight over the privacy practices of children’s aid societies and school boards notwithstanding Bill 194, the new system will mean that these entities will have to comply with regulations set by the Minister on the one hand, and the provisions of FIPPA and MFIPPA on the other. The fact that conflicts between the two regimes will be resolved in favour of privacy legislation means that it is even conceivable that the regulations could set requirements or standards that are lower than what is required under FIPPA or MFIPPA – creating an unnecessarily confusing and misleading system. Another odd feature of the scheme is that Bill 194 will require “reports to be submitted to the Minister or a specified individual in respect of the collection, use, retention and disclosure” of digital information relating to children or youth (s. 9(b)). It is possible that the regulations will specify that it is the Privacy Commissioner to whom the reports should be submitted. If it is, then it is once again difficult to see why a parallel regime is being created. If it is not, then the Commissioner will be continuing her oversight of privacy in schools and children’s aid societies without access to all the relevant data that might be available. It seems as if Bill 194 contemplates two separate sets of measures. One addresses the proper governance of the digital personal information of children and youth in schools and children’s aid societies. This is a matter for the Privacy Commissioner, who should be given any additional powers she requires to fulfil the government’s objectives. Sections 9 and 10 of Bill 194 could be incorporated into FIPPA and MFIPPA, with modifications to require reporting to the Privacy Commissioner. This would automatically bring oversight and review under the authority of the Privacy Commissioner. The second objective of the bill seems to be to provide the government with the opportunity to issue directives regarding the use of certain technologies in the classroom or by school boards. This is not unreasonable, but it is something that should be under the authority of the Minister of Education (not the Minister of Public and Business Service Delivery). It is also something that might benefit from a more open and consultative process. I would recommend that the framework be reworked accordingly. Schedule 2: FIPPA Amendments Schedule 2 consists of amendments to the Freedom of Information and Protection of Privacy Act. These are important amendments that will introduce data breach notification and reporting requirements for public sector entities in Ontario that are governed by FIPPA (although, interestingly, not those covered by MFIPPA). For example, a new s. 34(2)(c.1) will require the head of an institution to include in their annual report to the Commissioner “the number of thefts, losses or unauthorized uses or disclosures of personal information recorded under subsection 40.1”. The new subsection 40.1(8) will require the head of an institution to keep a record of any such data breach. Where a data breach reaches the threshold of creating a “real risk that a significant harm to an individual would result” (or where any other circumstances prescribed in regulations exist), a separate report shall be made to the Commissioner under s. 40.1(1). This report must be made “as soon as feasible” after it has been determined that the breach has taken place (s. 40.1(2)). New regulations will specify the form and contents of the report. There is a separate requirement for the head of the institution to notify individuals affected by any breach that reaches the threshold of a real risk of significant harm (s. 40.1(3)). The notification to the individual will have to contain, along with any prescribed information, a statement that the individual is entitled to file a complaint with the Commissioner with respect to the breach, and the individual will have one year to do so (ss. 40.1(4) and (5)). The amendments also identify the factors relevant in determining if there is a real risk of significant harm (s. 40.1(7)). The proposed amendments also provide for a review by the Commissioner of the information practices of an institution where a complaint has been filed under s. 40.1(4), or where the Commissioner “has other reason to believe that the requirements of this Part are not being complied with” (s. 49.0.1).) The Commissioner can decide not to review an institution’s practices in circumstances set out in s. 49.0.1(3). Where the Commissioner determines that there has been a contravention of the statutory obligations, she has order-making powers (s. 49.0.1(7)). Overall, this is a solid and comprehensive scheme for addressing data breaches in the public sector (although it does not extend to those institutions covered by MFIPPA). In addition to the data breach reporting requirements, the proposed amendments will provide for whistleblower protections. They will also specifically enable the Privacy Commissioner to consult with other privacy commissioners (new s. 59(2)), and to coordinate activities, enter into agreements, and to provide for handling “of any complaint in which they are mutually interested.” (s. 59(3)). These are important amendments given that data breaches may cross provincial lines, and Canada’s privacy commissioners have developed strong collaborative relationships to facilitate cooperation and coordination on joint investigations. These provisions make clear that such co-operation is legally sanctioned, which may avoid costly and time-consuming court challenges to the commissioners’ authority to engage in this way. The amendments also broaden s. 61(1)(a) of FIPPA which currently makes it an offence to wilfully disclose personal information in contravention of the Act. If passed, it will be an offence to wilfully collect, use or disclose information in the same circumstances. Collectively the proposed FIPPA amendments are timely and important. Summary of Recommendations: On artificial intelligence in the broader public sector: 1. Amend the Preamble to Bill 194 to address the importance of human rights values in the development and deployment of AI systems for the broader public sector.
2. Add a purpose section to the AI portion of Bill 194 that reads: The purpose of this Part is to ensure that artificial intelligence systems adopted and used by public sector entities are developed, adopted, operated and maintained in manner that is transparent and accountable and that respects the privacy and human rights of Ontarians. 3. Amend s. 5(3) to read: 5(3) A public sector entity to which this section applies, shall, in accordance with the regulations, develop and implement an accountability framework respecting their use of the artificial intelligence system. At a minimum, such a framework will include: a) The specification of reporting channels for internal or external complaints or concerns about the operation of the artificial intelligence system; b) Record-keeping requirements for complaints and concerns raised under subparagraph 5(3)(a), as well as for responses thereto.
On Digital Technology Affecting Individuals Under Age 18: 1. Incorporate the contents of ss. 9 and 10 into FIPPA and MFIPPA, with the necessary modification to require reporting to the Privacy Commissioner. 2. Give the authority to issue directives regarding the use of certain technologies in the classroom or by school boards to the Minister of Education and ensure that an open and consultative public engagement process is included.
Published in
Privacy
Monday, 03 June 2024 12:26
Privacy for the Digital Age: The Supreme Court of Canada's decision in R. v. BykovetsApologies for a somewhat longer than usual post - but the Supreme Court of Canada's decision in R. v. Bykovets both interesting and important.... The Supreme Court of Canada’s decision in R v. Bykovets is significant for two reasons. The first is that it affirms an understanding of privacy that is in keeping with the realities of contemporary and emerging technologies. The second is that it does so by the narrowest of margins, laying bare the tension between two very different ways of understanding privacy in a technological age. While this is a victory for privacy rights, it should leave celebrants in a sober mood. The appellant Bykovets had been convicted of 14 offences relating to credit card fraud and unlawful credit card purchases. During their investigation, Calgary police approached Moneris, a third-party payment processing company, to obtain the IP address linked to specific fraudulent online purchases. Moneris complied with the request. Police then sought a production order to compel the relevant internet service provider (ISP) to provide the customer name and address (CNA) information associated with the IP address. With this information, they were able to obtain search warrants for the accused’s home. At trial, the appellant challenged these search warrants, arguing that when the police obtained his IP address from Moneris without a production order, they violated his right to privacy under the Canadian Charter of Rights and Freedoms. Bykovets was convicted. The trial judge found that there was no reasonable expectation of privacy in an IP address because an IP address on its own did not disclose a “biographical core” of information (at para 24). The majority of the Court of Appeal agreed with a strong dissent from Justice Veldhuis. R v. Bykovets builds on the 2014 decision of the Supreme Court of Canada in R. v. Spencer. In Spencer, the Court tackled an issue that had bedeviled lower courts for several years, resulting in inconsistent decisions. The issue was whether there was a reasonable expectation of privacy in CNA information. Until Spencer, it was unclear whether police could simply ask ISPs for CNA information linked to an IP address without the need for a production order. The argument was that a person had no reasonable expectation of privacy in their name and address, and so police did not require judicial authorization to access it. The Supreme Court of Canada ruled in Spencer that a request for this information in a context where it would be linked to online activities raised a reasonable expectation of privacy. Bykovets addresses the issue of the status of the address itself – prior to its linkage with CNA information. Justice Karakatsanis, writing for a majority of the Supreme Court of Canada in Bykovets, emphasized the importance of a robust right to privacy in a data-driven society. The first line of her decision states: “The Internet has shifted much of the human experience from physical spaces to cyberspace” (at para 1). The IP address is a vital connector between online activities and the individual who engages in them. Justice Karakatsanis rejects an approach that assesses privacy rights in this information “based on police’s stated intention to use the information they gather in only one way” (at para 6), namely to obtain a production order to further link the IP address to an ISP who can provide the CNA information. In her view, the reasonable expectation of privacy must be understood according to a normative standard, which focuses on “what privacy should be – in a free, democratic and open society – balancing the individual’s right to be left alone against the community’s insistence on protection” (at para 7). In her view, an IP address can be linked to deeply personal information about online activities that can, on its own, reveal the identity of the individual even if a further production order for CNA information is not sought. According to Justice Karakatsanis, “an IP address is the first digital breadcrumb that can lead the state on the trail of an individual’s Internet activity” (at para 9). It is “the key that can lead the state through the maze of a user’s Internet activity and is the link through which intermediaries can volunteer that user’s information to the state.” (at para 13). She goes on to note that “[i]f s. 8 is to meaningfully protect the online privacy of Canadians in today’s overwhelmingly digital world, it must protect their IP addresses” (at para 28). All parties agreed that there was a subjective expectation of privacy in IP addresses. The real issue was whether this expectation was objectively reasonable. In order to assess the reasonableness of the expectation, it is necessary first to define the subject matter of the search. The Crown characterized it as an IP address that would allow police to continue their investigation. Justice Karakatsanis found that the Crown’s description was “artificially narrow” (at para 37) and rejected an approach that focused on the declared intent of an agent of the state. In her view, additional caution is warranted when the subject matter of a search relates to digital data. She noted that the police were not really interested in an IP address; rather, they were interested in what it would reveal. Although the police planned to get a Spencer warrant before linking the IP address to CNA information, Justice Karakatsanis observed that this was not the only way in which an IP address could be used to derive information about an individual. She stated: “Online activity associated to the IP address may itself betray highly person information without the safeguards of judicial pre-authorization” (at para 43). The majority next considered other relevant factors in the assessment of a reasonable expectation of privacy, including the place where the search takes place. In the U.S., an individual cannot have a reasonable expectation of privacy in information in the hands of third parties. Justice Karakatsanis affirmed the Supreme Court of Canada’s rejection of this ‘third-party doctrine’ in section 8 jurisprudence. Control is not a determinative factor. In the context of ISP’s, the only way to keep an IP address out of the hands of third parties is to not use the internet – which in today’s society is not a meaningful choice. Although the place of a search can be relevant to the reasonableness of an expectation of privacy, it is also not determinative. Justice Karakatsanis noted that “’online spaces are qualitatively different’ from physical spaces” (at para 49, citing R. v. Ramelson at para 49). She referred to the internet as creating “a broad, accurate, and continuously expanding permanent record” (at para 50), that can be more revealing than most physical spaces. As a result, the fact that the search did not intrude on the territorial privacy rights of the accused was not significant. Another factor is the private nature of the subject matter, often referred to as the “biographical core of personal information which individuals in a free and democratic society would wish to maintain and control from dissemination to the state” (at para 51, quoting R. v. Plant at p. 293). Justice Karakatsanis adoped a normative approach with aspirational qualities. On this view, a reasonable expectation of privacy “cannot be assessed according to only one use of the evidence” (at para 53) as asserted by the police. She stated: “The unique and heightened privacy interests in personal computer data flows from its potential to expose deeply revealing information” (at para 55). This is not a suggestion that police hide behind innocuous explanations of purported use; rather, the key is “the potential of a particular subject matter to reveal an individual’s biographical core to the state” (at para 57). According to Justice Karakatsanis, . . . the ever-increasing intrusion of the Internet into our private lives must be kept in mind in deciding this case. It is widely accepted that the Internet is ubiquitous and that vast numbers of Internet users leave behind them a trail of information that others gather up to different ends, information that may be pieced together to disclose deeply private details. [. . . ] This social context of the digital world is necessary to a functional approach in defining the privacy interest afforded under the Charter to the information that could be revealed by an IP address (at para 58). Justice Karakatsanis rebuffed arguments by the Crown that the IP address is useless without the CNA obtained with a Spencer warrant. An IP address can convey intimate information about online user activity even absent CNA data. Further, the online activity can be correlated with other available data which could ultimately lead to the identification of the individual. In such a context, a Spencer warrant offers little practical protection. It is the IP address which is “the key to unlocking an Internet user’s online activity” (at para 69). Given this analysis, it is unsurprising that the majority of the Court concludes that there is a reasonable expectation of privacy in IP addresses. The majority centres the role of the private sector in the amassing of information about online activities, giving these third parties “immense informational power” (at para 75). Justice Karakatsanis observes that “By concentrating this mass of information with private third parties and granting them the tools to aggregate and dissect that data, the Internet has essentially altered the topography of privacy under the Charter. It has added a third party to the constitutional ecosystem, making the horizontal relationship between the individual and state tripartite” (at para 78). The result is that the state has an enhanced information capacity, as they have many routes for access to this information. Justice Karakatsanis observes that these companies “respond to frequent requests by law enforcement and can volunteer all activity associated with the requested IP address. Private corporate citizens can volunteer granular profiles of an individual user’s Internet activity over days, weeks, or months without ever coming under the aegis of the Charter” (at para 10). The majority acknowledges that the important privacy concerns flowing from this massive concentration of personal information need to be balanced against the legitimate interest in “[s]afety, security and the suppression of crime” (at para 11, citing R v. Tessling, at para 17). Justice Karakatsanis notes that digital technologies have enhanced the ability of criminals to perpetrate crime and to evade law enforcement. However, she observes that judicial authorization is “readily available” (at para 11). She characterizes the burden on state authorities to obtain the necessary authorizations as “not onerous” (at para 12), given the increased availability of telewarrants. Further, she states that “the burden imposed on the state by recognizing a reasonable expectation of privacy in IP addresses pales compared to the substantial privacy concerns implicated in this case” (at para 86). Justice Côté writes for the four dissenting justices. The difference in approach between majority and dissent could hardly be more stark. While the majority opinion begins with a discussion of how closely linked IP addresses are to the details of our online activities, the dissenting opinion opens with a discussion of the police investigation into fraudulent activities that led to the charges against the accused. For the dissent, retrieving the IP address from the financial intermediary was just a first step in the investigation. Justice Côté framed the issue as “whether the appellant had a reasonable expectation of privacy in the IP addresses alone – without any other information linking the addresses to him as an Internet user – in the circumstances of this case” (at para 95). This is the crux of the difference between majority and dissenting opinions – how to characterize the information accessed by the police in this case. Although the dissenting justices accept that an IP address links an individual to their online activities, but they find that there are two ways to make that connection. One is by asking an ISP to provide the CNA information linked to the IP address (as was the case here). The other is to connect an individual to the IP address by linking their various online activities. For the dissenting justices, if the first method is used, and if a warrant will later be obtained to require an ISP to provide the necessary CNA information, an initial warrant is not needed to obtain the IP address from the intermediary. Whether a warrant is needed, then, depends upon the steps the police plan to take – a matter which is not transparent to the company that must decide whether to voluntarily share the information. In reaching their conclusion, the dissenting justices differ from the majority on the issue of reasonable expectation of privacy. In particular, Justice Côté takes a different approach to characterizing the subject matter of the search, and the reasonable expectation of privacy. On the question of the subject matter of the search, she emphasized that it was important to consider “what the police were really after” (at para 123, citing R v. Marakah, at para 15). In her view, this means considering “the capacity of the precise information sought to give rise to inferences or to reveal further information” (at para 123). In her view, Spencer aligns with this approach – once an IP address is linked to CNA information, then it can reveal the individual’s online activities. In this case, the precise information sought by police was the “raw IP addresses alone” (at para 128), which in isolation reveal very little information. A subsequent production order would be sought to match these addresses to CNA information. The dissenting justices dismissed the majority’s concerns that the IP address could be used to identify an individual from their online activities. First, they note, this was not what the police did in this case. Second, if the police were to use the second method to identify an individual, they would need a warrant. However, according to Justice Côté, this “is an issue for another day in a case where the situation actually arises on the facts” (at para 135). In her view, the police followed a clear series of steps, and the IP address was only one step, with the identification of the individual as a further step for which a production order would be obtained. According to the dissent, “to effectively hold that any step taken in an investigation engages a reasonable expectation of privacy . . . would upset the careful balance that this Court has struck between the interest of Canadians in actual privacy and the interest of Canadians in not hindering law enforcement” (at para 139). On the issue of the reasonable expectation of privacy, Justice Côté dismissed the idea that the IP address was itself ‘private’ information. She emphasized that ‘on these facts’, the IP address did not reveal any core biographical information. She insisted that the case be decided only on the actual evidentiary record, not on speculation about what might have been done. The dissenting justices analogized between leaving behind fingerprints at a crime scene and leaving behind one’s IP address on websites one visits online. Justice Côté writes “[i]t cannot be seriously suggested that a police investigation that involves dusting for fingerprints and keeping them – without more – could engage a reasonable expectation of privacy. The same – again, without more – is true of obtaining an IP address” (at para 154). What this overlooks, however, is the fact that obtaining an IP address requires a request to a private sector organization that holds that information, and that has privacy obligations to its customers. Although the Personal Information Protection and Electronic Documents Act (PIPEDA)allows for the sharing of information with law enforcement without knowledge or consent, this is tricky territory for organizations. It is also different from collecting fingerprints from a crime scene to which the police have access. The very issue before the Court was what steps are necessary in order to gain access to the information held by private sector companies. For the dissenting justices, another factor in assessing a reasonable expectation of privacy – and another point of difference with the majority – is the place of the search. This is tied to territorial notions of privacy under which the strongest protection is with respect to a person’s home. According to the dissent, the place of the search is the database of the credit card processor, and this diminishes any objectively reasonable expectation of privacy on the part of the accused. With respect, in a context in which people in their homes interact in digital environments on a daily and routine basis, this is 19th century reasoning that is a poor fit for the information age. The approach of the dissenting justices also overlooks the fact that laws such as PIPEDA are permissive when it comes to data sharing by organizations with law enforcement. Under section 7(3)(c.1) of PIPEDA, an organization may disclose personal information without the knowledge or consent of the individual to a government actor upon request by that actor where the purpose is law enforcement or investigation. The only check on this data sharing without knowledge or consent is the Charter. If there is a reasonable expectation of privacy in the data being shared, then police require judicial authorization. Charter rights in this context are extremely important – particularly given the vast quantities of often highly sensitive personal information in the hands of private sector organizations. This volume and variety of information has only been increasing and will continue to do so exponentially. To say that the police can request the digital equivalent of a skeleton key from a private organization without a warrant so long as they only intend to use that key to open a particular lock, is to effectively surrender essential Charter rights to privacy in exchange for a “trust me” approach to policing that runs counter to the very idea of Charter rights. The private sector organization is required to trust the police when handing over the information, and society must trust that the police will only use this data appropriately. Yet, the right to be free from unreasonable search or seizure is premised on the very idea that some searches and seizures are unreasonable. Charter rights set important boundaries. In a digital society, the boundary between agents of the state and everything one does online is a fundamentally important one. It deserves to be guarded against intrusion. Charter cases often arise in contexts in which persons have been accused of dangerous and/or antisocial activities that we wish to see stopped. In cases such as Bykovets, it is easy to be impatient with adding superficially unnecessary steps to complicate investigations. But we need also to bear in mind the research and reporting we see on systemic racism in policing in Canada, of the misuse of police powers to stalk or harass women, and the potential for abuse of personal information when it is made too readily available to authorities. Although Charter rights may be cast as an interference in legitimate investigations, they are also a crucial safeguard against excess and abuse of authority. The digital data held by private sector companies can render us naked in the eyes of state authorities. The Charter is not a blindfold that leaves police fumbling in the dark. Rather, it is a protective cloak that each of us wears – until judicial authorization directs otherwise. For the majority in Bykovets, the goal is not to interfere with online investigations; rather, it is to “better reflect what each reasonable Canadian expects from a privacy perspective and from a crime control perspective” (at para 86). Finding a reasonable expectation of privacy in IP addresses “significantly reduces the potential of any “arbitrary and even discriminatory” exercises of discretion” (at para 87) by the state. It also removes from the private sector decision-making about what information (and how much of it) to disclose to the state. The majority characterizes its approach as ensuring “that the veil of privacy all Canadians expect when they access the Internet is only lifted when an independent judicial officer is satisfied that providing this information to the state will serve a legitimate law enforcement purpose.” (at para 90)
Published in
Privacy
Thursday, 16 May 2024 12:51
BC Court Confirms Application of Provincial Privacy Law to Federal Political PartiesA battle over the protection of personal information in the hands of federal political parties (FPPs) has been ongoing now for several years in British Columbia. The BC Supreme Court has just released a decision which marks a significant defeat for the FPPs in their quest to ensure that only minimal privacy obligations apply to their growing collection, use and disclosure of personal information. Although the outcome only green-lights the investigation by BC’s Office of the Information and Privacy Commissioner into the Liberal, New Democrat and Conservative parties’ compliance with the province’s Personal Information Protection Act (PIPA), it is still an important victory for the complainants. The decision affirms the constitutional applicability of PIPA to the FPPs. The tone of the decision also sends a message. Its opens with: “The ability of an individual to control their personal information is intimately connected to their individual autonomy, dignity and privacy.” Justice Weatherill confirms that “These fundamental values lie at the heart of democracy” (at para 1). The dispute originated with complaints brought in 2019 by three BC residents (the complainants) who sought access under PIPA to their personal information in the hands of each of the three main FPPs in their BC ridings. They wanted to know what information had been collected about them, how it was being used, and to whom it was being disclosed. This access right is guaranteed under PIPA. By contrast no federal law – whether relating to privacy or to elections – provides an equivalent right with respect to political parties. The Canada Elections Act (CEA) was amended in 2018 to include a very limited obligation for FPPs to have privacy policies approved by the Chief Electoral Officer (CEO), published, and kept up to date. These provisions did not include access rights, oversight, or a complaints mechanism. When the responses of the FPPs to the complainants’ PIPA requests proved inadequate, the complainants filed complaints with the OIPC, which initiated an investigation. Disappointingly, the FPPs resisted this investigation from the outset. They challenged the constitutional basis for the investigation, arguing that the BC law could not apply to FPPs. This issue was referred to an outside adjudicator, who heard arguments and rendered a decision in March 2022. He found that the term “organization” in PIPA included FPPs that collected information about BC residents and that PIPA’s application to the FPPs was constitutional. In April 2022, the FPPs individually filed applications for judicial review of this decision. The adjudicator ruled that he would pause his investigation until the constitutional issues were resolved. In June of 2023, while the judicial review proceedings were ongoing, the government tabled amendments to the CEA in Bill C-47. These amendments (now passed) permit FPPs to “collect, use, disclose, retain and dispose of personal information in accordance with the party’s privacy policy” (s. 385.1). Section 385.2(3) states: “The purpose of this section is to provide for a national, uniform, exclusive and complete regime applicable to registered parties and eligible parties respecting their collection, use, disclosure, retention and disposal of personal information”. The amendments were no doubt intended to reinforce the constitutional arguments being made in the BC litigation. In his discussion of these rather cynical amendments, Justice Weatherill quoted extensively from statements of the Chief Electoral Officer of Canada before the Senate Standing Committee on Legal and Constitutional Affairs in which he discussed the limitations of the privacy provisions in the CEA, including the lack of substantive rights and the limited oversight/enforcement. The CEO is quoted as stating “Not a satisfactory regime, if I’m being perfectly honest” (at para 51). Support for extension of privacy obligations to political parties has been gaining momentum, particularly considering increasingly data-driven strategies, the use of profiling and targeting by political parties, concerns over the security of such detailed information and general frustration over politicians being able to set their own rules for conduct that would be considered unacceptable by any other actors in the public and private sectors. Perhaps sensing this growing frustration, the federal government introduced Bill C-65 in March of 2024. Among other things, this bill would provide some enforcement powers to the CEO with respect to the privacy obligations in the CEA. Justice Weatherill declined to consider this Bill in his decision, noting that it might never become law and was thus irrelevant to the proceedings. Justice Weatherill ruled that BC’s PIPA applies to organizations, and that FPPS active in the province fall within the definition of “organization”. The FPPs argued that PIPA should be found inoperative to the extent that it is incompatible with federal law under the constitutional doctrine of paramountcy. They maintained that the CEA addressed the privacy obligations of political parties and that the provincial legislation interfered with that regime. Justice Weatherill disagreed, citing the principle of cooperative federalism. Under this approach, the doctrine of paramountcy receives a narrow interpretation, and where possible “harmonious interpretations of federal and provincial legislation should be favoured over interpretations that result in incompatibility” (at para 121). He found that while PIPA set a higher standard for privacy protection, the two laws were not incompatible. PIPA did not require FPPs to do something that was prohibited under the federal law – all it did was provide additional obligations and oversight. There was no operational conflict between the laws – FPPs could comply with both. Further, there was nothing in PIPA that prevented the FPPs from collecting, using or disclosing personal information for political purposes. It simply provided additional protections. Justice Weatherill also declined to find that the application of PIPA to FPPs frustrated a federal purpose. He found that there was no evidence to support the argument that Parliament intended “to establish a regime in respect of the collection and use of personal information by FPPs” (at para 146). He also found that the evidence did not show that it was a clear purpose of the CEA privacy provisions “to enhance, protect and foster the FPPs’ effective participation in the electoral process”. He found that the purpose of these provisions was simply to ensure that the parties had privacy policies in place. Nothing in PIPA frustrated that purpose; rather, Justice Weatherill found that even if there was a valid federal purpose with respect to the privacy policies, “PIPA is in complete alignment with that purpose” (at para 158). Justice Weatherill also rejected arguments that the doctrine of interjurisdictional immunity meant that the federal government’s legislative authority over federal elections could not be allowed to be impaired by BC’s PIPA. According to this argument the Chief Electoral Officer was to have the final say over the handling of personal information by FPPs. The FPPs argued that elections could be disrupted by malefactors who might use access requests under PIPA in a way that could lead to “tying up resources that would otherwise be focused on the campaign and subverting the federal election process” (at para 176). Further, if other provincial privacy laws were extended to FPPs, it might mean that FPPs would have to deal with multiple privacy commissioners, bogging them down even further. Justice Weatherill rejected these arguments, stating: Requiring FPPs to disclose to British Columbia citizens, on request, the personal information they have about the citizen, together with information as to how it has been used and to whom it has been disclosed has no impact on the core federal elections power. It does not “significantly trammel” the ability of Canadian citizens to seek by lawful means to influence fellow electors, as was found to have been the case in McKay. It does not destroy the right of British Columbians to engage in federal election activity. At most, it may have a minimal impact on the administration of FPPs. This impact is not enough to trigger interjurisdictional immunity. All legislation carries with it some burden of compliance. The petitioners have not shown that this burden is so onerous as to impair them from engaging with voters. (at para 182). Ultimately, Justice Weatherill ruled that there was no constitutional barrier to the application of PIPA. The result is that the matter goes back to the OIPC for investigation and determination on the merits. It has been a long, drawn out and expensive process so far, but at least this decision is an unequivocal affirmation of the application of basic privacy principles (at least in BC) to the personal information handling practices of FPPs. It is time for Canada’s political parties to accept obligations similar to those imposed on private sector organizations. If they want to collect, use and disclose data in increasingly complex data-driven voter profiling and targeting activities they need to stop resisting the commensurate obligations to treat that information with care and to be accountable for their practices.
Published in
Privacy
Tagged under
Tuesday, 19 March 2024 09:41
AI, Human Rights, and Canada's Proposed AI and Data ActArtificial intelligence technologies have significant potential to impact human rights. Because of this, emerging AI laws make explicit reference to human rights. Already-deployed AI systems are raising human rights concerns – including bias and discrimination in hiring, healthcare, and other contexts; disruptions of democracy; enhanced surveillance; and hateful deepfake attacks. Well-documented human rights impacts also flow from the use of AI technologies by law enforcement and the state, and from the use of AI in armed conflicts. Governments are aware that human rights issues with AI technologies must be addressed. Internationally, this is evident in declarations by the G7, UNESCO, and the OECD. It is also clear in emerging national and supranational regulatory approaches. For example, human rights are tackled in the EU AI Act, which not only establishes certain human-rights-based no-go zones for AI technologies, but also addresses discriminatory bias. The US’s NIST AI Risk Management Framework (a standard, not a law – but influential nonetheless) also addresses the identification and mitigation of discriminatory bias. Canada’s Artificial Intelligence and Data Act (AIDA), proposed by the Minister of Industry, Science and Economic Development (ISED) is currently at the committee stage as part of Bill C-27. The Bill’s preamble states that “Parliament recognizes that artificial intelligence systems and other emerging technologies should uphold Canadian norms and values in line with the principles of international human rights law”. In its substantive provisions, AIDA addresses “biased output”, which it defines in terms of the prohibited grounds of discrimination in the Canadian Human Rights Act. AIDA imposes obligations on certain actors to assess and mitigate the risks of biased output in AI systems. The inclusion of these human rights elements in AIDA is positive, but they are also worth a closer look. Risk Regulation and Human Rights Requiring developers to take human rights into account in the design and development of AI systems is important, and certainly many private sector organizations already take seriously the problems of bias and the need to identify and mitigate it. After all, biased AI systems will be unable to perform properly, and may expose their developers to reputational harm and possibly legal action. However, such attention has not been universal, and has been addressed with different degrees of commitment. Legislated requirements are thus necessary, and AIDA will provide these. AIDA creates obligations to identify and mitigate potential harms at the design and development stage, and there are additional documentation and some transparency requirements. The enforcement of AIDA obligations can come through audits conducted or ordered by the new AI and Data Commissioner, and there is also the potential to use administrative monetary penalties to punish non-compliance, although what this scheme will look like will depend very much on as-yet-to-be-developed regulations. AIDA, however, has some important limitations when it comes to human rights. Selective Approach to Human Rights Although AIDA creates obligations around biased output, it does not address human rights beyond the right to be free from discrimination. Unlike the EU AI Act, for example, there are no prohibited practices related to the use of AI in certain forms of surveillance. A revised Article 5 of the EU AI Act will prohibit real-time biometric surveillance by law enforcement agencies in publicly accessible spaces, subject to carefully-limited exceptions. The untargeted scraping of facial images for the building or expansion of facial recognition databases (as occurred with Clearview AI) is also prohibited. Emotion recognition technologies are banned in some contexts, as are some forms of predictive policing. Some applications that are not outright prohibited, are categorized as high risk and have limits imposed on the scope of their use. These “no-go zones” reflect concerns over a much broader range of human rights and civil liberties than what we see reflected in Canada’s AIDA. It is small comfort to say that the Canadian Charter of Rights and Freedoms remains as a backstop against government excess in the use of AI tools for surveillance or policing; ex ante AI regulation is meant to head off problems before they become manifest. No-go zones reflect limits on what society is prepared to tolerate; AIDA sets no such limits. Constitutional litigation is expensive, time-consuming and uncertain in outcome (just look at the 5-4 splint in the recent R. v. Bykovets decision of the Supreme Court of Canada). Further, the application of AIDA to the military and intelligence services is expressly excluded from AIDA’s scope (as is the application of the law to the federal public service). Privacy is an important human right, and privacy rights are not part of the scope of AIDA. The initial response is that such rights are dealt with under privacy legislation for public and private sectors and at federal, provincial and territorial levels. However, such privacy statutes deal principally with data protection (in other words, they govern the collection, use and disclosure of personal information). AIDA could have addressed surveillance more directly. After all, the EU has top of its class data protection laws, but still places limits on the use of AI systems for certain types of surveillance activities. Second, privacy laws in Canada (and there are many of them) are, apart from Quebec’s, largely in a state of neglect and disrepair. Privacy commissioners at federal, provincial, and territorial levels have been issuing guidance as to how they see their laws applying in the AI context, and findings and rulings in privacy complaints involving AI systems are starting to emerge. The commissioners are thoughtfully adapting existing laws to new circumstances, but there is no question that there is need for legislative reform. In issuing its recent guidance on Facial Recognition and Mugshot Databases, the Office of the Information and Privacy Commissioner of Ontario specifically identified the need to issue the guidance in the face of legislative gaps and inaction that “if left unaddressed, risk serious harms to individuals’ right to privacy and other fundamental human rights.” Along with AIDA, Bill C-27 contains the Consumer Privacy Protection Act (CPPA) which will reform Canada’s private sector data protection law, the Personal Information Protection and Electronic Documents Act (PIPEDA). However, the CPPA has only one AI-specific amendment – a somewhat tepid right to an explanation of automated decision-making. It does not address the data scraping issue at the heart of the Clearview AI investigation, for example (where the core findings of the Commissioner remain disputed by the investigated company) and which prompted the articulation of a no-go zone for data-scraping for certain purposes in the EU AI Act. High Impact AI and Human Rights AIDA will apply only to “high impact” AI systems. Among other things, such systems can adversely impact human rights. While the original version of AIDA in Bill C-27 left the definition of “high impact” entirely to regulations (generating considerable and deserved criticism), the Minister of ISED has since proposed amendments to C-27 that set out a list of categories of “high impact” AI systems. While this list at least provides some insight into what the government is thinking, it creates new problems as well. This list identifies several areas in which AI systems could have significant impacts on individuals, including in healthcare and in some court or tribunal proceedings. Also included on the list is the use of AI in all stages of the employment context, and the use of AI in making decisions about who is eligible for services and at what price. Left off the list, however, is where AI systems are (already) used to determine who is selected as a tenant for rental accommodation. Such tools have extremely high impact. Yet, since residential tenancies are interests in land, and not services, they are simply not captured by the current “high impact” categories. This is surely an oversight – yet it is one that highlights the rather slap-dash construction of the AIDA and its proposed amendments. As a further example, a high-impact category addressing the use of biometrics to assess an individual’s behaviour or state of mind could be interpreted to capture affect recognition systems or the analysis of social media communications, but this is less clear than it should be. It also raises the question as to whether the best approach, from a human rights perspective, is to regulate such systems as high impact or whether limits need to be placed on their use and deployment. Of course, a key problem is that this bill is housed within ISED. This is not a bill centrally developed that takes a broader approach to the federal government and its powers. Under AIDA, medical devices are excluded from the category of “high impact” uses of AI in the healthcare context because it is Health Canada that will regulate AI-enabled medical devices, and ISED must avoid treading on its toes. Perhaps ISED also seeks to avoid encroaching on the mandates of the Minister of Justice, or the Minister of Public Safety. This may help explain some of the crabbed and clunky framing of AIDA compared to the EU AI Act. It does, however, raise the question of why Canada chose this route – adopting a purportedly comprehensive risk-management framework housed under the constrained authority of the Minister of ISED. Such an approach is inherently flawed. As discussed above, AIDA is limited in the human rights it is prepared to address, and it raises concerns about how human rights will be both interpreted and framed. On the interpretation side of things, the incorporation of the Canadian Human Rights Act’s definition of discrimination in AIDA combined with ISED’s power to interpret and apply the proposed law will give ISED interpretive authority over the definition of discrimination without the accompanying expertise of the Canadian Human Rights Commission. Further, it is not clear that ISED is a place for expansive interpretations of human rights; human rights are not a core part of its mandate – although fostering innovation is. All of this should leave Canadians with some legitimate concerns. AIDA may well be passed into law – and it may prove to be useful in the better governance of AI. But when it comes to human rights, it has very real limitations. AIDA cannot be allowed to end the conversation around human rights and AI at the federal level – nor at the provincial level either. Much work remains to be done.
Published in
Privacy
Monday, 11 March 2024 15:45
Investigation of AI-Enabled Remote Proctoring Software under Public Sector Privacy Law Leads to AI RecommendationsOntario’s Information and Privacy Commissioner has released a report on an investigation into the use by McMaster University of artificial intelligence (AI)-enabled remote proctoring software. In it, Commissioner Kosseim makes findings and recommendations under the province’s Freedom of Information and Protection of Privacy Act (FIPPA) which applies to Ontario universities. Interestingly, noting the absence of provincial legislation or guidance regarding the use of AI, the Commissioner provides additional recommendations on the adoption of AI technologies by public sector bodies. AI-enabled remote proctoring software saw a dramatic uptake in use during the pandemic as university classes migrated online. It was also widely used by professional societies and accreditation bodies. Such software monitors those writing online exams in real-time, recording both audio and video, and using AI to detect anomalies that may indicate that cheating is taking place. Certain noises or movements generate ‘flags’ that lead to further analysis by AI and ultimately by the instructor. If the flags are not resolved, academic integrity proceedings may ensue. Although many universities, including the respondent McMaster, have since returned to in-person exam proctoring, AI-enabled remote exam surveillance remains an option where in-person invigilation is not possible. This can include in courses delivered online to students in diverse and remote locations. The Commissioner’s investigation related to the use by McMaster University of two services offered by the US-based company Respondus: Respondus Lockdown Browser and Respondus Monitor. Lockdown Browser consists of software downloaded by students onto their computers that blocks access to the internet and to other files on the computer during an exam. Respondus Monitor is the AI-enabled remote proctoring application. This post focuses on Respondus Monitor. AI-enabled remote proctoring systems have raised concerns about both privacy and broader human rights issues. These include the intrusiveness of the constant audio and video monitoring, the capturing of data from private spaces, uncertainty over the treatment of personal data collected by such systems, adverse impacts on already marginalised students, and the enhanced stress and anxiety that comes from both constant surveillance and easily triggered flags. The broader human rights issues, however, are an uncomfortable fit with public sector data protection law. Commissioner Kosseim begins with the privacy issues, finding that Respondus Monitor collects personal information that includes students’ names and course information, images of photo identification documents, and sensitive biometric data in audio and video recordings. Because the McMaster University Act empowers the university to conduct examinations and appoint examiners, the Commissioner found that the collection was carried out as part of a lawfully authorized activity. Although exam proctoring had chiefly been conducted in-person prior to the pandemic, she found that there was no “principle of statute or common law that would confine the method by which the proctoring of examinations may be conducted by McMaster to an in-person setting” (at para 48). Further, she noted that even post-pandemic, there might still be reasons to continue to use remote proctoring in some circumstances. She found that the university had a legitimate interest in attempting to curb cheating, noting that evidence suggested an upward trend in academic integrity cases, and a particular spike during the pandemic. She observed that “by incorporating online proctoring into its evaluation methods, McMaster was also attempting to address other new challenges that arise in an increasingly digital and remote learning context” (at para 50). The collection of personal information must be necessary to a lawful authorized activity carried out by a public body. Commissioner Kosseim found that the information captured by Respondus Monitor – including the audio and video recordings – was “technically necessary for the purpose of conducting and proctoring the exams” (at para 60). Nevertheless, she expressed concerns over the increased privacy risks that accompany this continual surveillance of examinees. She was also troubled by McMaster’s assertion that it “retains complete autonomy, authority, and discretion to employ proctored online exams, prioritizing administrative efficiency and commercial viability, irrespective of necessity” (at para 63). She found that the necessity requirement in s. 38(2) of FIPPA applied, and that efficiency or commercial advantage could not displace it. She noted that the kind of personal information collected by Respondus Monitor was particularly sensitive, creating “risks of unfair allegations or decisions being made about [students] based on inaccurate information” (at para 66). In her view, “[t]hese risks must be appropriately mitigated by effective guardrails that the university should have in place to govern its adoption and use of such technologies” (at para 66). FIPPA obliges public bodies to provide adequate notice of the collection of personal information. Commissioner Kosseim reviewed the information made available to students by McMaster University. Although she found overall that it provided students with useful information, students had to locate different pieces of information on different university websites. The need to check multiple sites to get a clear picture of the operation of Respondus Monitor did not satisfy the notice requirement, and the Commissioner recommended that the university prepare a “clear and comprehensive statement either in a single source document, or with clear cross-references to other related documents” (at para 70). Section 41(1) of FIPPA limits the use of personal information collected by a public body to the purpose for which it was obtained or compiled, or for a consistent purpose. Although the Commissioner found that the analysis of the audio and video recordings to generate flags was consistent with the collection of that information, the use by Respondus of samples of the recordings to improve its own systems – or to allow third party research – was not. On this point, there was an important difference in interpretation. Respondus appeared to define personal information as personal identifiers such as names and ID numbers; it treated audio and video clips that lacked such identifiers as “anonymized”. However, under FIPPA audio and video recordings of individuals are personal information. No provision was made for students either to consent to or opt out of this secondary use of their personal information. Commissioner Kosseim noted that Respondus had made public statements that when operating in some jurisdictions (including California and EU members states) it did not use audio or video recordings for research or to improve its products or services. She recommended that McMaster obtain a similar undertaking from Respondus to not use its students’ information for these purposes. The Commissioner also noted that Respondus’ treating the audio and video recordings as anonymized data meant that it did not have adequate safeguards in place for this personal information. Respondus’ Terms of Service provide that the company reserved the right to disclose personal information for law enforcement purposes. Commissioner Kosseim found that McMaster should require, in its contact with Respondus, that Respondus notify it promptly of any compelled disclosure of its students’ personal information to law enforcement or to government, and to limit any such disclosure to the specific information it is legally required to disclose. She also set a retention limit for the audio and video recordings at one year, with confirmation to be provided by Respondus of deletions after the end of this period. One of the most interesting aspects of this report is the section titled “Other Recommendations” in which the Commissioner addresses the adoption of an AI-enabled technology by a public institution in a context in which “there is no current law or binding policy specifically governing the use of artificial intelligence in Ontario’s public sector.” (at para 134). The development and adoption of these technologies is outpacing the evolution of law and policy, leaving important governance gaps. In May 2023, the Commissioner Kosseim and Commissioner DeGuire of the Ontario Human Rights Commission issued a joint statement urging the Ontario government to take action to put in place an accountability framework for public sector AI. Even as governments acknowledge that these technologies create risks of discriminatory bias and other potential harms, there remains little to govern AI systems outside the piecemeal coverage offered by existing laws such as, in this case, FIPPA. Although the Commissioner’s interpretation and application of FIPPA addressed issues relating to the collection, use and disclosure of personal information, there remain important issues that cannot be addressed through privacy legislation. Commissioner Kosseim acknowledged that McMaster University had “already carried out a level of due diligence prior to adopting Respondus Monitor” (at para 138). Nevertheless, given the risks and potential harms of AI-enabled technologies, she made a number of further recommendations. The first was to conduct an Algorithmic Impact Assessment (AIA) in addition to a Privacy Impact Assessment. She suggested that the federal government’s AIA tool could be a useful guide while waiting for one to be developed for Ontario. An AIA could allow the adopter of an AI system to have better insight into the data used to train the algorithms, and could assess impacts on students going beyond privacy (which might include discrimination, increased stress, and harms from false positive flags). She also called for meaningful consultation and engagement with those affected by the adoption of the technology taking place both before the adoption of the system and on an ongoing basis thereafter. Although the university may have had to react very quickly given that the first COVID shutdown occurred shortly before an exam period, an iterative engagement process even now would be useful “for understanding the full scope of potential issue that may arise, and how these may impact, be perceived, and be experienced by others” (at para 142). She noted that this type of engagement would allow adopters to be alert and responsive to problems both prior to adoption and as they arise during deployment. She also recommended that the consultations include experts in both privacy and human rights, as well as those with technological expertise. Commissioner Kosseim also recommended that the university consider providing students with ways to opt out of the use of these technologies other than through requesting accommodations related to disabilities. She noted “AI-powered technologies may potentially trigger other protected grounds under human rights that require similar accommodations, such as color, race or ethnic origin” (at para 147). On this point, it is worth noting that the use of remote proctoring software creates a context in which some students may need to be accommodated for disabilities or other circumstances that have nothing to do with their ability to write their exam, but rather that impact the way in which the proctoring systems read their faces, interpret their movements, or process the sounds in their homes. Commissioner Kosseim encouraged McMaster University “to make special arrangements not only for students requesting formal accommodation under a protected ground in human rights legislation, but also for any other students having serious apprehensions about the AI-enabled software and the significant impacts it can have on them and their personal information” (at para 148). Commissioner Kosseim also recommended that there be an appropriate level of human oversight to address the flagging of incidents during proctoring. Although flags were to be reviewed by instructors before deciding whether to proceed to an academic integrity investigation, the Commissioner found it unclear whether there was a mechanism for students to challenge or explain flags prior to escalation to the investigation stage. She recommended that there be such a procedure, and, if there already was one, that it be explained clearly to students. She further recommended that a public institution’s inquiry into the suitability for adoption of an AI-enabled technology should take into account more than just privacy considerations. For example, the public body’s inquiries should consider the nature and quality of training data. Further, the public body should remain accountable for its use of AI technologies “throughout their lifecycle and across the variety of circumstances in which they are used” (at para 165). Not only should the public body monitor the performance of the tool and alert the supplier of any issues, the supplier should be under a contractual obligation to inform the public body of any issues that arise with the system. The outcome of this investigation offers important lessons and guidance for universities – and for other public bodies – regarding the adoption of third-party AI-enabled services. For the many Ontario universities that adopted remote proctoring during the pandemic, there are recommendations that should push those still using these technologies to revisit their contracts with vendors – and to consider putting in place processes to measure and assess the impact of these technologies. Although some of these recommendations fall outside the scope of FIPPA, the advice is still sage and likely anticipates what one can only hope is imminent guidance for Ontario’s public sector.
Published in
Privacy
Tagged under
Monday, 08 January 2024 14:22
Ontario proposes to introduce provision regarding use of AI in hiringOntario is currently holding public hearings on a new bill which, among other things, introduces a provision regarding the use of AI in hiring in Ontario. Submissions can be made until February 13, 2024. Below is a copy of my submission addressing this provision.
The following is my written submission on section 8.4 of Bill 149, titled the Working for Workers Four Act, introduced in the last quarter of 2023. I am a law professor at the University of Ottawa. I am making this submission in my individual capacity. Artificial intelligence (AI) tools are increasingly common in the employment context. Such tools are used in recruitment and hiring, as well as in performance monitoring and assessment. Section 8.4 would amend the Employment Standards Act to include a requirement for employers to provide notice of the use of artificial intelligence in the screening, assessment, or selection of applicants for a publicly advertised job position. It does not address the use of AI in other employment contexts. This brief identifies several weaknesses in the proposal and makes recommendations to strengthen it. In essence, notice of the use of AI in the hiring process will not offer much to job applicants without a right to an explanation and ideally a right to bring any concerns to the attention of a designated person. Employees should also have similar rights when AI is used in performance assessment and evaluation. 1. Definitions and exclusions If passed, Bill 149 would (among other things) enact the first provision in Ontario to directly address AI. The proposed section 8.4 states: 8.4 (1) Every employer who advertises a publicly advertised job posting and who uses artificial intelligence to screen, assess or select applicants for the position shall include in the posting a statement disclosing the use of the artificial intelligence. (2) Subsection (1) does not apply to a publicly advertised job posting that meets such criteria as may be prescribed. The term “artificial intelligence” is not defined in the bill. Rather, s. 8.1 of Bill 149 leaves the definition to be articulated in regulations. This likely reflects concerns that the definition of AI will continue to evolve along with the rapidly changing technology and that it is best to leave its definition to more adaptable regulations. The definition is not the only thing left to regulations. Section 8.4(2) requires regulations to specify the criteria that will allow publicly advertised job postings to be exempted from the disclosure requirement in s. 8.4(1). The true scope and impact of s. 8.4(1) will therefore not be clear until these criteria are prescribed in regulations. Further, s. 8.4 will not take effect until the regulations are in place. 2. The Notice Requirement The details of the nature and content of the notice that an employer must provide are not set out in s. 8.4, nor are they left to regulations. Since there are no statutory or regulatory requirements, presumably notice can be as simple as “we use artificial intelligence in our screening and selection process”. It would be preferable if notice had to at least specify the stage of the process and the nature of the technique used. Section 8.4 is reminiscent of the 2022 amendments to the Employment Standards Act which required employers with more than 25 employees to provide their employees with notification of any electronic monitoring taking place in the workplace. As with s. 8.4(1), above, the main contribution of this provision was (at least in theory) enhanced transparency. However, the law did not provide for any oversight or complaints mechanism. Section 8.4(1) is similarly weak. If an employer fails to provide notice of the use of AI in the hiring process, then either the employer is not using AI in recruitment and hiring, or they are failing to disclose it. Who will know and how? A company that is found non-compliant with the notice requirement, once it is part of the Employment Standards Act, could face a fine under s. 132. However, proceedings by way of an offence are a rather blunt regulatory tool. 3. A Right to an Explanation? Section 8.4(1) does not provide job applicants with any specific recourse if they apply for a job for which AI is used in the selection process and they have concerns about the fairness or appropriateness of the tool used. One such recourse could be a right to demand an explanation. The Consumer Privacy Protection Act (CPPA), which is part of the federal government’s Bill C-27, currently before Parliament, provides a right to an explanation to those about whom an automated decision, prediction or recommendation is made. Sections 63(3) and (4) provide: (3) If the organization has used an automated decision system to make a prediction, recommendation or decision about the individual that could have a significant impact on them, the organization must, on request by the individual, provide them with an explanation of the prediction, recommendation or decision. (4) The explanation must indicate the type of personal information that was used to make the prediction, recommendation or decision, the source of the information and the reasons or principal factors that led to the prediction, recommendation or decision. Subsections 63(3) and (4) are fairly basic. For example, they do not include a right of review of the decision by a human. But something like this would still be a starting point for a person seeking information about the process by which their employment application was screened or evaluated. The right to an explanation in the CPPA will extend to decisions, recommendations and predictions made with respect to employees of federal works, undertakings, and businesses. However, it will not apply to the use of AI systems in provincially regulated employment sectors. Without a private sector data protection law of its own – or without a right to an explanation to accompany the proposed s. 8.4 – provincially regulated employees in Ontario will be out of luck. In contrast, Quebec’s recent amendments to its private sector data protection law provide for a more extensive right to an explanation in the case of automated decision-making – and one that applies to the employment and hiring context. Section 12.1 provides: 12.1. Any person carrying on an enterprise who uses personal information to render a decision based exclusively on an automated processing of such information must inform the person concerned accordingly not later than at the time it informs the person of the decision. He must also inform the person concerned, at the latter’s request, (1) of the personal information used to render the decision; (2) of the reasons and the principal factors and parameters that led to the decision; and (3) of the right of the person concerned to have the personal information used to render the decision corrected. The person concerned must be given the opportunity to submit observations to a member of the personnel of the enterprise who is in a position to review the decision. Section 12.1 thus combines a notice requirement with, at the request of the individual, a right to an explanation. In addition, the affected individual can “submit observations” to an appropriate person within the organization who “is in a position to review the decision”. This right to an explanation is triggered only by decisions that are based exclusively on automated processing of personal information – and the scope of the right to an explanation is relatively narrow. However, it still goes well beyond Ontario’s Bill 149, which creates a transparency requirement with nothing further. 4. Scope Bill 149 applies to the use of “artificial intelligence to screen, assess or select applicants”. Bill C-27 and Quebec’s law, both referenced above, are focused on “automated decision-making”. Although automated decision-making is generally considered a form of AI (it is defined in C-27 as “any technology that assists or replaces the judgment of human decision-makers through the use of a rules-based system, regression analysis, predictive analytics, machine learning, deep learning, a neural network or other technique”) it is possible that in an era of generative AI technologies, the wording chosen for Bill 149 is more inclusive. In other words, there may be uses of AI that are not decision-making, predicting or recommending, but that can still used in screening, assessing or hiring processes. However, it should be noted that Ontario’s Bill 149 is also less inclusive than Bill C-27 or Quebec’s law because it focuses only on screening, assessment or selecting applicants for a position. It does not apply to the use of AI tools to monitor, evaluate or assess the performance of existing employees or to make decisions regarding promotion, compensation, retention, or other employment issues – something which would be covered by Quebec’s law (and by Bill C-27 for employees in federally regulated employment). Although arguably the requirements regarding electronic workplace monitoring added to the Employment Standards Act in 2022 might provide transparency about the existence of electronic forms of surveillance (which could include those used to feed data to AI systems), these transparency obligations apply only in workplaces with more than 25 employees, and there are no employee rights linked to the use of these data in automated or AI-enabled decision-making systems. 5. Discriminatory Bias A very significant concern with the use of AI systems for decision-making about humans is the potential for discriminatory bias in the output of these systems. This is largely because systems are trained on existing and historical data. Where such data are affected by past discriminatory practices (for example, a tendency to hire men rather than women, or white, able-bodied, heterosexual people over those from equity-deserving communities) then there is a risk that automated processes will replicate and exacerbate these biases. Transparency about the use of an AI tool alone in such a context is not much help – particularly if there is no accompanying right to an explanation. Of course, human rights legislation applies to the employment context, and it will still be open to an employee who believes they have been discriminated against to bring a complaint to the Ontario Human Rights Commission. However, without a right to an explanation, and in the face of proprietary and closed systems, proving discrimination may be challenging and may require considerable resources and expertise. It may also require changes to human rights legislation to specifically address algorithmic discrimination. Without these changes in place, and without adequate resourcing to support the OHRC’s work to address algorithmic bias, recourse under human rights legislation may be extremely challenging.
6. Conclusion and Recommendations This exploration of Bill 149’s transparency requirements regarding the use of AI in the hiring process in Ontario reveals the limited scope of the proposal. Its need for regulations in order take effect has the potential to considerably delay its implementation. It provides for notice but not for a right to an explanation or for human review of AI decisions. There is also a need to make better use of existing regulators (particularly privacy and human rights commissions). The issue of the use of AI in recruitment (or in the workplace more generally in Ontario) may require more than just tweaks to the Employment Standards Act but may also demand amendments to Ontario’s Human Rights Code and perhaps even specific privacy legislation at the very least aimed at the employment sector in Ontario. Recommendations: 1. Redraft the provision so that the core obligations take effect without need for regulations or ensure that the necessary regulations to give effect to this provision are put in place promptly. 2. Amend s. 8.4 (1) to either include the elements that are required in any notice of the use of an AI system or provide for the inclusion of such criteria in regulations (so long as doing so does not further delay the coming into effect of the provision). 3. Provide for a right to an explanation to accompany s. 8.4(1). An alternative to this would be a broader right to an explanation in provincial private sector legislation or in privacy legislation for employees in provincially regulated sectors in Ontario, but this would be much slower than the inclusion of a basic right to an explanation in s. 8.4. The right to an explanation could also include a right to submit observations to a person in a position to review any decision or outcome. 4. Extend the notice requirement to other uses of AI to assess, evaluate and monitor the performance of employees in provincially regulated workplaces in Ontario. Ideally, a right to an explanation should also be provided in this context. 5. Ensure that individuals who are concerned that they have been discriminated against by the use of AI systems in recruitment (as well as employees who have similar concerns regarding the use of AI in performance evaluation and assessment) have adequate and appropriate recourse under Ontario’s Human Rights Code, and that the Ontario Human Rights Commission is adequately resourced to address these concerns.
Published in
Privacy
Monday, 11 December 2023 06:58
Data Governance for AI under Canada's Proposed AI and Data Act (AIDA Amendments Part IV)The federal government’s proposed Artificial Intelligence and Data Act (AIDA) (Part III of Bill C-27) - contained some data governance requirements for anonymized data used in AI in its original version. These were meant to dovetail with changes to PIPEDA reflected in the Consumer Privacy Protection Act (CPPA) (Part I of Bill C-27). The CPPA provides in s. 6(5) that “this Act does not apply in respect of personal information that has been anonymized.” Although no such provision is found in PIPEDA, this is, to all practical effects, the state of the law under PIPEDA. PIPEDA applies to “personal information”, which is defined as “information about an identifiable individual”. If someone is not identifiable, then it is not personal information, and the law does not apply. This was the conclusion reached, for example, in the 2020 Cadillac Fairview joint finding of the federal Privacy Commissioner and his counterparts from BC and Alberta. PIPEDA does apply to pseudonymized information because such information ultimately permits reidentification. The standard for identifiability under PIPEDA had been set by the courts as a “’serious possibility’ that an individual could be identified through the use of that information, alone or in combination with other available information.” (Cadillac Fairview at para 143). It is not an absolute standard (although the proposed definition for anonymized data in C-27 currently seems closer to absolute). In any event, the original version of AIDA was meant to offer comfort to those concerned with the flat-out exclusion of anonymized data from the scope of the CPPA. Section 6 of AIDA provided that: 6. A person who carries out any regulated activity and who processes or makes available anonymized data in the course of that activity must, in accordance with the regulations, establish measures with respect to (a) the manner in which data is anonymized; and (b) the use or management of anonymized data. Problematically, however, AIDA only provided for data governance with respect to this particular subset of data. It contained no governance requirements for personal, pseudonymized, or non-personal data. Artificial intelligence systems will be only as good as the data on which they are trained. Data governance is a fundamental element of proper AI regulation – and it must address more than anonymized personal data. This is an area where the amendments to AIDA proposed by the Minister of Industry demonstrate clear improvements over the original version. To begin with, the old s. 6 is removed from AIDA. Instead of specific governance obligations for anonymized data, we see some new obligations introduced regarding data more generally. For example, as part of the set of obligations relating to general-purpose AI systems, there is a requirement to ensure that “measures respecting the data used in developing the system have been established in accordance with the regulations” (s. 7(1)a)). There is also an obligation to maintain records “relating to the data and processes used in developing the general-purpose system and in assessing the system’s capabilities and limitations” (s. 7(2)(b)). There are similar obligations the case of machine learning models that are intended to be incorporated into high-impact systems (s. 9(1)(a) and 9(2)(a)). Of course, whether this is an actual improvement will depend on the content of the regulations. But at least there is a clear signal that data governance obligations are expanded under the proposed amendments to AIDA. Broader data governance requirements in AIDA are a good thing. They will apply to data generally including personal and anonymized data. Personal data used in AI will also continue to be governed under privacy legislation and privacy commissioners will still have a say about whether data have been properly anonymized. In the case of PIPEDA (or the CPPA if and when it is eventually enacted), the set of principles for the development and use of generative AI issued by federal, provincial, and territorial privacy commissioners on December 8, 2023 make it clear that the commissioners understand their enabling legislation to provide them with the authority to govern a considerable number of issues relating to the use of personal data in AI, whether in the public or private sector. This set of principles send a strong signal to federal and provincial governments alike that privacy laws and privacy regulators have a clear role to play in relation to emerging and evolving AI technologies and that the commissioners are fully engaged. It is also an encouraging example of federal, provincial and territorial co-operation among regulators to provide a coherent common position on key issues in relation to AI governance.
Published in
Privacy
Friday, 08 December 2023 09:00
Oversight and Enforcement in the AIDA Amendments (Part III of a series)This is Part III of a series of posts that look at the proposed amendments to Canada’s Artificial Intelligence and Data Act (which itself is still a Bill, currently before the INDU Committee for study). Part I provided a bit of context and a consideration of some of the new definitions in the Bill. Part II looked at the categories of ‘high-impact’ AI that the Bill now proposes to govern. This post looks at the changed role of the AI and Data Commissioner.
The original version of the Artificial Intelligence and Data Act (Part II of Bill C-27) received considerable criticism for its oversight mechanisms. Legal obligations for the ethical and transparent governance of AI, after all, depend upon appropriate oversight and enforcement for their effectiveness. Although AIDA proposed the creation of an AI and Data Commissioner (Commissioner), this was never meant to be an independent regulator. Ultimately, AIDA placed most of the oversight obligations in the hands of the Minister of Industry – the same Minister responsible for supporting the growth of Canada’s AI sector. Critics considered this to be a conflict of interest. A series of proposed amendments to AIDA are meant to address these concerns by reworking the role of the Commissioner. Section 33(1) of AIDA makes it clear that the AI and Data Commissioner will be a “senior official of the department over which the Minister presides”, and their appointment involves being designated by the Minister. This has not changed, although the amendments would delete from this provision language stating that the Commissioner’s role is “to assist the Minister in the administration and enforcement” of AIDA. The proposed amendments elevate the Commissioner somewhat, giving them a series of powers and duties, to which the Minister can add through delegation (s. 33(3)). So, for example, it will be the newly empowered Commissioner (Commissioner 2.0) who receives reports from those managing a general-purpose or high impact system where there are reasonable grounds to suspect that the use of the system has caused serious harm (s. 8.2(1)(e), s. 11(1)(g)). Commissioner 2.0 can also order someone managing or making available a general-purpose system to provide them with the accountability framework they are required to create under s. 12 (s. 13(1)) and can provide guidance or recommend corrections to that framework (s. 13(2)). Commissioner 2.0 can compel those making available or managing an AI system to provide the Commissioner with an assessment of whether the system is high impact, and in relation to which subclass of high impact systems set out in the schedule. Commissioner 2.0 can agree or disagree with the assessment, although if they disagree, their authority seems limited to informing the entity in writing with their reasons for disagreement. More significant are Commissioner 2.0’s audit powers. Under the original version of AIDA, these were to be exercised by the Minister – the powers are now those of the Commissioner (s. 15(1)). Further, Commissioner 2.0 may order (previously this was framed as “require”) that the person either conduct an audit themselves or that the person engage the services of an independent auditor. The proposed amendments also empower the Commissioner to conduct an audit to determine if there is a possible contravention of AIDA. This strengthens the audit powers by ensuring that there is at least an option that is not at least somewhat under the control of the party being audited. The proposed amendments give Commissioner 2.0 additional powers necessary to conduct an audit and to carry out testing of an AI system (s. 15(2.1)). Where Commissioner 2.0 conducts an audit, they must provide the audited party with a copy of the report (s. 15(3.1)) and where the audit is conducted by the person responsible or someone retained by them, they must provide a copy to the Commissioner (s. 15(4)). The Minister still retains some role with respect to audits. He or she may request that the Commissioner conduct an audit. In an attempt to preserve some independence of Commissioner 2.0, the Commissioner, when receiving such a request, may either carry out the audit or decline to do so on the basis that there are no reasonable grounds for an audit, so long as they provide the Minister with their reasons (s. 15.1(1)(b)). The Minister may also order a person to take actions to bring themselves into compliance with the law (s. 16) or to cease making available or terminate the operation of a system if the Minister considers compliance to be impossible (s. 16(b)) or has reasonable grounds to believe that the use of the system “gives rise to a risk of imminent and serious harm” (s. 17(1)). As noted above, Commissioner 2.0 (a mere employee in the Minister’s department) will have order making powers under the amendments. This is something the Privacy Commissioner of Canada, an independent agent of Parliament, appointed by the Governor in Council, is hoping to get in Bill C-27. If so, it will be for the first time since the enactment of PIPEDA in 2000. Orders of Commissioner 2.0 or the Minister can become enforceable as orders of the Federal Court under s. 20. Commissioner 2.0 is also empowered to share information with a list of federal or provincial government regulators where they have “reasonable grounds to believe that the information may be relevant to the administration or enforcement by the recipient of another Act of Parliament or of a provincial legislature.” (s. 26(1)). Reciprocally, under a new provision, federal regulators may also share information with the Commissioner (s. 26.1). Additionally, Commissioner 2.0 may “enter into arrangements” with different federal regulators and/or the Ministers of Health and Transport in order to assist those actors with the “exercise of their powers or the performance of their functions and duties” in relation to AI (s. 33.1). These new provisions strengthen a more horizontal, multi-regulator approach to governing AI which is an improvement in the Bill, although this might eventually need to be supplemented by corresponding legislative amendments – and additional funding – to better enable the other commissioners to address AI-related issues that fit within their areas of competence. The amendments also impose upon Commissioner 2.0 a new duty to report on the administration and enforcement of AIDA – such a report is to be “published on a publicly available website”. (s. 35.1) The annual reporting requirement is important as it will increase transparency regarding the oversight and enforcement of AIDA. For his or her part, the Minister is empowered to publish information, where it is in the public interest, regarding any contravention of AIDA or where the use of a system gives rise to a serious risk of imminent harm (ss. 27 and 28). Interestingly, AIDA, which provides for the potential imposition of administrative monetary penalties for contraventions of the Act does not indicate who is responsible for setting and imposing these penalties. Section 29(1)(g) makes it clear that “the persons or classes of persons who may exercise any power, or perform any duty or function, in relation to the [AMP] scheme” is left to be articulated in regulations. The AIDA also makes it an offence under s. 30 for anyone to obstruct or provide false or misleading information to “the Minister, anyone acting on behalf of the Minister or an independent auditor in the exercise of their powers or performance of their duties or functions under this Part.” This remains unchanged from the original version of AIDA. Presumably, since Commissioner 2.0 would exercise a great many of the oversight functions, this is meant to apply to the obstruction or misleading of the Commissioner – but it will only do so if the Commissioner is characterized as someone “acting on behalf of the Minister”. This is not language of independence, but then there are other features of AIDA that also counter any view that even Commissioner 2.0 is truly independent (and I mean others besides the fact that they are an employee under the authority of the Minister and handpicked by the Minister). Most notable of these is that should the Commissioner become incapacitated or absent, or should they simply never be designated by the Minister, it is the Minister who will exercise their powers and duties (s. 33(4)). In sum, then, the proposed amendments to AIDA attempt to give some separation between the Minister and Commissioner 2.0 in terms of oversight and enforcement. At the end of the day, however, Commissioner 2.0 is still the Minister’s hand-picked subordinate. Commissioner 2.0 does not serve for a specified term and has no security of tenure. In their absence, the Minister exercises their powers. It falls far short of independence.
Published in
Privacy
Tagged under
|
Electronic Commerce and Internet Law in Canada, 2nd EditionPublished in 2012 by CCH Canadian Ltd. Intellectual Property for the 21st CenturyIntellectual Property Law for the 21st Century: Interdisciplinary Approaches |