Tags
access to information
AI
AIDA
AI governance
AI regulation
Ambush Marketing
artificial intelligence
big data
bill c11
Bill c27
copyright
data governance
data protection
data strategy
freedom of expression
Geospatial
geospatial data
intellectual property
Internet
internet law
IP
open courts
open data
open government
personal information
pipeda
Privacy
smart cities
trademarks
transparency
|
Monday, 11 March 2024 15:45
Investigation of AI-Enabled Remote Proctoring Software under Public Sector Privacy Law Leads to AI RecommendationsOntario’s Information and Privacy Commissioner has released a report on an investigation into the use by McMaster University of artificial intelligence (AI)-enabled remote proctoring software. In it, Commissioner Kosseim makes findings and recommendations under the province’s Freedom of Information and Protection of Privacy Act (FIPPA) which applies to Ontario universities. Interestingly, noting the absence of provincial legislation or guidance regarding the use of AI, the Commissioner provides additional recommendations on the adoption of AI technologies by public sector bodies. AI-enabled remote proctoring software saw a dramatic uptake in use during the pandemic as university classes migrated online. It was also widely used by professional societies and accreditation bodies. Such software monitors those writing online exams in real-time, recording both audio and video, and using AI to detect anomalies that may indicate that cheating is taking place. Certain noises or movements generate ‘flags’ that lead to further analysis by AI and ultimately by the instructor. If the flags are not resolved, academic integrity proceedings may ensue. Although many universities, including the respondent McMaster, have since returned to in-person exam proctoring, AI-enabled remote exam surveillance remains an option where in-person invigilation is not possible. This can include in courses delivered online to students in diverse and remote locations. The Commissioner’s investigation related to the use by McMaster University of two services offered by the US-based company Respondus: Respondus Lockdown Browser and Respondus Monitor. Lockdown Browser consists of software downloaded by students onto their computers that blocks access to the internet and to other files on the computer during an exam. Respondus Monitor is the AI-enabled remote proctoring application. This post focuses on Respondus Monitor. AI-enabled remote proctoring systems have raised concerns about both privacy and broader human rights issues. These include the intrusiveness of the constant audio and video monitoring, the capturing of data from private spaces, uncertainty over the treatment of personal data collected by such systems, adverse impacts on already marginalised students, and the enhanced stress and anxiety that comes from both constant surveillance and easily triggered flags. The broader human rights issues, however, are an uncomfortable fit with public sector data protection law. Commissioner Kosseim begins with the privacy issues, finding that Respondus Monitor collects personal information that includes students’ names and course information, images of photo identification documents, and sensitive biometric data in audio and video recordings. Because the McMaster University Act empowers the university to conduct examinations and appoint examiners, the Commissioner found that the collection was carried out as part of a lawfully authorized activity. Although exam proctoring had chiefly been conducted in-person prior to the pandemic, she found that there was no “principle of statute or common law that would confine the method by which the proctoring of examinations may be conducted by McMaster to an in-person setting” (at para 48). Further, she noted that even post-pandemic, there might still be reasons to continue to use remote proctoring in some circumstances. She found that the university had a legitimate interest in attempting to curb cheating, noting that evidence suggested an upward trend in academic integrity cases, and a particular spike during the pandemic. She observed that “by incorporating online proctoring into its evaluation methods, McMaster was also attempting to address other new challenges that arise in an increasingly digital and remote learning context” (at para 50). The collection of personal information must be necessary to a lawful authorized activity carried out by a public body. Commissioner Kosseim found that the information captured by Respondus Monitor – including the audio and video recordings – was “technically necessary for the purpose of conducting and proctoring the exams” (at para 60). Nevertheless, she expressed concerns over the increased privacy risks that accompany this continual surveillance of examinees. She was also troubled by McMaster’s assertion that it “retains complete autonomy, authority, and discretion to employ proctored online exams, prioritizing administrative efficiency and commercial viability, irrespective of necessity” (at para 63). She found that the necessity requirement in s. 38(2) of FIPPA applied, and that efficiency or commercial advantage could not displace it. She noted that the kind of personal information collected by Respondus Monitor was particularly sensitive, creating “risks of unfair allegations or decisions being made about [students] based on inaccurate information” (at para 66). In her view, “[t]hese risks must be appropriately mitigated by effective guardrails that the university should have in place to govern its adoption and use of such technologies” (at para 66). FIPPA obliges public bodies to provide adequate notice of the collection of personal information. Commissioner Kosseim reviewed the information made available to students by McMaster University. Although she found overall that it provided students with useful information, students had to locate different pieces of information on different university websites. The need to check multiple sites to get a clear picture of the operation of Respondus Monitor did not satisfy the notice requirement, and the Commissioner recommended that the university prepare a “clear and comprehensive statement either in a single source document, or with clear cross-references to other related documents” (at para 70). Section 41(1) of FIPPA limits the use of personal information collected by a public body to the purpose for which it was obtained or compiled, or for a consistent purpose. Although the Commissioner found that the analysis of the audio and video recordings to generate flags was consistent with the collection of that information, the use by Respondus of samples of the recordings to improve its own systems – or to allow third party research – was not. On this point, there was an important difference in interpretation. Respondus appeared to define personal information as personal identifiers such as names and ID numbers; it treated audio and video clips that lacked such identifiers as “anonymized”. However, under FIPPA audio and video recordings of individuals are personal information. No provision was made for students either to consent to or opt out of this secondary use of their personal information. Commissioner Kosseim noted that Respondus had made public statements that when operating in some jurisdictions (including California and EU members states) it did not use audio or video recordings for research or to improve its products or services. She recommended that McMaster obtain a similar undertaking from Respondus to not use its students’ information for these purposes. The Commissioner also noted that Respondus’ treating the audio and video recordings as anonymized data meant that it did not have adequate safeguards in place for this personal information. Respondus’ Terms of Service provide that the company reserved the right to disclose personal information for law enforcement purposes. Commissioner Kosseim found that McMaster should require, in its contact with Respondus, that Respondus notify it promptly of any compelled disclosure of its students’ personal information to law enforcement or to government, and to limit any such disclosure to the specific information it is legally required to disclose. She also set a retention limit for the audio and video recordings at one year, with confirmation to be provided by Respondus of deletions after the end of this period. One of the most interesting aspects of this report is the section titled “Other Recommendations” in which the Commissioner addresses the adoption of an AI-enabled technology by a public institution in a context in which “there is no current law or binding policy specifically governing the use of artificial intelligence in Ontario’s public sector.” (at para 134). The development and adoption of these technologies is outpacing the evolution of law and policy, leaving important governance gaps. In May 2023, the Commissioner Kosseim and Commissioner DeGuire of the Ontario Human Rights Commission issued a joint statement urging the Ontario government to take action to put in place an accountability framework for public sector AI. Even as governments acknowledge that these technologies create risks of discriminatory bias and other potential harms, there remains little to govern AI systems outside the piecemeal coverage offered by existing laws such as, in this case, FIPPA. Although the Commissioner’s interpretation and application of FIPPA addressed issues relating to the collection, use and disclosure of personal information, there remain important issues that cannot be addressed through privacy legislation. Commissioner Kosseim acknowledged that McMaster University had “already carried out a level of due diligence prior to adopting Respondus Monitor” (at para 138). Nevertheless, given the risks and potential harms of AI-enabled technologies, she made a number of further recommendations. The first was to conduct an Algorithmic Impact Assessment (AIA) in addition to a Privacy Impact Assessment. She suggested that the federal government’s AIA tool could be a useful guide while waiting for one to be developed for Ontario. An AIA could allow the adopter of an AI system to have better insight into the data used to train the algorithms, and could assess impacts on students going beyond privacy (which might include discrimination, increased stress, and harms from false positive flags). She also called for meaningful consultation and engagement with those affected by the adoption of the technology taking place both before the adoption of the system and on an ongoing basis thereafter. Although the university may have had to react very quickly given that the first COVID shutdown occurred shortly before an exam period, an iterative engagement process even now would be useful “for understanding the full scope of potential issue that may arise, and how these may impact, be perceived, and be experienced by others” (at para 142). She noted that this type of engagement would allow adopters to be alert and responsive to problems both prior to adoption and as they arise during deployment. She also recommended that the consultations include experts in both privacy and human rights, as well as those with technological expertise. Commissioner Kosseim also recommended that the university consider providing students with ways to opt out of the use of these technologies other than through requesting accommodations related to disabilities. She noted “AI-powered technologies may potentially trigger other protected grounds under human rights that require similar accommodations, such as color, race or ethnic origin” (at para 147). On this point, it is worth noting that the use of remote proctoring software creates a context in which some students may need to be accommodated for disabilities or other circumstances that have nothing to do with their ability to write their exam, but rather that impact the way in which the proctoring systems read their faces, interpret their movements, or process the sounds in their homes. Commissioner Kosseim encouraged McMaster University “to make special arrangements not only for students requesting formal accommodation under a protected ground in human rights legislation, but also for any other students having serious apprehensions about the AI-enabled software and the significant impacts it can have on them and their personal information” (at para 148). Commissioner Kosseim also recommended that there be an appropriate level of human oversight to address the flagging of incidents during proctoring. Although flags were to be reviewed by instructors before deciding whether to proceed to an academic integrity investigation, the Commissioner found it unclear whether there was a mechanism for students to challenge or explain flags prior to escalation to the investigation stage. She recommended that there be such a procedure, and, if there already was one, that it be explained clearly to students. She further recommended that a public institution’s inquiry into the suitability for adoption of an AI-enabled technology should take into account more than just privacy considerations. For example, the public body’s inquiries should consider the nature and quality of training data. Further, the public body should remain accountable for its use of AI technologies “throughout their lifecycle and across the variety of circumstances in which they are used” (at para 165). Not only should the public body monitor the performance of the tool and alert the supplier of any issues, the supplier should be under a contractual obligation to inform the public body of any issues that arise with the system. The outcome of this investigation offers important lessons and guidance for universities – and for other public bodies – regarding the adoption of third-party AI-enabled services. For the many Ontario universities that adopted remote proctoring during the pandemic, there are recommendations that should push those still using these technologies to revisit their contracts with vendors – and to consider putting in place processes to measure and assess the impact of these technologies. Although some of these recommendations fall outside the scope of FIPPA, the advice is still sage and likely anticipates what one can only hope is imminent guidance for Ontario’s public sector.
Published in
Privacy
Thursday, 24 March 2022 09:19
Anti-SLAPP application fails in remote-proctoring lawsuit
Note: My paper The Surveillant University: Remote Proctoring, AI and Human Rights is forthcoming in the Canadian Journal of Comparative and Contemporary Law. It explores a necessity and proportionality approach to the adoption by universities of remote proctoring solutions. Although the case discussed in the post below addresses a different set of issues, it does reflect some of the backlash and resistance to remote proctoring. In 2020, the remote AI-enabled exam proctoring company Proctorio filed suit for copyright infringement and breach of confidence lawsuit against Ian Linkletter, a BC-based educational technologist. It also obtained an interim injunction prohibiting Linkletter from downloading or sharing information about Proctorio’s services from its Help Center or online Academy. Linkletter had posted links on Twitter to certain ‘unlisted’ videos on the company’s YouTube channel. His tweets were highly critical of the company and its AI-enabled exam surveillance software. He responded to the suit and the interim injunction with an application to have the underlying action thrown out under BC’s Protection of Public Participation Act (PPPA). This anti-SLAPP (strategic litigation against public participation) statute allows a court to dismiss proceedings that arise from an expression on a matter of public interest made by the applicant. On March 11, 2022, Justice Milman of the BC Supreme Court handed down his decision rejecting the PPPA application. Linkletter first became concerned with Proctorio (a service to which the University of British Columbia subscribed at the time) after a University of British Columbia (UBC) student had her chat logs with Proctorio published online by the company after she complained about the service she received during an exam. In order to learn more about Proctorio, Linkletter developed a ‘sandbox’ course for which he was the instructor. This enabled him to access Proctorio’s online Help Center and its ‘Academy’ via UBC. These sites provide information and training to instructors. The Help Center had a number of videos available through YouTube. The URLs for these videos were unlisted, which meant that they were not searchable through YouTube’s site, although anyone with the link could access the video. Mr. Linkletter posted some of these links to Twitter, expressing his concerns with the contents of the videos. The company disabled the links, and created new ones. Linkletter also posted a screenshot of the Academy website with a message indicating that the original links were not available. Justice Milman did not hesitate to find that the applicant had expressed himself on a matter of public interest. He noted that the software adopted by UBC “has generated controversy, there and elsewhere, due to concerns about its perceived invasiveness and what is thought by some to be its disparate and discriminatory impacts on some students.” (at para 3). The onus shifted to the respondent Proctorio to demonstrate the substantial merit of its proceedings, the lack of a valid defence by the applicant, and the seriousness of the harm it would suffer relative to the public interest in the expression. The threshold to be met by Proctorio was to demonstrate “that there are grounds to believe that its underlying claim is legally tenable and supported by evidence that is reasonably capable of belief such that the claim can be said to have a real prospect of success” (at para 56). Proctorio’s lawsuit is essentially based on three intellectual property claims. The first of these was a breach of confidence claim relating to the unlisted YouTube video links. To succeed with this claim, the information at issue must be confidential; the circumstances under which it was communicated must give rise to an obligation of confidence; and the defendant must have made unauthorized use of the information to the detriment of the party communicating it. Justice Milman found that the respondent met the threshold of ‘substantial merit’ on this cause of action. What Linkletter posted publicly on Twitter were links to videos. Proctorio claimed that it was these videos (along with a screen shot of a message on its Academy website) that were the confidential information it sought to protect. Although there are a number of factors that a court will take into account in assessing the confidentiality of information, the information must have a confidential nature and the party seeking to protect it must have taken appropriate steps to protect its confidentiality. Unlisted YouTube video links are not publicly searchable, yet anyone with the link can access the content – and YouTube’s terms of service permit the sharing of unlisted links. However, Justice Milman found that Linkletter accessed Proctorio’s videos (and their links) via Proctorio’s website, which had its own terms of service to which Linkletter had clicked to agree. Those terms prohibit the copying or duplication of the materials found in their Help Centre – although they do not identify any of the content as confidential information. Canadian courts have found users of websites to be bound by terms of service regardless of whether they have read them; it is not a stretch to find that Linkletter had a contractual obligation not to share the contents. However, when it comes to taking the steps necessary to protect the confidentiality of information, one can question whether terms of service buried in links on a website – and that do not specifically identify the material as confidential – constitute a confidentiality or non-disclosure agreement. There was evidence that much of the material could be found elsewhere on the internet. It was also available to tens of thousands of instructors who were given access to the site at the discretion of university clients, not Proctorio. Justice Milman noted that “none of the videos stated on their face that they were commercially sensitive or should be kept from public view” (at para 64). He also found that “the choice to make them available on a public platform like YouTube when more secure options could have been used, dilutes the strength of Proctorio’s case” (at 64). In these circumstances, the court’s ruling that the confidential information claim had sufficient merit seems generous. In order to make out a claim of breach of confidence, it is also necessary for the plaintiff to show that the defendant made use of the information to the company’s detriment. Although the information was used to criticize the company, it is hard to see how Proctorio suffered any real damage particular to this breach of confidence. Much of the content was available through other sources, and the court described the company’s assertions that the videos could permit students to game their algorithms or could reveal their algorithmic secrets to competitors as ‘speculative’. Nonetheless, Justice Milman found enough here to satisfy the Proctorio’s onus to repel the PPPA application. The copyright infringement argument depended upon a finding that the sharing of a hyperlink amounted to the sharing of the content that could be accessed by following the link. In spite of the fact that there is Canadian case law that suggests that sharing hyperlinks is not copyright infringement, Justice Milman was prepared to distinguish these cases. He found it significant that the materials were not publicly available except to those who had access to the links; sharing the links amounted to more than just pointing people to information otherwise available on the internet. Having found likely infringement, Justice Milman next considered available defences. He found that Linkletter did not meet the test for fair dealing as set out by the Supreme Court of Canada in CCH Canadian. It was conceded by Proctorio that Linkletter passed the first part of the fair dealing test – that the dealing was for a purpose listed in ss. 29, 29.1 or 29.2 of the Copyright Act. Presumably it was for the purposes of criticism or comment, although this is not made explicit in the decision. In assessing the fair dealing criteria, however, Justice Milman found that Linkletter’s circulation of the links on social media mitigated against fair dealing, as did the fact that anyone who followed the link had access to the full work. On ‘alternatives to the dealing’, Justice Milman noted that rather than share the videos publicly, Linkletter could have reported on what he saw in the videos (although he earlier had found the videos (or the links to the videos – it is not entirely clear) to be confidential information). He could also have referred to other publicly available sources on the contents of the videos to make his point. On the issue of the nature of the work, Justice Milman found that the works were confidential (thus working against a finding of fair dealing) “even if most of the information in the videos was already available elsewhere on the internet”. Oddly, then, the fair dealing analysis not only underscores the fact that the material was largely publicly available, it suggests that an alternative to providing links to the videos was to discuss their contents freely. This suggests that the issue was not really the confidentiality of the content, but the fact that Linkletter had breached contractual terms of service in order to provide access to it. On the final fair dealing criterion, the effect of the dealing on the work, Justice Milman found that by making the videos available through their links, “Mr. Linkletter created a risk that Proctorio’s product would be rendered less effective for its intended purposes (because students could more easily anticipate how instructors can configure the settings) and its proprietary information more readily available to competitors.” (at para 112). He conceded that this risk was ‘speculative’ given the amount of information about Proctorio’s services already in the public domain. Justice Milman found that, on balance, the fair dealing defence was not available to Linkletter. He also found that the defence of ‘user-generated content’ was not applicable. Justice Milman declined to find that there had been circumvention of technical protection measures by Linkletter. He found that Linkletter had gained access to the materials by legitimate means. His subsequent copyright infringing acts were carried out without avoiding, bypassing, removing, deactivating or impairing any effective technology, device or component as required by s. 41.1 of the Copyright Act. The final element of the test under the PPPA is that the interest of the plaintiff in carrying on with the action must outweigh its deleterious effects on expression and public participation. Justice Milman found that this test was met, notwithstanding the fact that he also found that the “corresponding harm that Proctorio has been able to demonstrate is limited” (at para 124). He found that the risks identified by Proctorio of students circumventing its technology or competitors learning how its software worked were “unlikely to materialize”. Nonetheless, he found that Linkletter’s actions “compromised the integrity of its Help Center and Academy screens, which were put in place in order to segregate the information made available to instructors and administrators from that intended for students and members of the public” (at para 126). He credited the interim injunction for limiting the adverse impacts in this regard. However, he was critical of the broad scope of that injunction and narrowed it to ensure that Linkletter was not enjoined from sharing or linking to content available from public sources. Justice Milman also noted that Linkletter remained free to express his views, as have been others who have also criticized Proctorio online. The breach of copyright and breach of confidence claims in this case are weak, although their consideration is admittedly superficial given that this is not a decision on the merits. The court found just enough in the copyright and breach of confidence claims to keep them on the right side of the PPPA. Clearly Proctorio objects to the provision of direct public access to its instructional videos beyond the tens of thousands of instructors who have access to them each year – and who are apparently otherwise free to discuss their content in public fora. In this case, Proctorio quickly mitigated any harm by changing the links in question. It could also deny Linkletter access to its services on the basis that he breached the terms of use, and can better protect its content by no longer providing it on as unlisted content on YouTube. The narrowed injunction leaves Linkletter free to criticize Proctorio and to link to other publicly available information on the internet. In the circumstances, even if the underlying lawsuit is not a SLAPP suit, as Justice Milman concludes, it is hard to fathom why it should continue to consume scare judicial resources.
Published in
Copyright Law
|
Electronic Commerce and Internet Law in Canada, 2nd EditionPublished in 2012 by CCH Canadian Ltd. Intellectual Property for the 21st CenturyIntellectual Property Law for the 21st Century: Interdisciplinary Approaches |