Evaluation of the Entry/Exit Program

Final Report

February 2025

Program Evaluation Division, Internal Audit and Program Evaluation Directorate

Note: [redacted] appears where sensitive information has been removed in accordance with the Access to Information Act and the Privacy Act.

Table of Contents

Program overview, systems, and management

The CBSA’s Entry/Exit (E/E) program was established in 2021 for the Agency to collect more complete, accurate, and reliable travel history information for persons entering and departing Canada, enabling the Agency to “close the loop” on travellers’ movements into and out of the country. These more complete traveller records are intended for operational use, including for assessing traveller compliance, making admissibility decisions upon arrival in Canada, and enhancing intelligence and investigative processes.

The E/E program was delivered through a project that launched in 2013 and closed in 2021. It involved: investments in technology to improve the accuracy and reliability of entry information in the land mode and to reconcile entry and exit data; and amendments to legislation and establishment of Memorandums of Understanding (MOUs) to collect exit information from third parties in the land and air modes.

The CBSA collects traveller entry information in the land, air, rail and marine modes and receives exit information in the air and land modes from third parties as follows:

• In the air mode, exit data is collected from commercial air carriers who are legislatively required to provide biographic data for all passengers on outbound international flights, prior to and after departure. [Redacted]
• In the land mode, exit information is collected from the United States Customs and Border Protection (US CPB). Under the E/E program, entry data is collected and shared between the US and Canada through an MOU, such that a record of an entry into one country can be considered a record of exit from the other.

Prior to the establishment of the E/E program, the CBSA did not collect this traveller exit information.

Note: [Redacted]

Systems that support the E/E program

Data collection source systems:

The E/E information collected by CBSA or received from external partners is housed in source systems. These source systems are: Entry Exit Information System (EXIS), Data Acquisition System (DAS), and primary systems such as Integrated Primary Inspection Line (IPIL) and Primary Inspection Kiosk (PIK).

The entry and exit information in these source systems is reconciled by the Agency's matching solutions.

Matching passage information:

[Redacted]

User interface systems to access matched E/E data:

CBSA systems [Redacted] for E/E data include the Integrated Primary Inspection Line (IPIL) and Passage Query Tool (PQT). These systems are frequently used for traveller, intelligence and enforcement operations. E/E data matched through IBM MDM is shared with CBSA’s other government department (OGD) partners. CBSA’s PQT also has the option to use IBM MDM to access E/E data.

Program management

The E/E Program is managed by the Travellers Branch (TB). Internal stakeholders, who also receive funding to support E/E activities, include:

• The Information, Science and Technology Branch (ISTB), for database management and IT system support;
• The Strategic Policy Branch (SPB), for data analytics; and,
• The Intelligence and Enforcement Branch (IEB), for activities related to intelligence and targeting.

E/E data use

The E/E traveller history records are used by internal and external stakeholders for the management of the programs under their own mandates. CBSA staff use the data as follows:

Activity	Users	Systems accessed include:
Administer and enforce Canada's immigration and customs laws (e.g. check travel dates to determine duty and tax exemptions, ensure permanent residents reside in Canada for the required period of time in order to maintain eligibility, or identify individuals who do not leave Canada at the end of their period of authorized stay)	Travellers Branch Intelligence and Enforcement Branch Regional districts Regional Intelligence and Enforcement Operations Division	IPIL, PQT, and IBQ
[Redacted]
Share information with relevant Government of Canada (GC) departments and agencies for the management of their own programs	Travellers Branch	Source systems (EXIS, DAS, IPIL), or IBM MDM

The CBSA shares E/E information with the following external stakeholders (see details in Annex A ):

• Immigration, Refugees and Citizenship Canada (IRCC)
• Royal Canadian Mounted Police (RCMP)
• Canadian Security Intelligence Service (CSIS)
• Employment and Social Development Canada (ESDC)
• Canada Revenue Agency (CRA)
• US Customs and Border Protection (US CBP)
• Organizational structure

Organizational structure

Evaluation scope and methods

Scope

The CBSA’s E/E program provides data to other federal organizations; however, this was not a horizontal evaluation. Given the E/E program has only been operational since 2021-2022, the evaluation conducted an implementation evaluation focused on assessing the performance of the CBSA’s E/E program, including:

• The adequacy of performance measurement, including data availability in support of the program’s key performance measures.
• The extent to which the E/E program is making progress towards achieving its expected results, with a focus on the immediate outcomes:
  • Ability to satisfy commitments to partners and stakeholders
  • Closing the loop on traveller history in the land and air mode
• The extent to which the resources and tools are in place to support efficient and effective delivery of the E/E program.
• Identifying any course corrections that could improve the program or help realize the program’s intermediate outcomes.

Lines of evidence

The evaluation conducted a document review, literature review, stakeholder interviews and field research, as well as an operational and financial data review. Additional details of data collection methods used and data limitations can be found in Annex C.

Overall findings and recommendations

The E/E program has allowed for the Agency to further close the loop on travellers’ movements into and out of the country by collecting exit data in the land and air modes and matching it to existing traveller entry records. Access to both entry and exit data is important for supporting operational needs.

• The program has achieved its target for reconciling entry and exit records in the air mode.
• The E/E data is generally contributing as intended to supporting the needs of stakeholders within and outside the CBSA. OGD partners are able to leverage the data to support their national security, law enforcement, immigration, and social and income security mandates. Within the Agency, E/E data is used to support traveller processing at the border, and to assist inland enforcement in closing warrants.[Redacted]
• With respect to commercial intelligence, an unintended benefit of E/E data is its use in combination with commercial import and export data to provide a more holistic view of illicit commercial activities.

The evaluation made three recommendations to address the following areas for improvement:

• The program is not collecting all the required performance data to demonstrate the achievement of results and to make future course-corrections, where necessary.
• The CBSA is capturing traveller entry and exit data necessary to close the loop on the majority of travellers’ movements into and out of the country and progress is being made towards reconciling most entry and exit records using the IBM MDM system. However, reconciliation of data in the land mode requires some improvement, [Redacted]
• While the IBM MDM system is able to systematically match the majority of entry and exit passage data to create traveller history records, Agency stakeholders do not generally use the system’s matched records. [Redacted]
• Program expenditures do not align with the roles and responsibilities of Agency stakeholders, as identified at E/E Project closure. Not having updated roles and responsibilities for the E/E program in its steady state makes it difficult to determine if all funded activities are required for program delivery. There is also a lack of clarity around decision making and governance, mainly for the infrastructure technology component of the program.

Despite these challenges, the E/E program continues to be useful to the Agency, as it provides the necessary data for operations, intelligence, as well as for meeting the needs of external stakeholders.

Additional considerations

Recommendations and summary of associated findings

The program’s Performance Measurement Framework requires improvement

The E/E program does not have a robust performance measurement framework in place to ensure results align to program objectives, to collect performance data, and to track the achievement of immediate and intermediate results. The evaluation found different Key Performance Indicators (KPIs) across numerous draft and official documents that are not consistent and do not align to one another.

The evaluation focused on assessing against the KPIs found in the 2018 Treasury Board Submission Results Annex. Of the ten KPIs developed to measure E/E program results, four out of six immediate outcome KPIs could be measured by the evaluation team; however, only one out of four intermediate outcome KPIs could be measured using the data and methodology identified by the program. In measuring these KPIs, challenges were raised concerning data availability and related IT functionality, low prioritization of measuring and reporting on the KPIs, and knowledge of performance measurement roles and responsibilities.

An official set of KPIs to measure and report on the program’s ongoing results is required, as well as appropriate data collection, analysis, and reporting methodology.

[Redacted]

The investment into the IBM MDM system for reconciling and storing E/E data does not align with the operational use of the program’s data.

The implementation of the E/E program has allowed the CBSA to capture entry and exit data that provides a more complete traveller history in the land and air modes. The evaluation found that while entry and exit information is being used to support the Agency’s operations, the data that is reconciled by the IBM MDM system currently has limited application.

While the IBM MDM system’s rate of matching entry and exit information has improved over time, most of the Agency’s user interface systems are not linked to the IBM MDM matched data. [Redacted] Currently, overstays are being searched on a case-by-case basis, using E/E data in other systems.

The use of a master data management system is essential to entity resolution, whereby data specific to an individual can be captured, matched and stored. Entity Resolution is also an integral part of other important Agency initiatives (e.g. Traveller Modernization). However, given the limited use of matched IBM MDM E/E data for operational decision-making and its ongoing costs, it is unclear whether it is a cost-effective solution for the E/E program.

The use of E/E financial resources does not clearly align with the Program’s documented roles and responsibilities

During project closure, the project finalized a Responsible, Accountable, Consulted, and Informed (RACI) chart describing roles and responsibilities for the E/E program, post-implementation. The evaluation, however, found that some of the funded program activities did not align with the RACI document. For instance, some stakeholders were coding expenditures against the E/E program but were not identified in the RACI as playing a role in program delivery. It was suggested that the RACI chart is outdated and does not include all stakeholders currently supporting the E/E program, which has led to lack of clarity regarding roles and responsibilities of the program in its steady state, including for the management of the IBM MDM system. This could lead to challenges in determining whether all funded activities are contributing to effective program delivery.

[Redacted]

Consideration [Redacted]

The E/E program enables the CBSA to close the loop on the majority of travellers’ movements into and out of the country, through entry information collected in all modes, and exit information collected in land and air modes (the modes used most frequently by travellers). [Redacted]

Management Response and Action Plan (MRAP)

[Redacted]

Management response: The Vice President of Travellers Branch agrees with recommendation one.

Travellers Branch will conduct a thorough review of the current logic model and existing KPIs to ensure they align with the Entry/Exit program. The Travellers Policy and Programs Directorate (TPPD) will also work with implicated branches to draft, revise, and approve the final logic model and measurable KPIs that reflect the program's current state. Travellers Branch will continue to report on previously-developed KPIs contained within the Treasury Board Submission and the Benefit Realization Plan.

Travellers Branch will work with implicated branches to develop a mechanism to regularly collect performance data and report on the achievement of results.

[Redacted]

Management response: The Vice President of Travellers Branch agrees with the recommendation.

The Travellers Branch, in conjunction with Information Science and Technology Branch, Immigration and Enforcement Branch, Commercial and Trade Branch, and Strategic Policy Branch, will assess the performance and cost of the IBM MDM system for creating and managing unique IDs for travellers as well as evaluate how the IBM MDM system is enabling other business processes in the Agency.

The Travellers Branch will lead an assessment to determine how to proceed with the storage, reconciliation and use of E/E data, including impacts for data sharing with OGD stakeholders.

[Redacted]

Management Response: The Vice President of Travellers Branch supports the recommendation to review and update the E/E program Responsible, Accountable, Consulted, and Informed (RACI) chart and will work with internal stakeholders to ensure alignment between roles, responsibilities, and financial resource allocation for the Entry/Exit program.

A full review will be done and an updated RACI matrix will be created to accurately reflect the current state and will be approved by relevant branches.

Consideration: The Vice President of Travellers Branch should consider conducting the exploratory work on the feasibility of collecting exit information [Redacted].

Management response: The Vice President of Travellers Branch supports the consideration to conduct the exploratory work referred to in the 2013 TB Submission to identify the necessary resources and program changes required [Redacted] , including:

• Necessary authorities;
• System and IT infrastructure requirements; and
• The level of risk that remains [Redacted], and whether it aligns with the Agency’s risk threshold.

Detailed findings and supporting evidence

Finding 1: The E/E program does not have a robust performance measurement framework in place to collect performance data and track results achievement.

Program performance measurement documents were drafted in 2016, including a logic model, performance measurement strategy, indicator control sheets, and standard operating procedures. However, these documents do not reflect the program in its steady state.

A 2018 Treasury Board Submission Results Appendix listed expected results and contained several KPIs, which were not consistent with those developed in 2016. Interviewees thought the 2016 and 2018 KPIs were developed without fully understanding what the program would look like post-implementation.

Some stakeholders also expressed unfamiliarity with some of the KPIs under their responsibility and were not collecting related data or reporting against them. For instance, the business owner was not aware of the following KPIs, and the data needed to calculate them was undefined: [Redacted]

Some of the challenges with tracking results against existing E/E KPIs identified by interviewees were:

• Lack of available data;
• Lack of functionality in the current IT systems to generate reports;
• Lack of interoperability and communication silos between IT systems; and,
• Lack of prioritization of measuring and reporting on KPIs.

Of the ten immediate and intermediate program KPIs, only half could be measured using the identified methodologies and data elements. Annex B provides additional details on the challenges related to collecting E/E program results information.

Finding 2: Through the E/E Program, the CBSA is capturing traveller entry and exit data necessary to close the loop on the majority of travellers’ movements into and out of the country. [Redacted]

Entry Data

All modes: The E/E program surpassed its target of 95% for the systematic capture of entry information across all modes, meaning entry records are: captured automatically using document scanning technology; or typed into the system by officers. [Redacted]

Figure 1: % of biographic entry records systematically captured by the CBSA for all persons entering Canada (KPI 1.3).

Image description

The vertical bar graph consists of two bars. The first bar represents the capture of entry information across all modes in 2021-2022; the second bar represents the capture of entry information across all modes in 2022-2023.

The first bar, depicting 2021-2022 data, is divided into three categories.

The first category, starting at the base of the bar, comprises 92% of the bar and represents the category of entry information “Systematically captured through automated scanning technology”.

The second category, starting where the first category ended, comprises 7% of the bar and represents the category of entry information “Systematically captured by BSOs”.

The third, and final category, starting where the second category ended, comprises 1% of the bar and represents the category of entry information “Manually captured”.

The second bar, depicting 2022-2023 data, is divided into three categories.

The first category, starting at the base of the bar, comprises 89% of the bar and represents the category of entry information “Systematically captured through automated scanning technology”.

The second category, starting where the first category ended, comprises 8% of the bar and represents the category of entry information “Systematically captured by BSOs”.

The third, and final category, starting where the second category ended, comprises 3% of the bar and represents the category of entry information “Manually captured”.

Source : Passage Self Service Reporting, November 2023

Data available from 2011-12 through the 2013 TB Submission (Project and Expenditure Approval for the Entry/Exit Project) on the Agency’s rates of systematic capture of entry information show that at that time, 94.8% of entry information was captured systematically in air mode, and 72.5% in land mode.

Investments at land border crossings under the E/E program to improve CBSA’s digital collection of entry data included the deployment of technology such as: portable handheld devices to facilitate traveller processing, Licence Plate Readers, and scanning equipment for Radio Frequency Identification (RFID) enabled documents.

Exit Data

Air mode: The evaluation found that the CBSA is mostly receiving air exit information from air carriers within regulated timelines for all international commercial flights departing from Canada. By 2022-23, the E/E program had achieved its compliance target for carriers’ provision of required data on international flights departing from Canada.

Figure 2: % of post-departure air exit information received from air carriers within the regulated timelines for all international flights departing from Canada (KPI 1.4).

Image description

The vertical bar graph consists of two bars and a horizontal dashed line across the top of the bar graph indicating a compliance Target of 95%. The first bar represents compliance for carriers’ provision of required data on international flights departing from Canada in 2021-2022. The first bar reflects a 94% overall compliance rate, falling 1% below the dashed compliance Target line at the top of the graph.

The second bar represents compliance for carriers’ provision of required data on international flights departing from Canada in 2022-2023. The second bar reflects a 97% overall compliance rate, surpassing the dashed compliance Target line at the top of the graph.

Source: Air compliance data provided by Travellers Branch, March 2024

Pre-departure information is to be provided to the CBSA by commercial airlines at the following time intervals [Redacted]

Land mode: The CBSA receives biographic information (e.g. name, date of birth, nationality, etc.) from the US CBP through secure electronic channels for all travellers crossing from Canada into the US by land and vice versa. This information provides the CBSA a relatively complete dataset for travellers exiting Canada via the land mode. Interviewees agreed this data is received by the CBSA in a timely manner and in accordance with the MOU.

[Redacted]

Currently, the CBSA does not collect biometric entry or exit information, except in very specific circumstances, such as for immigration related documentation.

IBM Master Data Management (MDM) System

The IBM MDM [Redacted]

In order to close the loop on traveller history, entry and exit data is automatically reconciled using a data management system developed by IBM. The IBM MDM system was initially acquired by the Agency to support commercial programs. It was later leveraged for managing traveller data collected by the E/E program. [Redacted]

Finding 3: While the program has achieved its target for reconciling entry and exit records in the air mode and is making progress in the land mode [Redacted]

[Redacted]^{Footnote 1}

[Redacted]^{Footnote 2}

KPI 1.1: % of US entry records successfully reconciled against a traveller entry record previously acquired by the CBSA.

KPI 1.2: % of post-departure air exit records successfully reconciled against a traveller entry record previously acquired by the CBSA.

What can impact the IBM MDM matching?

[Redacted]

Data Stewards

Data Stewards manage the quality of the traveller data in the IBM MDM database. They support the IBM MDM matching [Redacted] by reviewing suspected duplicate records identified by the system. [Redacted]

In 2023-24, a total of 712,901 suspected duplicate tasks were sent to Data Stewards for review, representing 0.5% of all entry and exit passages. These tasks were dealt with by the Data Stewards as follows:

• 365,341 Merged using auto-collapse
• 120,661 Tasks completed by Data Stewards
• 226,899 Suspected duplicates not reviewed/collapsed by Data Stewards because there was insufficient capacity

The auto-collapsed and manually merged tasks contribute to the match rate and remove duplicate records (otherwise known as CBSA Party Identifiers or CPID(s)) from the system. Without the supplemental matching done by the Data Stewards, duplicates would remain unmatched, leading to less complete records.

With current capacity of 10 Data Stewards, between 150,000 and 160,000 tasks can be manually reviewed/collapsed each year. The remaining tasks are carried over and continue to wait in the worklist.

Finding 4: The data collected by the E/E program is being used by CBSA staff for operational decision-making. However, the data being used is not the matched data saved in the IBM MDM system.

Use of E/E data matched in IBM MDM

In order to access the matched data stored within the IBM MDM system, it must be linked to a user interface system that will display the data. However, most of the user interface systems at the CBSA are not linked to the data saved in the IBM MDM system; the PQT system is the only one that uses the IBM MDM matched data. [Redacted] User data indicates that throughout 2022-23 and 2023-24, 3.4% of the traveller history data accessed by CBSA operational staff from PQT was extracted from the IBM MDM system.

[Redacted]

Instead, CBSA staff use E/E data matched through alternative, CBSA-designed matching solutions. These were developed prior to E/E program implementation, in response to BSOs identifying gaps in travellers’ passage history when using the IBM MDM matched data. Like IBM MDM, these alternative matching solutions retrieve E/E data directly from source systems (e.g. EXIS). However, unlike IBM MDM, which requires exact matches of specific data, the CBSA-designed matching solutions [Redacted] Data users expressed a strong preference [Redacted] It also allows intelligence officers to access more data points than what the IBM MDM system is able to provide. It was noted, however, that the use of [Redacted] in order to find potential matches.

CBSA E/E Data Use

All interviewees noted that having access to both entry and exit data was important for supporting operational needs.

The E/E program provides data that was not previously available [Redacted]. CBSA staff stated that the exit records collected in air and land modes provide more complete traveller history information to track and conduct analysis of an individual’s movements into and out of Canada. It also helps determine how long an individual has been out of the country, [Redacted]. BSOs stated that they find this data useful for making admissibility decisions.^{Footnote 3}

In the air mode, CBSA operational staff indicated that the pre-departure information provided by commercial airlines is also contributing towards interception of outbound high-risk travellers and lookouts. The Targeting Travellers Intelligence (TTI) unit within IEB explained that the unmatched entry and exit records from source systems are regularly used and have contributed to intelligence leading to cannabis seizures and arrests.

[Redacted]

Finding 5: The E/E program is achieving other intended and unintended benefits for the CBSA.

Intended Benefits: Aside from benefits related to traveller processing and risk assessment, the E/E data is also used by the CBSA [Redacted]

Unintended Benefits: With respect to commercial intelligence, traveller entry and exit data provided an unintended benefit. [Redacted]

Finding 6: The IBM MDM matched data, combined with information from IRCC, was intended to support the automatic identification of overstays; however, this functionality is not available.

[Redacted]

Finding 7: The Agency is spending resources on the IBM MDM system for E/E data management; however, the value of IBM MDM for the E/E program is unclear given its limited operational use for decision making.

Overall cost and use of the IBM MDM system

In 2022-23, the Agency spent $6.53M on the IBM MDM system, which supports programs under its Commercial and Traveller streams. Of this total, $1.06M was spent in salaries and wages for the Data Stewards supporting E/E data matching, and $0.33M on licenses for traveller records. About $0.7M was spent on commercial licences, and the remaining $4.44M was spent on overall system maintenance and support. By 2023-24, the cost of the IBM MDM system increased to $8.45M, and the cost of CPID licenses had almost doubled.

Within the CBSA, the E/E data matched by the IBM MDM system is rarely used for traveller processing and intelligence purposes. [Redacted] . This disparity between investment and utilization raises questions about the value for money of the current E/E data management approach. Furthermore, the IBM MDM Lean review (June, 2024) suggested that IBM MDM is fully depreciated and there is a need to modernize the current aging infrastructure and architecture.

Outside IBM MDM, entry and exit data is also matched using CBSA in-house solutions, suggesting that redundancies may also exist. This highlights the need to identify whether these systems/functionalities are duplicative and whether efficiencies could be achieved.

It is important to note that at the time of this evaluation, the Agency was piloting the use of the IBM MDM system for its Traveller Modernization initiative. Given this initiative and the current use of IBM MDM for other data in the Agency, interdependencies may also exist and should be considered when determining if the IBM MDM system is the right data management system for the E/E program.

Licensing agreement

It is unclear if the current IBM MDM licensing agreement for E/E records management is cost-effective for the CBSA. Under the current licensing agreement with IBM, two types of license fees are paid by the CBSA: one for new licenses purchased, and one for annual maintenance of the licenses. The IBM MDM system requires a new license when:

• A new, unique traveller enters or exits Canada. As the number of unique travellers increases over time, more CPIDs are created in the system, requiring more licenses. Data analysis shows that, from 2022-23 to 2023-24, there was a 26% increase in the number of unique travellers entering or exiting Canada.
• The IBM MDM system is unable to match a passage to an existing traveller record and creates a ‘suspected duplicate’ record (due to unresolved data entry issues, such as errors or misspelling of names) that need to be resolved by data stewards. When one traveller has a duplicate CPID assigned to them in the IBM MDM system, the Agency must purchase more licenses than are necessary.

While actual program spending was within the allocated budget in 2021-22 and 2022-23 (99.7% and 91.2% of planned spending, respectively), under the current licensing agreement, program costs will increase over time. Since 2017, the Agency has paid $3,381,664 in annual licensing fees and to acquire new licenses as needed. It is expected that starting in 2025, additional funding for licenses will be required to accommodate the growing number of CPIDs. The program does not have a source of funding for E/E data storage and matching starting in fiscal year 2026-27, as most of the funding is currently sourced through B-based project funds.

The number of licenses required each year depends on the number of new travellers exiting and entering Canada. It is also not possible to forecast the cost of licenses, as the price that IBM sets for new licenses varies based on the number of licenses purchased and the negotiated price with the vendor. The IBM MDM Lean review (June, 2024) suggested the current E/E program allocation will not be sufficient to meet increasing costs over time.

Finding 8: The E/E program has been able to satisfy data sharing commitments to partners and stakeholders.

Data sharing with OGDs

E/E data sharing with OGD partners is intended to support the administration of their respective mandates (see Annex A). ESDC and CRA receive batch data from the CBSA’s IBM MDM solution. IRCC also queries IBM MDM data through their GCMS solution. This data supports their eligibility assessments for programs with residency requirements.

As the Agency mostly uses CBSA in-house matching solutions to reconcile entry and exit data, the travel history used by CBSA may not always match the travel history shared with OGDs that was matched through IBM MDM. [Redacted] While the rate of inaccurate identification could not be verified by the evaluation, this perceived unreliability of the data could be limiting the use of the data by OGD partners. Achieving the 95% match rate in IBM MDM could contribute to minimizing the issue. At the same time, E/E data provides an indication of compliance only and should be used by OGDs as such.

One of the key intended uses of the E/E data is to support IRCC immigration investigations and verification of compliance with eligibility requirements for their programs.

E/E data supports IRCC in determining: compliance with temporary resident authorized periods of stay; compliance with residency obligations; and, compliance with citizenship requirements. IRCC staff reported they use E/E data regularly and it is helpful in verifying information provided by clients.

IRCC stakeholders also suggested that the MDM matched data provided by the CBSA can sometimes be inconsistent with data from other sources, requiring IRCC officers to verify the data before using it.

All GoC partners interviewed expressed appreciation for the data and its benefit to achieving their national security, law enforcement, immigration, and social and income security mandates.

US CBP interviewees noted that the land data provided by the CBSA contributes to a more complete data set that is useful for seeing travel patterns.

OGD partners indicated they have key points of contact within the CBSA and the E/E program, with whom they engage as needed.

Some OGD partners expressed a desire for more frequent updates on key issues from the CBSA and for a forum for partners to engage in areas of mutual interest.

Finding 9: There is a desire from OGD partners to leverage E/E data for additional uses, though these uses are not permitted under the program’s data sharing authorities.

Partners noted that access to more E/E data could better support their needs, as follows:

• IRCC: Access to aggregate E/E data. IRCC’s GCMS is able to generate information on a case-by-case basis however, it is not able to provide aggregated E/E data. Having aggregated E/E data would contribute to large scale analysis (such as comparing the number of overstays by country) for policy development, and reporting.
• ESDC: Expanded access to E/E data to support the Temporary Foreign Workers (TFW) program. Interest was expressed in the TFW program accessing E/E information on when a worker has entered the country and left the country. The program currently relies on IRCC sharing information on a case-by-case basis. Having access to aggregated E/E data would contribute to a more holistic delivery of the program, in accordance with the Office of the Auditor General’s 2021 Report 13 ^{Footnote 4} .

The CBSA is limited by the Customs Act and the Exit Information Regulations that govern E/E information sharing, as well as by the Agency’s MOUs with OGDs, and is unable to fulfill these requests without changes to the corresponding authorities. Interviewees raised concerns around the limitation of the legislation for expanded information sharing.

Finding 10: Program stakeholders identified other operational challenges that prevent the optimal use of E/E data.

Stakeholders suggested the following could improve the quality and use of data for improved border management:

• 1) Clarification and communication of authorities
  When the Customs Act was amended to support the Entry/Exit program, it also solidified the Agency’s authority to intercept outbound travellers and goods. Officers can require goods to be reported despite any exemptions under the Act, allowing improved monitoring of exported goods. Officers can also examine goods whether or not they are reported, expanding the type of exported goods that can be examined as not all exported goods are reported. In addition, the amendment criminalized smuggling or attempting to smuggle prohibited, controlled, or regulated goods out of Canada. Prior, this only applied to goods coming into Canada.
• [Redacted]
• 3) Collection of exit information to match entry information.
  Under the Customs Act and the Exit Information Regulations, the Agency can collect only biographic information for those exiting Canada but can collect additional information on entry (e.g. more air passenger information such as Passenger Name Record (PNR) information, biometrics, and license plate information). Interviewees, including some OGD partners, expressed a desire to address this gap given their use of E/E data for intelligence purposes. Internally, stakeholders expressed that gathering this additional exit data may be used to improve IBM MDM matching of traveller passages.
• [Redacted]. It can also help to match entry and exit data for a traveller who may be using different travel documents at the land border if they are travelling in the same vehicle.
• However, amendments to the Act and Regulations, may be necessary to expand the type of exit information collected.

Finding 11: The use of E/E financial resources does not clearly align with the Program’s documented roles and responsibilities, leading to challenges in determining whether all funded activities are essential to program delivery.

The E/E program was implemented through a project that closed in 2021-22. The E/E Project Team developed a RACI chart describing the steady state roles and responsibilities across Agency stakeholders. However, when program expenditures were compared to the RACI document, it was not always clear why certain units were coding expenditures against the E/E program.

The E/E program receives approximately $9.5M annually. The average expenditure for the Entry/Exit Program (2021-2023) across internal stakeholders is as follows:

Figure 3: The average expenditure for the Entry/Exit Program (2021-2023) across internal stakeholders is as follows:

Image description

The horizontal bar graph represents the average expenditure for the Entry/Exit Program (2021-2023) across internal stakeholders.

The graph consists of four bars, each representing a branch of the CBSA.

The first bar, at the top of the graph indicates the Information, Science and Technology Branch makes up 66% of the average expenditure for the Entry/Exit Program (2021-2023).

The second bar, directly below the first bar on the graph indicates the Travellers Branch makes up 26% of the average expenditure for the Entry/Exit Program (2021-2023).

The third bar, directly below the second bar on the graph indicated the Strategic Policy Branch makes up 6% of the average expenditure for the Entry/Exit Program (2021-2023).

Finally, the fourth bar, directly below the third bar on the graph indicates the Intelligence and Enforcement Branch makes up 3% of the average expenditure for the Entry/Exit Program (2021-2023).

Source: Corporate Administrative System, November 2023

The IEB, SPB, and Travellers Branch units coding expenditures against the E/E program funding had clearly defined roles and responsibilities in the RACI and their contributions to E/E program delivery could be explained. However, for ISTB, more than half of the units coding expenditures against the E/E program did not have a specified role or responsibility identified in the RACI. Neither Travellers Branch (responsible for program management) nor ISTB could explain how some ISTB expenditures were directly related to E/E program delivery.

Travellers Branch interviewees suggested they also lack decision making authority to implement E/E system functionality releases, as these depend on ISTB performing scheduled system maintenance and upgrades.

Finally, there is also a lack of clarity around the IBM MDM system governance, responsibilities, contact points, and decision making. The IBM MDM Lean review also highlighted lack of awareness on the existence of governing bodies, responsible to make IBM MDM-related decisions.

Stakeholders suggested that the RACI is outdated and does not include all units currently supporting the E/E program. The Travellers Branch initiated a process to update the document in 2023, which could improve their understanding of program resource allocation.

Acronyms

API

Advance Passenger Information

B5

Border 5

BRP

Benefit Realization Plan

BSO

Border Services Officer

CBSA

Canada Border Services Agency

CPID

CBSA Party identifier

CRA

Canada Revenue Agency

CSIS

Canadian Security Intelligence Service

CTB

Commercial and Trade Branch

DAS

Data Acquisition System

E/E

Entry/Exit

EII

Enforcement Information Index

ESDC

Employment and Social Development Canada

EXIS

Entry Exit Information System

FCMB

Financial and Corporate Management Branch

GC

Government of Canada

GCMS

Global Case Management System

IBAS

Interdiction and Border Alert System

IBQ

Integrated Border Query

ICES

Integrated Customs Enforcement System

IEB

Intelligence and Enforcement Branch

IPIL

Integrated Primary Inspection Line

IRCC

Immigration, Refugees and Citizenship Canada

ISTB

Information, Science and Technology Branch

KPIs

Key Performance Indicators

MDM

Master Data Management

MOU

Memorandum of Understanding

NCMS

National Case Management System

NTC

National Targeting Centre

OGD

Other Government Department

OPI

Office of Primary Interest

PIK

Primary Inspection Kiosk

PIL

Primary Inspection Line

PNR

Passenger Name Record

POE

Port of Entry

PQT

Passage Query Tool

R&O

Rationalization and Optimization

RACI

Responsible, Accountable, Consulted, and Information

RCMP

Royal Canadian Mounted Police

RFID

Radio Frequency Identification

SOP

Standard Operating Procedures

SPB

Strategic Policy Branch

TB

Travellers Branch

TCI

Traveller Compliance Indicator

TFW

Temporary Foreign Workers

TM

Traveller Modernization

TTI

Targeting Travellers Intelligence unit

US CBP

U.S. Customs and Border Protection

Annexes

Annex A: E/E Data sharing with OGDs

Annex B: E/E Program Performance Measurement

Expected outcomes of the E/E program are as follows:

Immediate outcomes:

1. Improved ability to close the loop on traveller history in the land and air mode (outcome in the TB Sub)
   • The CBSA’s ability to “close the loop” is dependent on the systematic capture of entry information, the receipt of timely and accurate exit information from third-party partners (such as commercial airlines), as well as the CBSA’s ability to accurately reconcile (i.e. match) entry and exit records to the same individual. [Redacted]
2. Ability to satisfy commitments to partners and stakeholders (outcome from the E/E program Logic Model)
   • The CBSA shares E/E information with OGD partners through MOUs. The CBSA’s ability to share E/E data with OGD partners supports a government-wide approach to managing national security, as well as program integrity for certain benefit programs.

Intermediate outcomes:

1. Improved immigration enforcement (outcome in the TB Sub)
   • When the CBSA is able to identify people who have exceeded their period of authorized stay in Canada, it can direct immigration enforcement resources and activities accordingly.
2. Strengthened program integrity for the CBSA (border management) and partners (outcome developed based on the TB Sub and E/E program Logic Model)
   • The CBSA uses the E/E data for border management objectives and OGD partners use it for achieving their own program objectives such as increased compliance with Canada’s social benefit program eligibility requirements

Ultimate outcome:

1. Improved compliance with Canada’s immigration, citizenship, travel document laws and social benefit programs resulting in increased national security and public safety as well as savings for Canadians (outcome developed based on the E/E program Logic Model)

The evaluation came across the following challenges in measuring some of the E/E program KPIs (listed in the 2018 TB Submission):

Annex C: Evaluation Methodology and Limitations

Methodology

The evaluation used the following lines of evidence to gather information for the evaluation:

Document review: The evaluation team reviewed a number of program documents and correspondence to gather information about intended and emerging needs, performance measurement and progress towards results, as well as governance.

Literature Review: The evaluation team did a scan of existing literature to gather publicly available information about international practices.

Stakeholder Interviews & Field Research: The evaluation team conducted two regional site visits (Northern and Southern Ontario Regions) which included 11 group interviews with BSOs, enforcement officers and intelligence officers; 28 individual and group interviews with CBSA stakeholders from TB, IEB, ISTB, CTB, and FCMB; seven group interviews with OGD partners; and, two interviews with US CBP. Some additional responses were also collected in writing.

Operational and Financial Data Review: The evaluation team assessed available data from the Corporate Administrative System (CAS) on financial and allocated resources, as well as data from the following sources in order to determine progress towards results at the immediate level: Traveller Entry Exit, Primary Passage, Outbound Vetting - Hits and Traveller trip system.

Limitations

The evaluation team had difficulty finding sufficient literature to support some of the intended analysis to conduct a comparison with how other countries are collecting E/E data.

Performance data was available for only four of the six immediate outcome KPIs and one out of four intermediate KPIs. This limited the evaluation’s ability to fully assess the achievement of the intended outcomes.

Data was reviewed for two fiscal years (i.e. 2021-22 and 2022-23) as the E/E program only became operational in 2021-22. This limited the evaluation in identifying data trends to assess effectiveness and efficiency.

Some of the data was provided in calendar years and not FYs, limiting the ability to triangulate analyses.