What is AI4PI?

Artificial Intelligence for Performance Improvement (AI4PI) is an ISPI initiative that bridges evidence-based practices with emerging AI capabilities. It promotes the thoughtful, ethical, and human-centered application of AI in performance improvement — grounded in research, guided by principles, and driven by results.

This page provides an overview of the AI4PI Framework.

Learn more about AI4PI

Jump to a Section

Principle #1: Ensure Human Oversight of AI4PI Solutions

Key concepts

Transparency, explain-ability, demystification, human-AI collaboration, performance excellence, ability to interrupt

Ensuring that AI decisions are transparent and understandable to all stakeholders is crucial. This involves demystifying AI processes and outcomes so that everyone involved can comprehend and trust the AI's recommendations. Additionally, the ability to interrupt AI processes ensures that human oversight can be maintained at all times.

So clients

• Can confidently rely on AI4PI solutions, knowing that AI decisions are transparent and understandable.
• Feel empowered to make informed decisions with the understanding that AI4PI solutions are demystified and accountable.
• Can trust the AI recommendations with transparent, evidence-based decision-making.

Examples

• An AI-powered customer service tool that provides clear explanations for its recommendations, allowing service agents to understand and validate the AI's decisions before communicating with customers.
• A manufacturing AI system that collaborates with human workers to optimize production processes, providing real-time insights and explanations for its suggestions.
• An AI4PI solution in a healthcare setting that works alongside medical professionals to diagnose patients, offering transparent reasoning and references for its diagnostic recommendations and allowing doctors to make well-informed decisions.

Back to Top ↑

Principle #2: Enforce Data Integrity, Protection, and Privacy

Key concepts

Data protection, confidentiality, internal vs. public servers, data provenance, evidence-based recommendation, data preparation, privacy, and security

Protecting the source of data and their confidentiality is paramount, from ensuring the traceability of AI decision-making to protecting personal and organizational confidential data. Users of the AI4PI solution should be able to obtain the sources and reference data that inform AI recommendations. Neither organizational data nor personal data should be shared with public Large Language Models or publicly accessible AI tools, but should remain protected within the organization’s firewall.

So clients

• Can confidently rely on AI4PI solutions, knowing the origin of the data and logic behind AI decisions.
• Are assured that their personal and company confidential data are secure and not shared with the public internet.
• Feel empowered to make informed decisions with the understanding that AI4PI solutions are transparent and accountable.
• Can trust the AI recommendations with transparent, evidence-based decision making.

Examples

• A HIPAA-compliant, AI-powered health diagnostic tool that provides clinicians with the actual source documents used for the diagnostic and recommendations.
• A financial AI application that tracks and reports the actual data used for its recommendations for investments.
• An AI4PI system that operates within the organization’s network firewall and without any access to the public internet. It uses company data to optimize operations while enforcing strict data confidentiality protocols.

Back to Top ↑

Principle #3: Evaluate and Minimize AI-Induced Biases

Key concepts

Potential ethical biases, diversity, non-discrimination, fairness, equity, inaccuracy, bias mitigation

AI4PI solutions should be proactively analyzed to mitigate potential biases that could lead to skewed results. This would prevent AI from perpetuating societal injustices, generating inaccuracies, or engaging in potentially discriminatory behaviors.

So clients

• Are aware and able to recognize the various types of potential biases (Shestakova, 2023).
• Can depend on AI to enhance workplace productivity without introducing or perpetuating bias.
• Have a reasonable confidence that AI-generated insights and recommendations are derived from balanced and fair algorithms.
• Benefit from AI tools that promote equality and objectivity in performance-related assessments.

Examples

• An AI performance monitoring tool is regularly checked to ensure it does not unfairly evaluate employees based on age, gender, or cultural background, maintaining a focus on individual merits and contributions.
• An AI4PI learning platform used for professional development is refined to eliminate biases in training material recommendations, allowing for a personalized and equitable learning experience.
• An AI-driven project management assistant is adjusted to ensure equitable distribution of tasks and recognition of team members’ contributions based on results or accomplishments, thereby enhancing team spirit and productivity.
• A study uncovered racial bias in a hospital algorithm that was supposed to flag patients needing extra care management. It mistakenly identified patients with higher healthcare costs, who were predominantly white, due to historical disparities in spending on Black patients' healthcare (published in the Wall Street Journal, 2 December 2023).

Back to Top ↑

Principle #4: Establish AI4PI Solution’s Governance and Risk Management Strategies

Key concepts

Governance, leadership, risk categorization, risk mitigation, banned use or misuse, harmful systems, accountability, worker and consumer protection, self-declaration, transparency, framework

It is essential to create a risk-aware governance framework that categorizes AI operations, ensures AI practices adhere to ethical standards, and protects fundamental human rights.

The governance framework should ensure that AI4PI solutions are evaluated and categorized according to the level of risk they pose to individuals, organizations, and society:

• Low or minimal risks: Unlikely to cause harm (e.g., internal chatbot accessing only non-confidential content).
• High risks (manageable): Could cause harm but are mitigable (e.g., AI used in bonus allocation decisions).
• Unacceptable risks: Too risky without human decision-making (e.g., AI-only medical treatment decisions).

It is essential to regularly review AI4PI solutions, as risk levels can change over time. Solutions must also clearly declare their nature:

• Identify themselves as non-human when interfacing with users or generating content.
• Disclose their associated risk levels.

AI uses that violate human rights or manipulate users subconsciously must be prohibited. Banned practices include social scoring, disinformation, disempowering workers, discrimination, or misuse of biometric data.

AI4PI solutions promote individual, societal, and environmental well-being

• Societal well-being: Promotes democracy, mental/physical health, equity, and social trust.
• Environmental well-being: Reduces environmental footprint across the AI lifecycle.

So clients

• Can identify high- vs. low-risk AI applications during design.
• Understand the ethical context and limitations of AI tools.
• Can trust that AI systems declare their role and nature clearly.
• Are protected from manipulation and AI-based harm.
• Engage in meaningful dialogue around ethical AI development and its impacts.

Examples

• Banned use: An AI performance review system trained on biased data that could lead to unfair evaluations.
• Declaration: A customer service chatbot that informs users it is AI-powered from the start of the interaction.
• Harmful system: An AI used in clinical decision-making includes governance controls to ensure it never acts without human approval.

Back to Top ↑

Process Step #1: Determine Needs, Organization’s Readiness, and Potential Applications

This step focuses on identifying where AI can add value and whether the organization is ready to implement AI-driven solutions. It uses performance analysis to uncover needs, clarify opportunities, and ensure AI aligns with strategic goals and change capacity.

Key Concepts

The deployment of Human-Centered AI should be purposefully aligned to address specific, identified needs or opportunities. The needs analysis should not only consider short-term factors but also analyze the root causes of the performance gaps and anticipate evolving requirements, ensuring its continuous relevance and value enhancement over time. ISPI’s Performance Improvement standards should be applied to conduct a performance analysis, quantify the performance gaps, and reveal the actual root causes.

So clients

• Evaluate their organization's readiness and potential challenges to embrace a new AI4PI solution.
• Have identified and articulated the specific need, opportunity and root-causes that the AI4PI solution will be addressing.
• Understand how AI can augment the capabilities of individual workers and teams and improve operational systems.
• Recognize the potential of AI to reorganize workplace systems.

Solutions Address Three Levels of AI4PI Needs

• Level 1 – Worker/Individual: AI assistants that support personal or role-specific productivity.
• Level 2 – Work/Workplace Systems: AI automators that change workflows, team structure, or business processes.
• Level 3 – Society/Mega Impact: AI that transforms education, health, policy, or sustainability at a macro level.

Back to Top ↑

Process Step #2: Define Efficient and Effective Human-Centered AI4PI Solutions

Key concepts

customization, evolving needs, unique requirements, fine-tuning, Retrieval Augmented Generation (RAG), prompt engineering

AI4PI solutions should be customized to serve the specific needs of the organization and/or realize new opportunities. This means that the solution should be tailored to the organization's unique challenges and use relevant organization data in its responses. For example, if an organization is facing a problem with employee turnover, the solution should be designed to address the specific factors and root causes that are causing employees to leave.

Customizing a solution to meet the organization's specific needs increases AI accuracy, likelihood of success, solution effectiveness, and overall impact.

So clients

• Benefit from AI4PI solutions that are tailored to their specific needs and requirements.
• Obtain specific, customized responses to their unique challenges.
• Experience a productive, symbiotic relationship between workers, the work, the workplace, and AI. 
• Engage with AI technologies that are continuously refined to address evolving individual and organizational needs.

Methodologies to customize an AI4PI solution include

• Shot prompting: Uses the same API conversation ID. This means that the AI model is able to access the context of the previous conversation, which can help it to generate more relevant and informative responses.
• Retrieval-Augmented Generation (RAG): Extracts relevant knowledge from a dataset and combines it with the AI response to produce a custom output. This can help make AI responses more personalized and tailored to the individual user.
• Fine-tuning: This process adapts a model by making small adjustments with a small amount of task-specific information. It can help improve the AI model's performance on a specific task, such as generating text or answering questions.

Effective Prompt Engineering techniques

• Clear Instructions: Provide concise and precise instructions to the AI, including details, desired format, and length of output, to get more relevant and accurate answers.
• Reference Text: Supply the AI with reference texts, citations, or examples to guide its responses, particularly when there's a risk of fabricating information.
• Splitting Complex Tasks: To reduce error rates and improve response accuracy, break down complex tasks into simpler steps, using the output of earlier tasks as inputs for later ones.
• Time to 'Think': Allow the AI to process information thoroughly by structuring prompts that encourage thoughtful responses. These prompts could involve inner monologue sequences or questions that guide the model's reasoning.
• External Tools: Enhance AI capabilities by integrating external tools for tasks better suited to specialized functions, such as code execution engines or embedding-based search for document retrieval.
• Testing: Measure the performance of prompt modifications systematically, using a comprehensive testing methodology, to ensure prompt adjustments lead to overall performance improvement.

Back to Top ↑

Process Step #3: Design and Develop the AI - Human Collaboration in the AI4PI Solution

Key concepts

user interface (UX), usability, user experience, context, point of need, optimum collaboration, unique human skills

Engineering human-machine user interfaces should be done carefully to ensure an adequate, relevant and contextually appropriate use of the AI4PI solution.

So clients

• Can interact with AI in a way that feels natural and produces worthy outcomes at the point of need.
• Have confidence in the AI's ability to understand and respond to complex needs with precision.
• Benefit from AI interfaces that are tailored to specific tasks, enhancing user experience and performance.

Examples

• A company providing AI-powered internal chatbots should ensure that their chatbots are tailored to the actual needs of their employees. For example, a chatbot about pay and benefits should be tested to answer key concerns and questions of the workforce, with the goal of improving employees’ morale.
• A hospital that uses AI-powered medical imaging software ensures that it is easy to use for radiologists and other healthcare professionals. The software should be able to interpret images quickly and accurately and provide clear and concise reports that can be effectively used by medical professionals.

Back to Top ↑

Process Step #4: Document the Solution Accuracy and Trustworthiness

Key concepts

accuracy measurement, reliability testing, stress test, confidence level, hallucination, confusion matrix, technical robustness

The accuracy of the AI output should be systematically assessed using a thorough, step-by-step methodology. The findings of the Accuracy evaluation should be reported to the organization's leadership and AI4PI solution users.

The goal is to ensure that AI actions are reliable and conform to the expected standards of operation. A systematic plan should be implemented to measure and improve AI accuracy (KPI) over time. For instance, implementing a feedback loop into the AI4PI solution can minimize instances of AI 'hallucinations' and incorrect responses.

• Clients have confidence in the accuracy and reliability of AI outputs.
• They understand the continuous improvement process of enhancing AI performance.
• They are assured of AI4PI solutions' adherence to established standards through regular, transparent accuracy assessments.

Methodology for measuring and improving AI accuracy over time

• Compare expert (SME) recommendations to AI-generated outputs to assess initial AI accuracy.
• Use a confusion matrix to evaluate AI performance, calculating overall accuracy by considering true positives, false negatives, true negatives, and false positives.
• Conduct multiple rounds of testing to fine-tune the AI4PI solution parameters.
• Capture the corrections and insights into a database to help customize and fine tune the AI model.
• Conduct a stress test to validate the reliability or robustness of the AI4PI solution.

Back to Top ↑

Process Step #5: Deploy and Audit the AI4PI Solution

Key concepts

testing, audits, accountability, reporting, technical robustness, safety, AI solution levels, implementation plan, change management

The deployment of AI4PI solutions demands a rigorous testing and auditing process to confirm the compliance with ethical standards, operational effectiveness, and alignment with organizational and societal values. Regular and thorough auditing of AI systems is crucial to validate their performance, adherence to ethical norms and organizational principles.

These human-led audits should include extensive evaluations of the AI accuracy, reliability, usability in the users’ actual environment, personal data protection, bias reduction, and the examination of indirect, second-order impacts.

• Clients can trust that AI4PI solutions are functioning as intended and in a manner that upholds ethical standards.
• They have assurance in the AI's alignment with both organizational objectives and broader societal values.
• They receive transparent insights into the AI's performance and impact, fostering a culture of accountability.

Examples

• Data Protection Audit: An AI-driven health records system undergoes rigorous data privacy audits to ensure patient information is handled securely and in compliance with healthcare regulations.
• Bias Mitigation Audit.
• Risk Audit to correctly categorize the risk associated with the AI4PI solution.
• Usability Test.
• Accuracy Tests and Stress Tests.

Back to Top ↑

Process Step #6: Demonstrate Outcome and Overall Impact

Key concepts

performance metrics, results and outcome, first and second order impact on individual, organizational and societal well-being, ROI, tangible and intangible benefits

A thorough analysis of AI's impact is essential to mitigate risks and maximize the cost/benefit ratio. The impact evaluation should encompass the expected tangible and intangible benefits of the AI4PI solution, as well as second-order impacts and broader implications of AI on individuals, organizations, and society (three levels described in Standard 5).

Tangible benefits like improved work efficiency allow employees to spend more time on higher-value activities, such as relationship building, consulting, coaching, and mentoring, which enhances the impact of their work.

AI initiatives can also result in intangible benefits, such as improved customer satisfaction and brand reputation by allowing employees to spend more time with customers and less time with administrative duties. Although the intangible benefits of AI can be difficult to quantify, they can be significant and should not be overlooked when evaluating the ROI of an AI initiative.

• Clients gain a holistic view of how AI affects various aspects of work and life.
• They understand both the tangible and intangible benefits AI brings to their operations and overall mission.
• They can make informed decisions about AI investments, considering their full range of impacts.
• They have clear insights into positive and negative second-order impacts that could be detrimental to individuals, organizations, or society.

Examples

Impact on:

• Individuals: productivity improvement, time to focus on higher-order tasks (thinking, analyzing, solving, effectiveness improvement)
• Teams/Organizations: time to market, failure rates, revenue increase, operational margin, brand reputation, competitiveness, diversity and inclusion
• Society: net gain/loss in employment, environmental impact, growth opportunities versus widening gaps
• Tangible Benefits: An AI-enhanced project management tool increases efficiency, freeing up team members for high-value tasks like client relations and strategic planning.
• Intangible Benefits: Implementing an AI customer service platform leads to higher customer satisfaction and enhanced brand reputation.

Back to Top ↑

Thank you for exploring the AI4PI Framework

Together, we can ensure AI enhances—not replaces—human performance. If you're inspired to help shape the future of performance improvement through ethical and practical AI practices, we invite you to get involved.