FRANKFURT SCHOOL

BLOG

Auditing with intelligent co-pilots: Generative Artificial Intelligence in Audit
Executive Education / 28 November 2023
  • Share

  • 2533

  • 0

  • Print
Marco Schreyer holds a degree in business informatics and has several years of experience in forensic data analysis in auditing practice. He is currently researching the possible applications of deep learning methods in internal and external auditing at the University of St. Gallen (HSG). He is a lecturer on the "Certified Fraud Manager" and "Certified Audit Data Scientist" certificate programmes at Frankfurt School.

To Author's Page

More Blog Posts
Balancing work and study as a Master in Applied Data Science student
Digital Transformation and AI: A road map for industries and tech firms
Venturing into new dimensions: Virtual collaboration in 3D spaces

The use of artificial intelligence (AI) in auditing is increasingly providing valuable support for auditors in a wide range of audit activities. For example, AI models are already being used in the context of audits of annual financial statements for audit sampling[1], journal entry testing[2], or the audit of notes disclosures.[3]

Figure 1: Overview of co-piloted auditing in the context of the audit process

The idea of co-piloted auditing describes a paradigm shift in the auditing profession, in which auditors and AIs work together on the basis of so-called foundation models[4], to bring their complementary skills to the auditing process:

  • The auditor exhibits a profound client-specific understanding through their domain expertise and based on endogenous company data including the environment, bookings or contracts. This enables them to evaluate audit evidence in a business context, apply generally accepted auditing standards and exercise professional judgement.
  • The Foundation Model exhibits a broad knowledge base learned from large amounts of exogenous company data including financial news, financial reports or satellite images. This enables the model to evaluate a wide range of audit evidence, e.g. to identify anomalies, make initial assessments, and forecast trends.

Foundation Artificial Intelligence Models

At the same time, foundation AI Models, such as OpenAI’s GPT-4[5] or Google’s LaMDA[6] , can be adapted for a wide range of diverse tasks.

Generally, these models undergoe a two-stage training process. In the initial pre-training phase, the models are trained using large amounts of data to learn general patterns and structures. In the subsequent fine-tuning phase, the models are adapted using a small amount of domain specific data to learn how to solve a specific task. This two-step training process enables the fine-tuning of a pre-trained Foundation AI Model for a variety of different tasks, e.g. programming software, analysing customer feedback, or assisting in learning processes.

The ChatGPT[7] application, released by OpenAI, and the Bard[8] application, introduced by Google enable the fine-tuning of the underlying Foundation AI Model GPT-4 or LaMDA using so-called prompts. These prompts denote textual inputs that enable the model to learn from auditors through natural language. A series of such text entries, known as prompt protocols, can be used to fine-tune Foundation AI models for various audit tasks.

Foundation Models in the auditing practice

 An audit prompt protocol divides an audit task into successive learning steps. The auditor use the protocol to guide the fine-tuning of the Foundation AI model. In the audit practice, following a prompt protocol comprising the following five prompts has proven proven to be effective:

  • Prompt 1: Explanation of the task and objective.
  • Prompt 2: Explanation of the procedure for solving the task.
  • Prompt 3: Explanation of the input required to solve the task.
  • Prompt 4: Explanation of the output or the expected solution.
  • Prompt 5: Examples of task and solution.

After the successful fine-tuning, the Foundation Model is able to independently perform the learned testing task. Figure 2 shows the result of a fine-tuned GPT-4 model to perform Journal Entry Testing[9]. The Foundation Model acquired the skill to detect unusual entries in a financial accounting data by means of a corresponding audit prompt protocol.

Figure 2: Example result of a journal entry test according to ISA 240. Auditor’s prompt (top) and Foundation Model’s output (bottom)

Transforming the audit profession

The fine-tuning of Foundation AI Models is increasingly opening up new opportunities for auditors. Currently, the application still extends to supposedly basic tasks such as materiality assessment, transaction auditing and notes analysis. In the future, the co-piloted auditing paradigm will increasingly transform the audit profession through the application of artificial intelligence.

The Certified Audit Data Scientist certificate embraces this technological shift and teaches modern AI-supported audit procedures.

————————————————————————————————————————————————————————————————————————————–

This blog post is an excerpt of the following paper published by the author together with co-authors: “Artificial Intelligence Co-Piloted Auditing”, Gu, H., Schreyer, M. and Moffitt, K. and Vasarhelyi, M. A. (available online).

Marco Schreyer studied computer science and business adminstration with multiple years of experience in forensic data analysis in auditing. Currently, he’s researching how deep learning methods can be used in both internal and external audits at the University of St.Gallen and the Rutgers University. Additionally, Marco Schreyer teaches in the Certified Audit Data Scientist and Certified Fraud Manager certification courses at the Frankfurt School.

————————————————————————————————————————————————————————————————————————————–

[1] Schreyer, M., Gierbl, A.S., Ruud, F. and Borth, D., 2022. Sample selection by applying artificial intelligence-learning representative samples from journal entries in audit practice. Expert Focus, (02), pp.10-18.

[2] Schreyer, M., Baumgartner, M., Ruud, F. and Borth, D., 2022. Artificial Intelligence in Internal Audit as a Contribution to Effective Governance Deep Learning Based Detection of Accounting Anomalies in Audit Practice. Expert Focus, (01), pp.39-44.

[3] Ramamurthy, R., Pielka, M., Stenzel, R., Bauckhage, C., Sifa, R., Khameneh, T.D., Warning, U., Kliem, B. and Loitz, R., 2021, August. Alibert: Improved Automated List Inspection (ALI) with BERT. In Proceedings of the 21st ACM Symposium on Document Engineering (pp. 1-4).

[4] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E. and Brynjolfsson, E., 2021. On the Opportunities and Risks of Foundation Models. arXiv preprint arXiv:2108.07258.

[5] https://openai.com/research/gpt-4

[6] https://blog.google/technology/ai/lamda/

[7] https://chat.openai.com

[8] https://bard.google.com

[9] See International Auditing and Assurance Standards Board. 2009. International Standard on Auditing (ISA) 240: The Auditor’s Responsibilities Relating to Fraud in an Audit of Financial Statements. International Federation of Accountants (IFAC).

0 COMMENTS

Send