To Author's Page
The use of artificial intelligence (AI) in auditing is increasingly providing valuable support for auditors in a wide range of audit activities. For example, AI models are already being used in the context of audits of annual financial statements for audit sampling[1], journal entry testing[2], or the audit of notes disclosures.[3]
Figure 1: Overview of co-piloted auditing in the context of the audit process
The idea of co-piloted auditing describes a paradigm shift in the auditing profession, in which auditors and AIs work together on the basis of so-called foundation models[4], to bring their complementary skills to the auditing process:
At the same time, foundation AI Models, such as OpenAI’s GPT-4[5] or Google’s LaMDA[6] , can be adapted for a wide range of diverse tasks.
Generally, these models undergoe a two-stage training process. In the initial pre-training phase, the models are trained using large amounts of data to learn general patterns and structures. In the subsequent fine-tuning phase, the models are adapted using a small amount of domain specific data to learn how to solve a specific task. This two-step training process enables the fine-tuning of a pre-trained Foundation AI Model for a variety of different tasks, e.g. programming software, analysing customer feedback, or assisting in learning processes.
The ChatGPT[7] application, released by OpenAI, and the Bard[8] application, introduced by Google enable the fine-tuning of the underlying Foundation AI Model GPT-4 or LaMDA using so-called prompts. These prompts denote textual inputs that enable the model to learn from auditors through natural language. A series of such text entries, known as prompt protocols, can be used to fine-tune Foundation AI models for various audit tasks.
An audit prompt protocol divides an audit task into successive learning steps. The auditor use the protocol to guide the fine-tuning of the Foundation AI model. In the audit practice, following a prompt protocol comprising the following five prompts has proven proven to be effective:
After the successful fine-tuning, the Foundation Model is able to independently perform the learned testing task. Figure 2 shows the result of a fine-tuned GPT-4 model to perform Journal Entry Testing[9]. The Foundation Model acquired the skill to detect unusual entries in a financial accounting data by means of a corresponding audit prompt protocol.
Figure 2: Example result of a journal entry test according to ISA 240. Auditor’s prompt (top) and Foundation Model’s output (bottom)
The fine-tuning of Foundation AI Models is increasingly opening up new opportunities for auditors. Currently, the application still extends to supposedly basic tasks such as materiality assessment, transaction auditing and notes analysis. In the future, the co-piloted auditing paradigm will increasingly transform the audit profession through the application of artificial intelligence.
The Certified Audit Data Scientist certificate embraces this technological shift and teaches modern AI-supported audit procedures.
————————————————————————————————————————————————————————————————————————————–
This blog post is an excerpt of the following paper published by the author together with co-authors: “Artificial Intelligence Co-Piloted Auditing”, Gu, H., Schreyer, M. and Moffitt, K. and Vasarhelyi, M. A. (available online).
Marco Schreyer studied computer science and business adminstration with multiple years of experience in forensic data analysis in auditing. Currently, he’s researching how deep learning methods can be used in both internal and external audits at the University of St.Gallen and the Rutgers University. Additionally, Marco Schreyer teaches in the Certified Audit Data Scientist and Certified Fraud Manager certification courses at the Frankfurt School.
————————————————————————————————————————————————————————————————————————————–
[1] Schreyer, M., Gierbl, A.S., Ruud, F. and Borth, D., 2022. Sample selection by applying artificial intelligence-learning representative samples from journal entries in audit practice. Expert Focus, (02), pp.10-18.
[2] Schreyer, M., Baumgartner, M., Ruud, F. and Borth, D., 2022. Artificial Intelligence in Internal Audit as a Contribution to Effective Governance Deep Learning Based Detection of Accounting Anomalies in Audit Practice. Expert Focus, (01), pp.39-44.
[3] Ramamurthy, R., Pielka, M., Stenzel, R., Bauckhage, C., Sifa, R., Khameneh, T.D., Warning, U., Kliem, B. and Loitz, R., 2021, August. Alibert: Improved Automated List Inspection (ALI) with BERT. In Proceedings of the 21st ACM Symposium on Document Engineering (pp. 1-4).
[4] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E. and Brynjolfsson, E., 2021. On the Opportunities and Risks of Foundation Models. arXiv preprint arXiv:2108.07258.
[5] https://openai.com/research/gpt-4
[6] https://blog.google/technology/ai/lamda/
[7] https://chat.openai.com
[8] https://bard.google.com
[9] See International Auditing and Assurance Standards Board. 2009. International Standard on Auditing (ISA) 240: The Auditor’s Responsibilities Relating to Fraud in an Audit of Financial Statements. International Federation of Accountants (IFAC).
Frankfurt School gGmbH
Adickesallee 32-34
60322 Frankfurt am Main
