Development and evaluation of a computer-based learning pilot project for the teaching of holistic accounting concepts

Stephen Wedd and Margaret McKerchar
Orange Agricultural College, The University of Sydney

This paper reports on the development and evaluation of a computer-based self-instructional program for teaching holistic accounting concepts. It is the aim of the program to focus on the pedagogical use of CBL (computer-based learning) to improve conceptual understanding of accounting principles and to develop higher order thinking skills such as synthesis of information, evaluation and inferential judgements. The project was conducted in two phases, involving firstly the development of a CBL pilot program to present and simulate the linkages and impact of individual business decisions or transactions on the accounting system; and secondly an evaluation of the program to prove that the instructional strategies had led to a better conceptual understanding of accounting principles and greater student motivation. Evaluation results showed that students of the CBL group had a more favourable attitude to learning and their test scores were appreciably better.

Project background

This problem is further aggravated by specialised teaching of specific accounting content by individual accounting lecturers. While students may have learned some specific accounting principles very well in isolation, they are not able to see how different concepts are linked to each other and how changes in some parts of the accounting system may impact on the system as a whole.

It is the holistic understanding that we think is important to accounting teaching. Encel (1976) presented a similar view about accounting teaching when he made the criticism that the objectives of accounting education have concentrated too much on content and preparing students to practice, hence resulting in curricula which were too narrowly specialised and procedural. Similarly, Zeff and Dharan (1994) advocated that the goal of accounting education should be to produce accounting thinkers, rather than accounting technicians.

One typical example from our teaching is that while students may know very well technically how to enter various rural business transactions into appropriate accounting journals, they usually fail to predict and assess the impact of these decisions on the business health situation as can always be reflected in various accounting reports including the Profit and Loss Statements, Trading Accounts, and Balance Sheets.

It was our quest for teaching analytical thinking and conceptual understanding that gave birth to this project to focus on holistic understanding of the accounting system which is characterised by many intricate linkages and interconnectedness between components.

To achieve this teaching goal, traditional instructional methods such as lectures and print-based materials were considered inadequate to demonstrate and simulate the impact of business transactions on the accounting system and present linkages within it. As a result, a computer-based learning (CBL) approach was suggested to make use of the capabilities of the computer to show instant changes and simulate the impact of business transactions by way of clever interactive instructional design.

Computer technology, as commended by Beattie (1994), allows the presentation of three-dimensional and dynamic systems to the students, sometimes even putting the parameters of the system under students' control so they can manipulate the system and test hypotheses.

These capabilities of computer technology offered us the opportunities to show the dynamic changes in the accounting system caused by business transactions, to allow students to simulate impact and to predict consequences through the manipulation of data and scenarios, and finally to interpret and analyse the information for business decision making and inferential judgements.

Given the dynamic nature of the accounting system that we want to show, computer technology offers a teaching/learning system that is not otherwise possible by way of a two-dimensional representation on a blackboard or in a textbook. As a result, a computer-based self-instructional program was considered appropriate to provide supplementary tuition about the holistic accounting system through the use of computer simulations and case studies.

The development and evaluation process

The development phase:

The initial scripting of the program content was conducted by the subject expert. The important task here was to think carefully about what we were trying to teach and what the discrepancy was between the current state of our students' knowledge and our identified ideal. By keeping this focus we were able to avoid designing something that was inappropriate and to be clear about the capabilities of the technology.

Further development of the CBL occurred when the instructional designer reformatted the material into screens of information. The challenge here was to capitalise on the capabilities of the computer to present the accounting concepts in an interactive way using screen juxtapositions, timed display, animation, instant feedback and other techniques that were possible with the computer. In the design process, some flexible learning features offered by computer technology, such as learner control of study pathways and pace, were also incorporated.

The next stage was computer programming and screen design. After the instructional designer had reproduced the content in the form of screen layouts on paper, the programmer started to work from there using an authoring software. Throughout the programming process, the programmer needed at times to come back to the instructional designer to clarify and discuss ideas, and to adjust plans to accommodate the capabilities and constraints of the authoring program. Formative evaluation and feedback from the team on the prototype CBL program were frequent throughout the development process. This helped to identify problem areas, revise learning sequences and make adjustments in the early stages of development.

The CBL development has taken us much longer than we thought. It spanned a period of seven months with all team members working on the project on a voluntary basis while attending to other normal duties at work. Piemme (1988) roughly estimated that it may take a programmer 40 hours of work to produce one hour of content. However, the range can be from 50 to 500 hours (Criswell, 1989) for a project team to develop one hour of material for student use.

The development process resulted in a pilot program aiming to teach holistic accounting concepts by way of computer simulations and case studies. The program is in three parts.

Part One is an optional background module with a test in it to assess whether a student has the prerequisite knowledge to start with. If the test results show that the student needs to acquire some background knowledge about the accounting system, s/he is expected to work through the background module to learn some basic accounting concepts, such as 'the accounting equation', and the necessary terminology frequently used in the CBL program.

Part Two is a learning module about rural business transactions and how they are linked to various accounting reports. There are two components in Part Two. The first component is a demonstration by the computer in six scenarios about the linkages and impact caused by business transactions on the accounting system as reflected in financial reports such as Profit and Loss Statements, Trading Accounts and Balance Sheets.

The preceding and following screen shots were taken from one of the six scenarios which demonstrated the instant impact of a particular business transaction had on the accounting system. The impact was then explained with reference to the accounting equation, showing the relationship between equity, assets and liabilities.

The second component is the learner-controlled part in which students are given another six scenarios to predict the linkages and assess changes caused by the business transactions, and to work out the significance of these to the business health situation as interpreted from various accounting reports. The six scenarios presented aimed to test students?understanding of the linkages in the accounting system, reinforcing the concept of the accounting equation and the interconnected changes caused by business transactions.

Part Three is a learning module about transaction analysis in relation to the concepts of 'stability' and 'profitability'. Again, students are first presented with instructions and simulations about the analysis in case studies. Afterwards, it is their turn to work out the analysis themselves by measuring the changes caused by business transactions against the concepts of 'stability' and 'profitability' in both long- and short-terms.

The evaluation phase: The purpose of the evaluation process was to establish whether the identified learning problem was overcome as a result of the CBL program. Questions such as, 'Do they like it?' or 'Do they find it useful?', while important, are by themselves inadequate. The more fundamental evaluative questions should be: 'What is being learned by students?', 'How will they demonstrate this learning?' and 'Does the CBL program support students in achieving the learning objectives that we want them to achieve?'.

According to Beattie (1994), a good evaluation needs to establish whether a learning problem is being overcome, and to attempt to investigate deeply the specific learning experiences students are having as a result of the teaching intervention. In other words, in the context of our project, it was important to find out if the CBL program had helped to solve the learning problem that we were concerned about, that is, the lack of holistic understanding of accounting principles in our students.

The evaluation of the CBL program took several forms: peer review, student evaluation and pre- and post-tests.

An evaluation questionnaire adapted from the guidelines provided by Caffarella (1987) was designed to collect feedback from a small group of lecturers teaching rural accounting subjects at Orange Agricultural College. This peer review was done prior to the evaluation by students with the aim to obtain preliminary feedback about the teaching relevance and effectiveness of the CBL program. After the peer review, anything that was perceived as inappropriate, unclear or incorrect was eliminated and changed as considered necessary by the team. This resulted in a revised version of the CBL program ready for student evaluation.

For student evaluation, a questionnaire slightly modified from the one used in the peer review was administered to a group of rural accounting students who were given the CBL program to work through on their own. In the questionnaire, questions were directed towards user's satisfaction and perceived usefulness of the CBL program in helping them to learn the accounting principles presented.

Pre- and post-tests to measure what students had learned and hence the teaching effectiveness of the CBL program, were also conducted. The same tests were given to two groups of rural accounting students whose academic abilities and gender were matched to ensure that any discrepancies in the test results would be mainly attributable to the CBL program. One group was used as the control group who did the same pre- and post-tests without working through the CBL program.

The two tests, before and after, were almost identical, except that the questions in the post-test were randomised to avoid direct recall of answers from the pre-test. The two tests, each of twenty questions were set by the subject expert to test chiefly conceptual understanding of holistic accounting principles in relation to linkages and impact caused by business transactions on the accounting system as a whole. The tests were eventually marked by another accounting lecturer not involved in the project.

The purpose of the pre- and post-tests was to ascertain the impact of the CBL package on students' learning, in particular, their understanding of the holistic aspects of the accounting system. In other words, the research design involved mainly a comparison of CBL instruction versus no CBL instruction, and a measure of the difference. Though limiting it might seem, it was considered somewhat useful to shed light on the value of the CBL package for teaching interconnected accounting concepts.

Remember that the focus of this research project was not to revamp or change the entire instruction and curriculum in the current accounting subjects. Rather it was to develop a CBL package that aimed to supplement the current accounting teaching by emphasizing the holistic aspects of the accounting system.

Evaluation results

Student evaluation

Overall evaluation

Peer review

Pre- and post-tests results

Discussion of results

The responses of the two academics involved were equally positive. Although the number of peers involved was small, the review indicated an approval from them about the content and the appropriateness of the CBL program in teaching higher-order accounting concepts. It was encouraging to see that both lecturers were happy to use the program in their teaching and endorsed the program as an effective teaching tool.

The results from the pre- and post-tests were particularly pleasing. The difference between the pre- and post-test results within each of the two groups showed that all students (except one) who had worked through the program had improved their test results rather significantly. However, in the control group, the results were mixed. Three out of five students had showed some sign of improvement between the tests, but two students were going downhill in the post-test.

Conclusions and implications for further research

Though the pre- and post-tests are valid measures of the learning difference possibly caused by the CBL package, there is a need for further research. The purpose of further research is to investigate the effect of an overall course in rural accountancy and its ability to develop an holistic understanding versus the same course plus the CBL package that substitutes for the traditional teaching of the holistic accounting principles. In other words, the research design will involve a comparison of traditional teaching with CBL teaching of the same accountancy principles, and a measure of their difference in effectiveness.

Although the review by students and peers, and the test results indicated an overall positive evaluation of the CBL program, the CBL program was not without flaws. The project team was aware of the need to continuously improve its presentation and design, especially after the CBL program is adopted and integrated into the formal teaching of rural accounting subjects in future at Orange Agricultural College. Continuous development and evaluation of the CBL program is an integral part of the process to gauge its effectiveness in solving the learning problem identified.

Acknowledgements

References

Caffarella, E. P. (1987). Evaluating the new generation of computer-based instructional software. Educational Technology, 27(4), 19-24.

Criswell, E. L. (1989). The Design of Computer-Based Instruction. Macmillan Publishing Company, New York.

Encel, S. (1976). Education and politics. In S. D'ruso (ed.), Counterpoints: Critical Writings on Australian Education. Wiley International, Hong Kong.

Piemme, T. E. (1988). Computer-assisted learning and evaluation in medicine. JAMA, 260, 367-372.

Zeff, S. A. and Dharan, B. G. (1994). Readings and Notes on Financial Accounting: Issues and Controversies. McGraw-Hill, Inc., New York.