ICIME 2013 Proceedings of The 4th International Conference on IS Management and Evaluation by ACPIL

Proceedings of the 4th International I t ti l Conference C f on Information Systems Management and d Evaluation E l ti RMIT University Vietnam Ho Chi Minh City Vietnam 13-14 May 2013

Edited by Dr Blooma John , Dr Mathews Nkhoma and Dr Nelson Leung RMIT University, Ho Chi Minh City, Vietnam

A conference managed by ACPI, Reading, England

Proceedings of The 4th International Conference on Information Systems Management and Evaluation ICIME 2013 RMIT University Vietnam Ho Chi Minh City Vietnam 13-14 May 2013 Edited by Dr John Blooma, Dr Mathews Nkhoma and Dr Nelson Leung RMIT University Ho Chi Minh City Vietnam

Copyright The Authors, 2013. All Rights Reserved. No reproduction, copy or transmission may be made without written permission from the individual authors. Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper abstracts were read and selected by the conference panel for submission as possible papers for the conference. Many thanks to the reviewers who helped ensure the quality of the full papers. These Conference Proceedings have been submitted to Thomson ISI for indexing. Please note that the process of indexing can take up to a year to complete. Further copies of this book and previous yearâ&#x20AC;&#x2122;s proceedings can be purchased from http://academic-bookshop.com E-Book ISBN: 978-1-909507-20-3 E-Book ISSN: 2048-9854 Book version ISBN: 978-1-909507-18-0 Book Version ISSN: 2048-9846 The Electronic version of the Proceedings is available to download at ISSUU.com. You will need to sign up to become an ISSUU user (no cost involved) and follow the link to http://issuu/acpil Published by Academic Conferences and Publishing International Limited Reading UK 44-118-972-4148 www.academic-publishing.org

Contents Paper Title

Author(s)

Page No.

Preface

Committee

Biographies

vii

Risk Management for Digital Cultural Heritage Information in Malaysia

Zuraidah Abd Manaf, Aliza Ismail, Noor Masliana Razlan, Azizah Abd Manaf and Rusni Daruis

Investigating the Quality and Effectiveness of the Picture Archiving Communication System (PACS) at an Academic Hospital in South Africa

Bronwyn Abrahams and Michael Kyobe

Assessment of Knowledge Sharing Readiness: An Institute of Higher Learning Case Study

Nur Syufiza Ahmad Shukor, Haslinda Sutan Ahmad Nawi, Suzana Basaruddin and Nasrudin Md Rahim

Ishikawa Diagram of Critical Factors for Information Technology Investment Success: A Conceptual Model

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad

Persona Ontology for User Centred Design Professionals

Farshid Anvari and Hien Minh Thi Tran

Evaluating Strategic Information Systems Planning (Sisp) Performance Among Malaysian Government Agencies Using Organizational Learning-Based Model

Fazidah Abu Bakar, Mohd Adam Shaimi and Husnayati Hussin

Accessibility and Human Computing Interaction: Engaging and Motivating Students With Disabilities Through More Computer Empowerment

Bob Barrett

Achieving Improved Usability Outcomes With Different Usability Practitioner Roles

Vince Bruno and Martin Dick

Information Availability as Driver of Information Security Investments: A Systematic Review Approach

Duy Dang and Mathews Nkhoma

A Revised User Interface Design for the Discovery Search Tool “Summon”: A Case Study and Recommendations

Jamal El-Den, Lien Bui and Wendy Nguyen

The Study of the Relationship Between Individual Factors and Usage of ICT in Tehran’s Public Libraries

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati

Identification, Evaluation and Ranking of the key Factors to Contribute to the Implementation of Electronic Municipality

Marzieh Faridi Masouleh and Hadi Mohammadzadeh

E-Procurement in the Era of Cloud Computing

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire

104

A Comparative Study on Waseda e-Government Indicators Between Vietnam and Japan

Nguyen Manh Hien

111

Paper Title

Author(s)

The CIO in the Public Sector: The Role and Requisite Knowledge and Competencies

Val Hooper and Beverley Bunker

119

Trusting Your Bank in a Digitally Connected World – a New Zealand Investigation Into Customers Perceptions of Ethical Business Practice

Tony Hooper and Shivonne Londt

127

Changes in use, Perception and Attitude of First Year Students Towards Facebook and Twitter

Kevin Johnston, Mei-Miao Chen and Magnus Hauman

135

Analysis of Dissatisfiers That Inhibit Cloud Computing Adoption Across Multiple Customer Segments

Easwar Krishna Iyer, Arathi Krishnan, Gaurav Sareen and Tapan Panda

145

Explaining the Adoption of Complex Innovations by Reflexive Agents — A Critical Realist Perspective

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson

152

Adoption of Mobile Technology to Enhance Services at Academic Library

Eunice Mtshali and Tiko Iyamu

160

A Theoretical Review of the Impact of Test Automation on Test Effectiveness

Donovan Lindsay Mulder and Grafton Whyte

168

Contributing Factors of Cloud Computing Adoption: a Technology-OrganisationEnvironment Framework Approach

Mathews. Nkhoma, Duy Dang, Anthony De Souza-Daw

180

IT and Business Value–a Missing gap: A SME Case Study

Hiep Pham and Kevin Nguyen

189

Usability Testing for an Evaluation of ApplicationLearning Tutorials and Document-Based Instructor

Nantika Prinyapol

198

Using PRS to Enhance Student Learning Process: The Application of Revised Study Process Questionnaire-2 Factors

Ondris Pui, Narumon Sriratanaviriyakul, Wei Wei , Patrick Then and Zia-Yee Tho

206

Learning From a Failed ERP implementation: The Case of a Large South African Organization

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul

215

Using YouTube to Support Student Engagement for the Net Generation in Higher Education

Sumarie Roodt

223

Knowledge Creation in Students’ Software Development Teams

Mzwandile Shongwe

232

Defining Social Technologies

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė

239

Community ICT Hubs Sustainability: Result From a Field Study of ICT4D Project

Haslinda Sutan Ahmad Nawi, Nur Syufiza Ahmad Shukor, Suzana Basaruddin, Siti Fatimah Omar and Azizah Abdul Rahman

247

Sources of Disturbances Experienced During a Scrum Sprint

Maureen Tanner and Angela Mackinnon

255

How Reflective Professionals Design and Evaluate Financial Information Management Systems Courses

Hien Minh Thi Tran and Farshid Anvari

263

Implementation of a Corporate Performance Management System in a Large City

Mark Tudor and Mike Hart

272

Page No.

Paper Title

Author(s)

Digital Archiving and e-Discovery: Delivering Evidence in an age of Overload

Geert-Jan van Bussel and Hans Henseler

281

Customer Centricity: A Comparison of Organisational Systems With Social Media Applications

Grafton Whyte and Adolf Lamprecht

289

Using the Inkwenkwesi Model to Identify the Knowledge Requirements of a new Project Entrant

Dr Grafton Whyte and Desire Minnaar

300

An Investigation into the Effectiveness of Storytelling as Means of Sharing Tacit Knowledge

Grafton Whyte and Monica Ralake

309

The Role of IS in Residential Energy Management

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper

318

Does the Need to Belong Drive Risky Online Behavior?

Antony Zogg and Tony Hooper

328

PHD Papers

Page No.

335

Challenge in Sharing Tacit Knowledge: Academicians’ Behavior Towards Developing a web Portal for Sharing Research Ideas

Hafiza Adenan, Dang Merduwati Hashim, Adnan Jamaludin and Haziah Sa’ari

337

Acceptance of new Technologies in HR: ERecruitment in Organizations

Nicoleta Valentina Florea and Mihaela Badea

344

Identifying Entrepreneurial Competencies Which Lead to Innovative Performance in Malaysian Academic Libraries

Haziah Sa’ari, Rusnah Johare, Jafalizan Md. Jali, Asmad Rizal Umar and Hafiza Adenan

353

Information Sharing Problems and Construction Efficiency

Huan Vo-Tran, Brian Corbitt, Sittimont Kanjanabootra and Peter Macauley

361

Work in Progress

371

Modeling Practices of Competitive Intelligence in Moroccan Firms

Fatima-zzahra Cheffah and Mostafa Hanoune

373

Towards an e-Health Strategy for Vietnam - Issues and Approaches

Anna Shillabeer

377

Proposed Mobile Technology Acceptance Model of the Information Services in a Library Context

Sununthar Vonjaturapat and Singha Chaveesuk

385

iii

Preface

The Fourth International Conference on Information Systems Management and Evaluation is being hosted this year by RMIT University in Ho Chi Minh, Vietnam. ICIME provides a platform for individuals researching and working in the broad field of information systems management, including information technology evaluation to come together and exchange research ideas. We hope that this yearâ&#x20AC;&#x2122;s conference will provide you with plenty of opportunities to share your expertise with colleagues from around the world. This yearâ&#x20AC;&#x2122;s opening keynote address will be delivered by Ivy Wong, Next Mobile Limited, Hong Kong, China. And the second day will be opened with an address from Professor Grandon Gill from the University of South Florida, USA. ICIME 2013 received an initial submission of 116 abstracts. Of this 34 academic papers, 4 PhD papers and 3 work-inprogress papers have been accepted for these Conference Proceedings as a result of the double blind, peer review process of full paper submissions. These papers come from all parts of the world including Australia, India, Indonesia, Iran, Japan, Lithuania, Malaysia, Morocco, Namibia, Netherlands, New Zealand, Portugal, Romania, South Africa, Sweden, Thailand, USA and Vietnam. So ICIME is set once again to be a truly international conference! We wish you a most interesting and enjoyable time in Vietnam. Dr John Blooma Dr Mathews Nkhoma Dr Nelson Leung RMIT University, Ho Chi Minh City, Vietnam May 2013

Conference Executive Dr Nelson K.Y. Leung, RMIT International University Vietnam Dr Blooma John, RMIT International University Vietnam Dr Mathews Nkhoma, RMIT International University Vietnam

Mini Track Chairs

Prof. Dr. Abdel-Badeeh M. Salem, Ain Shams University, Cairo, Egypt. Dr. Farid Shirazi, The Ted Rogers school of Information Technology Management, Ryerson University, Toronto, Canada Tony Hooper, Victoria University of Wellington, Wellington, New Zealand Huan Vo-Tran, Business IT & Logistics, RMIT, Melbourne, Australia Dr Vince Bruno, Business IT & Logistics, RMIT, Melbourne, Australia

Conference Committee The conference programme committee consists of key individuals from countries around the world working and researching in the Information Management and Evaluation community. The following have confirmed their participation: Prof Nor Shahirza Abdul Karim (Universiti Teknologi Malaysia, Malaysia); Dr Mo'taz Amin Al Sa'eed (Al - Balqa' Applied University, Amman, Jordan); Prof. Hussein Al-Yaseen (Al-Ahliyya Amman University, Jordan, Jordan); Hasnain Baloch (King Abdulaziz University, Saudi Arabia); Dr Yongmei Bentley (University of Bedfordshire, UK); Felix Bollou (American University of Nigeria, Yola, Nigeria); Dr Samarjeet Borah (Sikkim Manipal Institute of Technology (SMU), India); Dr Irwin Brown (University of Cape Town, South Africa); Prof. Andy Bytheway (Cape Peninsula University of Technology, Cape Town, South Africa); Professor Sven Carlsson (School of Economics and Management, Lund University, Sweden); Dr Anil Cekic (Atilim University, Turkey); Prof Prasenjit Chatterjee (MCKV Institute of Engineering, India); Dr Phaik Kin Cheah (Universiti Tunku Abdul Rahman, Malaysia); France Cheong (RMIT University, Melbourne, Australia); Prasenjit Chetterjee (MCKV Institute of Engineering, India); Prof Jason Cohen (University of Witwatersrand, South Africa); Dr Barbara Crump (Massey University, New Zealand); Guillermo de Haro (Instituto de Empresa, Madrid , Spain); Prof Carina de Villiers (University of Pretoria, South Africa); Dr. Nomusa Dlodlo (CSIR, South Africa); Dr Ciprian Dobre (University Politehnica of Bucharest, Romania); Dr Babek Erdebilli (Atilim University, Turkey); Dr Erman Erkan (Atilim University, Ankara, Turkey); Prof Alptekin Erkollar (Halic University, Turkey); Samuel Fossa Wamba (School of Information Systems and Technology, Wollongong, Australia); Ken Grant (Ryerson University, Toronto, Canada); Dr Poyraz Gurson (Atilim University, Turkey); Dr. Michael Gurstein (Community Informatics Research, Vancouver, Canada); Dr Vered Holzmann (Tel-Aviv University / Holon Institute of Technology, Israel); Tony Hooper (Victoria University of Wellington, New Zealand); Val Hooper (Victoria University of Wellington, New Zealand); Dr Hon-Tat Huam (Quest International University Perak, Malaysia); Assoc Prof Dr Zuraini Ismail (University Technology Malaysia, Malaysia); Prof. Tiko Iyamu (University of Technology, Pretoria, South Africa); Dr Blooma John (RMIT International University Vietnam, Vietnam); Dr Kevin Johnston (University of Cape Town, South Africa); Dr. Erne Kassim (Universiti Teknologi MARA, Malaysia); Dr Caroline Khene (Rhodes University, South Africa); Professor Ranjan Kini (Indiana University Northwest, Gary, USA); Rembrandt Klopper (University of KwaZulu-Natal, South Africa); Prof. Jesuk Ko (Gwangju University, Korea); Prof. Rakesh Kumar (Haryana College of Technology & Management, Kaithal, India); Dr Mike Kyobe (University of Cape Town, South Africa); Miguel Leitao (ISEP, Portugal); Dr Nelson K.Y Leung (RMIT International University Vietnam, Vietnam); Prof Sam Lubbe (University of South Africa, South Africa); Prof Dalgobind Mahto (Green Hills Engineering College, India); Milos Maryska (University of Economics in Prague, Czech Republic); Dr Annette Mills (University of Canterbury, Christchurch, New Zealand); Peter Mkhize (UNISA, South Africa); Muhammad Izwan Mohd Badrillah (UITM, Malaysia); Shukor Sanim Mohd Fauzi (Universiti Teknologi Mara, Arau, Malaysia); Muhammad Naveed (IQRA University Peshawar, Pakistan); James Ngeru (Morgan State University, Baltimore, USA); Collins Ngwakwe (University of Limpopo South Africa -School of Accountancy., South Africa); Dr Emanuela-Alisa Nica ( Petre Andrei University from Iasi, Romania); Emil Numminen (Blekinge Institute of Technology, Karlskrona, Sweden); Ass. Prof. Birgit Oberer (Kadir Has Universitry, Turkey); Dr Tiago Oliveira (Universidade Nova de Lisboa, Portugal); Dr Abdelnaser Omran (School of Housing, Building and Planning, Universiti Sains Malaysia, Malaysia); Prof Patricia Ordóñez de Pablos (The University of Oviedo, Spain); Dr Roslina Othman (International Islamic University Malaysia, Kuala Lumpur, Malaysia); Dr Shaun Pather (e-Innovation Academy, Cape Peninsula University of Technology, Cape Town, South Africa); Zakieh Piri (Tabriz University of medical sciences, Iran); Shana Ponelis (University of Wisconsin-Milwaukee, USA); Mirja Pulkkinen (University of Jyväskylä, Finland); Dr Bahlol Rahimi (Urmia University of Medical Sciences, Iran); Prof Thurasamy Ramayah (Universiti Sains Malaysia, Malaysia); Dr. Surya Rathore (G. B. Pant University of Agriculture & Technology, Pantnagar, India); Dr Dimitris Rigas (De Montfort University, UK); Martyn Roberts (University of Portsmouth, UK); Dr Siti Rohimi (International Islamic University Malaysia (IIUM), Malaysia); Professor Narcyz Roztocki (State University of New York at New Paltz, USA); Dr Ilias Said (Universiti Sains Malaysia, Malaysia); Dr Jatinderkumar Saini (Sankalchand Patel College of Engineering, India); Prof. Abdel-Badeeh Salem (Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt);Prof Chaudhary Imran Sarwar (Creative Researcher, Lahore, Pakistan); Prof.Dr Joachim Schuler (University Pforzheim, Germany); Ejse Scott (University of Cape Town, South Africa); Zarehan Selamat (Multimedia University, Malaysia); Dr Kosheek Sewchurran (University of Cape Town, South Africa); Yousef Shahwan (Canadian University of Dubai, , United Arab v

Emirates); Dr. Ashok sharma (University of rajasthan ,, India); Farid Shirazi (Ryerson University, Canada); Shawren Singh (University of South Africa, South Africa); Dr Gregory Skulmoski (Cleveland Clinic Abu Dhabi , United Arab Emirates); Dr. Darijus Strasunskas (Norwegian University of Science and Technology, Trondheim, Norway); Christine Nya-Ling Tan (Multimedia University, Melaka, Malaysia); Nata van der Merwe (Cape Peninsula University of Technology, South Africa); Prof/Dr Gert van der Pijl (Erasmus School of Economics, The Netherlands); Huan Vo-Tran (RMIT University, Melbourne, Australia); Minhong Wang (The University of Hong Kong, Hong Kong); Dr Grafton Whyte (UWC, South Africa); Prof Mohammad Hossein Yarmohammadian (Health Management and Economic Research Center, Isfahan University of Medical Sciences, Iran); Nor'Aini Yusof (Universiti Sains Malaysia, Malaysia); Dr Savas Zafer Sahin (Atilim University, Turkey); Dr Malgorzata Zieba (Gdansk University of Technology, Poland)

Biographies Conference Chair Dr Nelson K.Y.Leung is an Assistant Professor at the RMIT International University Vietnam. He is also serving as the Founding President of the Vietnam Chapter of Association for Information Systems (VAIS), an Adjunct Researcher at the Payap University, and an Associate Editor for Interdisciplinary Journal of Information, Knowledge and Management (IJIKM) and International Journal of Intercultural Information Management (IJIIM). Nelson holds a PhD in Information Systems from University of Wollongong. Nelson has published widely in refereed books, journals and international conferences. His research interests include ontology mediation and development methodology, organizational knowledge management, education technology and supply chain management.

Programme Co-Chairs Dr Mathews Nkhoma holds a Ph.D. in Information Security. His major topics of research are: Zero day attacks and Vulnerability research, Information Systems Security, Transparency in information security system design, Network security investment model, Ethical hacking and Network defence, Network security management, Forensic Computing and evidence recovery including mobile devices, Cybercrime, Identity theft, consumer protection, trust and confidence, Impact of ICT in education. His research has been published in journals such as the International Journal of Information Management and Computer Security, Emerald, International journal of Information Security and Privacy- IGI Publishing. Mathews has international experience in teaching: leading Information systems and computer forensics courses in Africa, Europe, Middle East and Asia at both undergraduate and post graduate level. Blooma Mohan John is a Lecturer at RMIT International University, Vietnam. She completed her PhD from Nanyang Technological University (NTU), Singapore in 2011. Her research interest is in the area of Search Engines, Question Answering systems and Social Media. She completed her Master of Science in Information Studies from NTU, Singapore in 2004. Blooma obtained her Bachelor of Engineering Degree in Electronics and Communication Engineering from Bharathiyar University, India in 2001. She taught various IT related course at the Middle East College of Information Technology (Sultanate of Oman) and the Government Engineering College (India), NTU (Singapore) and RMIT International University (Vietnam). She was awarded the NTU PhD Research Scholarship for four years and was involved in working as a Teaching Assistant from 2007 - 2010. Blooma has published in various International Conferences and refereed Journals. Blooma is serving as a reviewer in various refereed Journals and as the Treasurer at Vietnam Association of Information Systems.

Keynote Speakers Grandon Gill teaches introductory and intermediate courses in programming for undergraduates and also teaches capstone courses in the MIS undergraduate, MS-MIS and Executive MBA programs. He has also taught a variety of IT courses during his tenure at USF, from computer systems concepts to doctoral case methods. He received USF’s Excellence in Undergraduate Teaching Award in 2007. Gill is passionate about using technology as a teaching tool and has studied distance learning, strategy, and practice, alternative course designs, and tools for course development and delivery, all under the general heading of informing science. His research in this area has been published in many journals, including Informing Science, Decision Sciences Journal of Innovative Education, the Journal of Information Systems Education, eLearn, and the Journal of IT Education. He has also published multiple times in MIS Quarterly, the MIS discipline’s leading journal—his most recent article considering the MIS fields from an informing science perspective. Ivy Wong is currently the Chief Executive Officer of Next Mobile Limited. Prior to joining the Group in September 2011, she has over 10 years of experience in online media business and worked as Chief Operating Officer in TVB.com Limited and Senior Director in Yahoo!(Hong Kong and Asia). Ms. Wong has a Bachelor of Commerce degree from University of Toronto, Canada majoring in International Economics and Marketing.

vii

Biographies of Presenting Authors Hafiza Adenan is a PhD student at Faculty Information Management of MARA Technology University, Malaysia. She has working experience in retailing industry and educational industry. In educational industry, she has experience as a lecturer at private university college in Malaysia. Nur Syufiza Ahmad Shukor is a senior lecturer in Department of Information Systems, UNISEL and currently pursuing PhD at UTM, Malaysia. Implemented many ICT projects specifically software development projects during previous work in industry. Her current research interests focus on the area of Information Systems and Knowledge Management. She is also a member of AIS and iKMS. Dr. Fazidah Abu Bakar earned her PhD. degree in Information Technology (IT) from International Islamic University Malaysia on Strategic Information System Planning. She works as an IT government officer and has been involved in various IT implementations in public sector. Currently, she is involved in the Malaysian National Single Window for Trade Facilitation project. Farshid Anvari BE, GradDipSc, GradDipIT, MACS has over 15 years of experience in software engineering and 12 years of experience in Power Engineering, specialising in hydro system modelling and reliability studies. His research interests include cognitive load, user-centered design, persona ontology and architecting systems that are economical, robust and reliable. Dr. Bob Barrett is a professor for the School of Business at the American Public University in Charles Town, West Virginia, USA. He lectures both nationally and internationally on the topics of Intellectual Capital, Knowledge Management, and Disability in the Workplace, e-Portfolios, and e-Learning. Dr Vince Bruno is a Lecturer at RMIT University, College of Business, school of Business IT and Logistics. He holds a Bachelor of Computer Science and a Master in computing (IT), and a Ph.D. His research interests include usability, IT education, databases technologies, application development and solving problem. Fatima-zzahra Cheffah is a PhD student at the Doctoral Education: Mathematics, Computer Science and Information Technologies, Information Technology and Modelling Laboratory, Faculty of Sciences Ben M’S.ik, Hassan II University - Mohammedia Casablanca, Morocco. Mr. Duy Dang is doing his research degree – Bachelor of Business (Honours) – at RMIT Melbourne. His research interests focus on information security management and other relevant topics. Marzieh Faridi Masouleh. I’m come from Rasht. I am 30 years old. I graduated from Islamic Azad University. In bachelor my field of study was software engineering and in Master it was Information Technology Management. Since, September 2012 I am Ph.D. Candidate in Department of Information Technology Management, Science and Research Branch Islamic Azad University, Tehran, Iran and I teach in university. Nicoleta Valentina Florea PhD Lecturer,, Management-Marketing Department, Valahia University of Targoviste, Romania, with experience in HRM. Master in Marketing (2004) and PhD thesis in Management (HR) (2011) and published 30 articles in HRM, in National and International Conferences and books on HR Management. Member of Research Center of VUT. Received doctoral scholarship, Paris Est, Creteil, Val-de-Marne, France, April-July, 2010. Antonio Grilo holds a PhD degree in Industrial Management by the University of Salford, UK. He is Assistant Professor of Industrial Engineering and Management at the Faculdade Ciências e Tecnologia da Universidade Nova de Lisboa, in doctoral, master and undergraduate degrees. He is also a member of the board of director of the research centre UNIDEMI. He has over 70 papers published in international scientific journals, and conferences. Mike Hart is Emeritus Professor of Information Systems at the University of Cape Town, South Africa. The main focus of his research and teaching is business intelligence. He is rated as an established researcher by the National Research Foundation. Hans Henseler is a computer scientist and holds a Ph.D. on Artificial Neural Networks. He is Associate Professor (Lector) EDiscovery at the HvA / Amsterdam University of Applied Sciences. He is director of an international consultancy on EDiscovery. Nguyen Manh Hien is PhD candidate at GSAPS, Waseda University, Japan. He is also a research assistant at Waseda Institute of e-Government since 2011. He earned an engineer degree in IT from Vietnam in 2000. He received his Master degree in Chief Information Technology (CIO) from GITS, Waseda University, Japan 2011. viii

Tony Hooper is a senior lecturer in the School of Information Management at Victoria University of Wellington and Programme Director of the Master of Information Management Programme. This research paper follows on from previous research into social networks presented to the 2012 ICIME Conference in Ankara, Turkey. Val Hooper is the Head of the School of Information Management of Victoria University of Wellington, New Zealand. Val holds a PhD in Information Systems and an MBA. She has a background in information systems, management and marketing. Her current research focuses on strategic alignment of IS, the IT productivity of SMEs, and IT-related behaviour. Kevin Johnston is an Associate Professor in the Department of Information Systems at the University of Cape Town. He worked for 24 years for companies such as De Beers, Liberty Life, Legal & General and BoE. Kevinâ&#x20AC;&#x2122;s main areas of research are ICT Management, Project Management, IS educational issues and Open Source Software. Easwar Krishna Iyer is Associate Professor in Marketing and Director-Admissions of Great Lakes Institute of Management. Easwar is Currently pursuing PhD [Cloud Computing Adoption] and a second MBA in Energy Management [University of Houston, Texas]. B Tech from College of Engineering, Trivandrum. M. Tech MOCE from IIT Kharagpur. PGPM from Great Lakes Institute of Management â&#x20AC;&#x201C; Gold Medalist. Michael Kyobe is A/Professor of Information System. He holds a PhD in Computer Information Systems and an MBA. Michael worked as a project manager and IT manager for several years and has consulted extensively with the public, and SMEs. His research interests include business-IT alignment, governance, computer security, ethics, knowledge management and SMEs. Marius Mihailescu is a lecturer and PhD candidate in the Department of Informatics at School of Economics and Management, Lund University in Sweden. His main research interests relate to design science research in IS. Marius has published in Electronic Journal of Information Systems and peer-reviewed conference papers such as ICIS, ECIS, AMICS, ICIME and ECITE. Hadi Mohammadzadeh is a Ph.D. candidate in the Institute of Applied Information Processing at the University of Ulm, Germany; His major research interests are in exploring issues in information extraction, information retrieval and Text/Web mining. He obtained his M.Sc. in Applied Mathematics from Sharif University of Technology in Tehran in 1996. Prior to that, he obtained his B.Sc. in Software Engineering from Ferdowsi University of Mashhad in 1993. From 1997 to 2007, he was a lecturer at the Universities of Tehran, K.N.Toosi University of Technology in Tehran, and Azad University in Dubai. Donovan Lindsay Mulder has 15 years of software engineering experience and has co-authored three software testing peer reviewed papers. Professionally, Donovan specializes in test process improvement and test automation which are the Donovan's key research areas. Donovan is thus a practitioner and an academic in the field of software testing. Donovan is working towards a PHD. Maryam Nakhoda is a PhD in Library and Information Science (LIS) and assistant professor at University of Tehran, faculty of Library and Information Science. She is the author of papers in Persian and English. Her research interests include Information and Communication Technology (ICT) usage in libraries, library management, and managing change in academic libraries. Kevin Nguyen is a lecturer in the Business Information Systems department at RMIT University Vietnam where he teaches computer networking, e-commerce and business communications. Kevin has mater degree in Computer Science and more than 13 years of experience an IT manager and corporate trainer in retail and logistics industries in the USA. Hiep Pham is a Senior Lecturer at RMIT Vietnam University. He has completed a Master degree in Advanced Information Systems at University of New South Wales and a MBA (Executive) at RMIT Vietnam University. He is actively conducting research in Information Management and Information Security Management. Dr. Nantika Prinyapol (Ph.D., University of Wollongong, Australia) is a director of Master program in Information Technology for Modern Enterprise, Faculty of Information Technology, Dhurakij Pundit University, Thailand. Her research is in workflow practices and web services development for business processes. She is interested in usability testing, human-computer interaction, multimedia design, and business process management. Ondris Pui is a lecturer at the school of Communication and Design, RMIT, University Vietnam. He has 10 years of experience in the creative industry before moving on to academics. His holds a Master in Multimedia (Design) from the University of Sydney. His research interest lies in technology in education and physical computing.

Anjali Ramburn is currently completing a PhD in Information Systems at the University of Cape Town. She is also teaching a number of undergraduate and postgraduate courses in the department. Anjali has published in various international conferences and journals. Her current research interests include mobile data services, knowledge management and ERP systems. Sumarie Roodt has a passion for developing teaching and learning toolkits that leverage emerging technologies in order to improve the learning experiences of the net generation and more recent generations. Her research interests are social computing, web 2.0 and web 3.0 for education and digital game-based learning. Haziah Sa’ari is a full time student under Faculty of Information Management, Universiti Teknologi MARA (UiTM) Malaysia. She has working experience about 14 years in Library and Information Management field with multiple companies such as broadcasting and architectural organisation. Dr. Anna Shillabeer. Qualification(s):B.A., Dip. Prog. Tech., B.I.T.(Hons.1st Class), PhD in medical informatics from the Flinders University of South Australia. Her research focuses on personalised medicine solutions and in particular the unique analytical and clinical needs of the medical domain with a focus on knowledge discovery software including data mining technologies. Anna’s academic experience spans 13 years in 4 Universities teaching at both undergraduate and Masters level and covers a broad range of technology and policy topics. Mzwandile Muzi Shongwe is a lecturer in the department of Information Studies, University of Zululand, South Africa. He is a PhD candidate in the department of Information Studies, University of KwaZulu- Natal, South Africa. He teaches undergraduate courses and supervises postgraduate research projects. His research interests are knowledge management, knowledge management systems and mobile technologies. Prof. dr. A. Skaržauskienė has degree of Doctor of Social Sciences from ISM, University of Management and Economics in Vilnius, Lithuania. In her work dr. A. Skaržauskienė applies both knowledge of management and modern leadershipcorrelated disciplines such as Business dynamics, Systems thinking, Chaos and complexity theories. Dr. A. Skaržauskienė was the couch in the Self-managing teams building project in European Parliament together with DEMOS Group Belgium Narumon Sriratanaviriyakul is a Senior Lecturer at RMIT University Vietnam. She has been working in tertiary education for 7 years and has extensive experiences in teaching diverse and multinational groups of students. Prior to academic background, Narumon had 7 years of working experience from IT and Tourism & Hospitality industries. Haslinda Sutan Ahmad Nawi, doctoral candidate from Faculty of Computing, UTM. Academician from UNISEL and research focuses on Information Systems and corresponding IT projects sustainability. Currently holds a Master of Information Technology degree awarded by UiTM, Malaysia with over eight years relevant industrial experience in IT projects management, implementation and operation. Member of PMI and AIS. Rūta Tamošiūnaitė is a PhD candidate in Management and Administration (Social Sciences), lecturer and junior researcher at Mykolas Romeris Univeristy, Lithuania. Rūta Tamošiūnaitė is a member of Academic association of Management and Administration (AVADA) with responsibilities in external communication sphere. Her interest fields are social technologies, decision making process, and science communication. Maureen Tanner, MCom: Maureen is a PhD associate and Senior Lecturer in the Department of Information Systems, University of Cape Town, South Africa. Her research interests lie in Global Software Development, Agile Methods, and the application of Agile Methods in the distributed software development setting. Hien Minh Thi Tran BE(Hons), MEngSc, GCCA, MCom, GradDipMgt, MA, CPA has 25 years experience as a professional engineer, software developer and systems accountant. She has taught professional staff and university students in various topics in engineering, accounting and software developments. Her research interests include financial information management systems, user centred design, accounting and education. Munirul Ula has a Bachelor Degrees in Electrical Engineering majoring in Telecommunication System and Data Communication, master degree in Electrical engineering majoring in Computer Simulation. Currently, he is doing a PhD in Computer Science majoring in Banking Information Security Governance. He has more than 5 years experience as researcher and he also has one year teaching experience tutoring in faculty of Electrical Engineering Universiti Teknologi Malaysia. Geert-Jan van Bussel is a certified archivist and holds a Ph.D. in information and archival studies. He is Associate Professor (Lector) Digital Archiving & Compliance at the HvA / Amsterdam University of Applied Sciences and Assistant Professor at the University of Amsterdam. He is director of an international consultancy on strategic information and records management and records auditing. x

Huan Vo-Tran is a lecturer within the School of Business IT & Logistics at RMIT University. He is currently completing a PhD in Information Management. Prior to becoming an academic he worked in various fields, which included project management, systems analysis and high school teaching. His areas of interest include information management, knowledge management and Web 2.0. Ming Hock Yew is currently a PhD candidate in RMIT Universityâ&#x20AC;&#x2122;s School of Business IT and Logistics. His research interest is in the area of Green IS and has presented a paper on this in ACIS 2012. Ming Hock is also a Singapore Certified Energy Manager and a Certified IT Project Manager (Associate). Sununthar Vongjaturapat is a multimedia technology lecturer at Burapha University, Thailand. Now I am studying in Ph.D. program at King Mongkut's Institute of Technology Ladkrabang, Thailand. My current research relate to IT adoption. My work has been published in the KMITL Journal of Information Technology, Thailand. Antony Zogg, B.Com, MIM graduated from Victoria University of Wellington in December 2011 with a Master of Information Management degree with Distinction. He is interested in the relationships between human behaviour and socialization in the real world, and the widespread occurrence of risky online behaviour exhibited by individuals in the social media.

Risk Management for Digital Cultural Heritage Information in Malaysia Zuraidah Abd Manaf1, Aliza Ismail1, Noor Masliana Razlan1, Azizah Abd Manaf2 and Rusni Daruis2 1 Faculty of Information Mangement, Universiti Teknologi MARA, Shah Alam, Malaysia 2 Advanced Informatics School (AIS), Universiti Teknologi Malaysia, Kuala Lumpur, Malaysia zuraidaham@salam.uitm.edu.my alizai@salam.uitm.edu.my masliana0315@puncakalam.uitm.edu.my azizah07@ic.utm.my rusni@ic.utm.my Abstract: A crucial component of the nation’s K‐economy envisions the availability of digital content exchange or content infrastructure. The idea of providing and presenting information via electronic means and formats truly fascinates managers of information agencies worldwide. This phenomenon obviously affects the thinking and perception of the managers and administrators of libraries, museums, archives and galleries. Presently, cultural institutions are actively connecting their collections to these emerging knowledge networks. They are creating innovative network services based on the digital surrogates of their current collections in a rich, interactive digital environment. A recent study has been made on the digitisation of cultural heritage resources in cultural institutions in Malaysia, namely the National Library, National Archives, National Museum and the National Art Gallery. The four institutions are currently very active in building their digital content and exploring initiatives to deliver the information, however the extent of sustainability of those resources in unknown. The aim of this paper is investigates the digital preservation strategies that are adopted by Malaysian cultural institutions. A multiple case studies approach is used to examine the potential risks to digital cultural resources in four (4) selected Malaysian cultural institutions. Data is collected through structured interviews with information professionals who are involved in digitisation works in the selected cultural institutions. Findings reveal that digital cultural heritage resources in Malaysia is at risk in terms of digitisation policies, selection criteria, cost, staffing, technology, storage, metadata management and copyright. This study is very significant to professionals who are involved in archiving the digital cultural heritage as it can be a guide for managing risks in preserving valuable digital resources. The result of this study can be a practical tool for managing risks not only for digital cultural heritage but also to all documents in digital format. It also provides an insight to other researchers to seriously investigate other risks of digital cultural resources besides the risks that have been identified in this study. Keywords: digital resources, risk management, cultural heritage resources, Malaysia, digitization

1. Introduction Lynch (2002) states that, it is becoming increasingly important to archive the valuable cultural heritage of history each year. In Malaysia, there are a number of institutions that taking care of the national cultural heritage such as National Archives of Malaysia, National Library of Malaysia, National Museum of Malaysia, National Art Gallery, National Heritage Department, National Heritage Registers, Malacca Museum Corporation and so forth. Besides the government bodies, there are also private institutions involved in preserving Malaysian cultural heritage which are Heritage of Malaysia Trust, Penang Heritage Trust, Perak Heritage Society, Sarawak Heritage Society, Society Atelier Sarawak, Arts‐Ed Penang and Melaka Heritage Society. In this information communication technology (ICT) age, the natural cultural heritages have been transformed to digital format so that the resources can be exploited and shared throughout a worldwide network in a secure and cost effective manner. This situation is supported by Zuraidah (2006) in her study when she states that in order to remain competitive, every organisation must find new and creative ways to stay in the ‘‘game’’ in today’s fast changing information and communications technology ( ICT) era. Most national cultural heritage institutions are seeing a move from the hegemony of the physical record to the hegemony of the digital object. Basically, digitisation converts materials from formats that can be read by people (analogue) to a format that can be read only by machines (digital). In addition, digitisation makes the cultural heritage accessible since the openness of the internet causes cultural heritage to be available to a wider audience. Digital resources also enhance the efficiency of scientific research which are sophisticated search‐and‐retrieval tools and interoperability of collections. Digitisation also plays an important role in

Zuraidah Abd Manaf et al. preserving cultural heritage over time. By providing online access to digitised materials for researchers and the general public the originals are protected from use. Subsequently digitisation is also part of the preservation policy (Zuraidah, 2006). Many institutions, including those who are mentioned above, have been creating or collecting digital information produced in a wide variety of standard and proprietary formats, including ASCII, common image formats, word processing, spreadsheet, and database documents. Each of these formats continues to evolve, becoming more complex as revised software versions add new features or functionality. According to JICC (2006), digital preservation is the series of actions and interventions required to ensure continued and reliable access to authentic digital objects for as long as they are deemed to be of value. In the last 2 decades, digital technology has enabled people to create, use, and be enriched by information in ways that were unthinkable a generation ago. The need to preserve digital assets is only a few decades old, but it is growing and becoming more pressing by the day (Levi, 2008). She adds that the stakes are high. Libraries, museums, universities, research centres, and other institutions that have long taken role in preserving the collective knowledge and cultural heritage can no longer ignore the digital preservation challenge. While today's digital preservation systems will almost certainly be improved, institutions around the world are moving forward with good, well‐designed plans. The libraries, museums and archives serve not only to safeguard those resources, but also to provide evidence of one type or another of the work’s provenance, which goes towards establishing the authenticity of that work.

2. Research background 2.1 Problem statement As organisations rely more and more on digital technology to produce, process, store, communicate, and use information in their activities, the quantity of records being created in electronic form have increased exponentially. The technological challenge is compounded by the continuing extension of information technology, making the cultural heritage information increasingly more diverse and complex (Lim, Ramaiah & Pitt, 2003). Besides that, in Betts (1999)’ case study, he asserts that digital cultural heritage information is at risk of disappearing of becoming inaccessible because of the deterioration of storage media such as magnetic tapes. Other concerns include ever‐changing data formats and the fact that software and hardware become obsolete quickly. The greatest challenge to electronic record keeping is the evolution of technology (Coombs, 1999). Therefore, in order to preserve this information, institutions must manage collections in a consistent and decisive manner. It is important to decide what should be preserved, in what priority, and with what techniques. Hence, they most often overlook the issue of sustainability of their materials that have been digitised. It is found that findings from previous research revealed that there are many issues involved in the management of digitisation of cultural heritage resources in the country. In addition, there are a number of risks related issues that might be faced by cultural institutions if their digital files are not being managed accordingly. Zuraidah and Aliza (2009) describe in their findings that among the risks related issues that would be faced by cultural institutions if their digital resources are not managed accordingly would include availability and accessibility of information; authenticity and integrity of information; long‐term storage of information; and information disaster recovery issues.

2.2 Research objectives The aim of this study is to identify the potential risks in preserving Malaysian digital cultural heritage information that are practiced by several cultural institutions in Malaysia. In addition, this research attempted to investigate the sustainability factors adopted by the institutions towards the digitised materials.

3. Methodology This research uses multiple case studies at four (4) public cultural institutions in Malaysia. According to Zach (2006), the case study method, and in particular the multiple–case studies design, offers researchers a proven tool for achieving a deep understanding of a specific phenomenon, for example, the practices of digitisation and handling digital preservation of cultural heritage information in their institutions. The case study tends to provide in depth information and intimate details about the particular case being studied. The multiple case study approach was selected since this method offered distinct advantages to this research as it helped the

Zuraidah Abd Manaf et al. researcher to look at how contemporary events taking place provided significant contributions to the research questions. Moreover, the rationale for multiple case designs derived from the fact that the study involved an inquiry focusing on what are the potential risks before and after digitization process was being carried out in the sample institutions. Yin (2004) has justified the use of case study as a research strategy when the research questions are descriptive or explanatory. As the nature of the research questions is indeed descriptive and explanatory, this case study approach is seen as the most viable method to employ. In addition to this the case study is employed to gain an in‐depth understanding of the situation. Yin (2004) notes that the case study method helps the researcher to get close (i.e. in‐depth and first hand) understanding of the situation so that direct observations and data collection could be done in their natural settings. The data were collected through interviews, which were then transcribed, coded and analysed. The analysis methods used were descriptive and comparative analysis. For the purpose of this article, only comparative analysis is discussed. The research was done by developing a rough working framework based on the existing literature and pilot interviews. The emerging model and evidence was toggled back and forth throughout the data gathering and writing. The population of this study is the information professionals consisting of librarians, archivists, curators, or Information Technology (IT) officers from the government institutions that keep cultural heritage resources in Malaysia. Purposive sampling is used in this study where only four (4) main cultural heritage institutions (CHIs) in Malaysia are selected. The four selected cases were chosen because they are the main cultural institution under Ministry of Information, Communication and Culture of Malaysia which have been involved in digitisation projects since 2000. The four selected cases are CASE A refers to a leading library organization in Malaysia; CASE B is the main archives organization in Malaysia, CASE C is the national heritage institution is to collect and preserve significant museum’s collections and CASE D is a statutory body set up to provide services in visual art exhibition, preservation and development in Malaysia. The research unit of analysis are information professionals who are dealing with digitisation of cultural heritage resources in their organisations.

4. Findings and discussion 4.1 Potential risks The findings revealed that the potential risks in the digitization of cultural heritage lies in the selection criteria, digitisation policy, cost, staffing, technology, storage, metadata and copyright issues. 4.1.1 Selection criteria According to Puglia (1999), items selected for digital collections should fit into the scope, purpose, and audience of the project itself as stated on the project description form. The findings found that most of the cultural institutions select materials to be digitised based on these criteria (arranged by the most preferred criteria):

Cultural value

Historical value

Materials that allow copyright access

Materials that have digitisation procedures and standard

Frequently used by users

Help them to save space without considering the value of resources

Educational and research purpose

Majority of the cultural institutions select materials to be digitised which have cultural and historical value and materials that allow copyright access. These WIU Libraries Digitization Policies Committee (2009) and Dasar Pendigitalan Koleksi Perpustakaan Negara Malaysia (2010) support these findings where both of the policies highlight cultural and historical value are the most essential resources need to be digitised. As cultural

Zuraidah Abd Manaf et al. institutions, all of the cases in this study select the right materials to be digitised since the selection of materials are based on the main function of their organisations. Hirtle, Hudson and Kenyon (2009) believed that copyright is an important issue to be considered in doing digitisation projects. It is found that all the respondents prefer to choose materials that are copyright free since they are not bounded with Copyright Act. However, as cultural institutions, they cannot limit selection to materials that are only free from copyright since there are a lot of resources out there which are valuable even though they are copyrighted. The risk of valuable materials not being digitised is present since the majority of the cultural institutions only prefer to choose materials that are copyright free to avoid problems in term of copyright without considering the valuable of copyrighted materials. The findings report that only two of the cases prefer to digitise materials that have digitisation procedures and standard. Another two cultural institutions do not consider this criterion because for them, they do not need specific standard and procedures in digitising the materials as their procedures are very simple. Nevertheless, these two cultural institutions should be aware that choosing the materials that already have digitisation procedures and standard will minimise risk of accessibility of the digital resources later. The findings shows that our digital cultural information is risky since the cultural institutions do not really prefer to choose materials with that do not have the digitisation procedures and standard. Furthermore, not all the cases deem to digitise materials that are frequently used by users. Prioritise for highly demand of resources by users should be the main criterion since as an information institution, clients should be the first priority. As stated in WIU Libraries Digitization Policies Committee (2009), items known to have high use should be given high priority, as should items known to have a high potential for use once they are digitised. For example, items with a specifically identified and interested audience. If cultural heritage does not provide resources based on users’ information needs, it shows that the institution does not want to excel as a successful information centre. Cultural institutions in this study are exposed to risk in selecting the right materials to be digitised where some of the cases have a preference for materials that only can save the space without considering the value of the resources. International digitisation Policy such as Dasar Pendigitalan Koleksi Perpustakaan Negara Malaysia (2010) and WIU Libraries Digitization Policies Committee (2009) do not highlight this as the preferred criterion since the most important element to consider is the value of the resources and not the size of them. Not all the cases consider the educational and research purpose even though it is an important that need to be thought about. The cultural heritage institutions might think that their core business is more to historical and cultural, so that is why educational and research are not the priority. 4.1.2 Digitisation policy More than half of the cases in this study do not have a standard digitisation policy. Only CASE A has developed their digitisation policy and it was published in July 2010 namely Dasar Pendigitalan Koleksi, Perpustakaan Negara Malaysia (2010). It is a complete digitisation policy and was developed based on the best practices of international organisation which are involved in digitisation projects earlier. The best practices are such as National Library of Australia and NISO. The other cases in this study except CASE A do not have a digitisation policy. However, CASE B is in developing of their digitisation policy and it is estimated to be published in 2014. The ongoing process of digitisation policies shows that CASE B is aware pertain to the importance of having a digitisation policy. However, the other two (2) cultural institutions do not have any since for them their digitisation process are not complicated as others. The result indicates that not all cultural institutions in Malaysia have systematic and procedures in digitising their resources. This is one of the risks for the digital resources since they are not produced in a proper way and this will affect the long‐term accessibility of the resources. The findings reveal that even though CASE B does not have a digitisation policy, they are able to minimise risks by following guidelines from NISO and NARA. According to CASE B, they choose to follow the international guideline because they want their digital resources fulfilled the international standard requirement so that the resources can be accessed internally and externally.

Zuraidah Abd Manaf et al. A study conducted by Mayesti, Rachman, and Yayan (2011) also reports that Centre of Quranic Studies Library does not have written policy on digitisation and digital preservation. However, the digitisation and digital preservation activities have been done by referring to a Standard Operating Procedure which contains a list of tasks and timeline that must be done by librarians. Chowdhury and Chowdhury (2003) explain that the digitisation and digital preservation policies or strategies must be developed since the institution today is managing digital collections. Besides that, as reported by Zuraidah (2006) in her study found that one of the institutions realised the importance of proper policies to guide the course of action for the achievement of their digitisation initiative goals. The policies that are relevant in digitisation are selection policy, access policy, content management policy, intellectual property policy and preservation policy. For those institutions that admitted not having any policy, they had the intention to develop it in the immediate future. A proper policy guideline consisting of the principles and strategies of digitisation are vital to support the organisation’s goals and objectives (Zuraidah, 2006). 4.1.3 Cost As stated in the findings, all of the cultural institutions are not allocated with specialised budget for digitisation. This condition is one of the barriers for Malaysian cultural institutions to excel and provide good digital collections. This result is supported by a study from Shariful (2011) in Bangladesh where in the research paper states that financial constraint is one of the major barriers for library digitisation in Bangladesh. In Bangladesh, very few libraries have separated budget for library digitization, so it is very hard for the libraries to undertake any sort of digitisation project due to lack of budget. Majority of the cases in this study are given one‐off budget if there is any projects that need them to digitise their collections. For instance, CASE B, which have been given large amount of budget for digitisation to support Economic Transformation Program (ETP) in 2010. Nevertheless, if the cultural institutions only get one‐off budget for digitisation, this situation will limit them to digitise their collections since such the special projects should have their certain target, for instances, project given to CASE B should digitise two (2) billion collections only. According to the findings, most of the cultural institutions do not other alternative budget for digitisation instead of budget from ministries. It is found that only CASE A has an alternative budget which is called Trust Account. This shows that only CASE A has effort to minimise risks in term of budget for emergency cases. It is very important for an organisation to measure the Return of Investment (ROI) after they invest for a certain huge budget, for instance in digitisation projects. In basic terms, according to Kaufman and Watstein (2008), ROI is just that: the income – or value – received as a result of an amount invested in an asset. In this study, most of cultural institutions measure the ROI by providing and sending the statistics of the digital collections usage, borrowed and bought by clients. This is an effective way for the institutions to reduce risks by investing money for unbeneficial projects. Most of the cultural institutions in this study could not afford to hire staffs special for digitisation works. They only hire temporary and contract staffs when they have digitisation projects. According to Hazen, Horrell and Merrill‐Oldham (1998), labour expenses, for instance, often reflect only a pro‐rated price per page that over‐looks the real cost of a full‐time employee. 4.1.4 Staffing Findings stated that majority of the cultural institutions hire staffs based on academic achievement and experiences. These two (2) criteria are very important since staffs that have suitable academic qualification and at least minimal experience is capable to produce quality of works. However, Zuraidah (2006) explains that the human factor is equally important as an asset and resource to organisation. Without knowledge, skills and the right attitude of the professionals, the initiatives will go nowhere and eventually collapse. Therefore, the findings reveal that in term of selecting staffs, the cultural institutions are not exposed to risks since they recruit them based on their abilities on qualification and experiences. The result reports that the staffs in cultural institutions are lack of training in digitisation works. Only one of the institutions takes training as an important part for their staffs, whereas the other cultural institutions more focus only on attending conference and symposium. It is good exposure for the staffs by attending conference and symposium. However, hands‐ on training is more effective for them to give clearer view of doing digitisation process in a non‐risk. It is a bad condition to one of the cultural institutions where they do not give formal training to the staffs because their

Zuraidah Abd Manaf et al. management always change staffs to digitise collections from time to time. This situation contributes bad impact to the digitised materials since the staff cannot develop the skills that will lead them to produce and better and quality digitised collections. Shariful (2011) states in his research paper that skilled manpower is must for the library digitisation activities, without that it will be the untouched dream all the time for the libraries of Bangladesh. For the digitisation process libraries need smart, intelligent personnel with adequate technical and ICT knowledge, whereas, the picture is totally different in researcher’s country, which is Bangladesh. Most of the libraries do not have that kind of personnel; few libraries have that kind of personnel but very small in numbers. Not all the cultural institutions have visits to other organisation which are advance in digitisation to look at their digitisation process. Only one of the cases occasionally visits the best practices to learn from them. Community Mental Health Oakland Country Authority (2011) defines best practice as a compilation of evidence based practice, promising practices, emerging practice, value‐based practice, and practice‐based evidence. As such, the adoption and implementation of “best practices” appears to have reached epidemic proportion (Sanwal, 2008; Todero, 2002; Argyris, 2000). The allure of a best practice is the leaders’ belief that there is truly one best practice and that implementing another organisation’s successful practices is a sort of magic bullet (Sanwal, 2008). There, findings show that only two of the cultural institutions taken visit to best practice organisation as consideration in cutting risks of having inappropriate digital cultural heritage information. In general, knowledge management can be defined as the achievement of the organization's goals by making the knowledge factor productive (Beijerse, 2000). It is the systematic management of knowledge‐related activities, practices, programs and policies within the enterprise (Wiig, 2000). Knowledge management activities aim to effectively apply an organization's knowledge to create new knowledge to achieve and maintain competitive advantage (Alavi & Leidner, 2001; Mason & Pauleen, 2003). In this study, findings reveal that all of cultural institutions practice the concept of knowledge management, especially after they have attended courses. They apply the sharing of knowledge in a special meeting to share the new knowledge and skills from the courses, workshop or training. One of the cases, which are CASE A, has a special channel for practising knowledge management which is by sharing their knowledge and ideas in a knowledge bank database. 4.1.5 Technology Zuraidah (2006) asserts in her study that digitisation is a costly exercise requiring high investment usually from public funds. There are significant risks to these investments due to the adoption of inappropriate technologies and standards. This can result in creating resources which are quickly obsolete and unusable or which require the investment to be repeated within a short time frame. In addition, Lusenet (2002) states that basically there are three ways in which digital materials become inaccessible which are degradation of the media on which they are stored, obsolescence of software making it impossible to read digital files and introduction of new computer systems and peripherals that cannot handle older materials. A digital resources management system a multi‐layer architecture to support the service and management in digital library or institution based on Linux and other open source software. A good system should function as: a platform for developing DL like applications; provide a information model to organize digital objects; store different types of digital data; support efficient access to these data by index and navigation; support loading of data resources and creation of meta data; and support interoperability through OAI, Z39.50 (Zhou, 2004). Findings report that only one of the cases has not a specialised system to manage the digital collections. This finding shows that three (3) of the institutions aware the significant of having a digital resources management system. Another cultural institution is going to have a system in 2014 and now the system is still in developing process. In term of linkage and interaction between other institutions, only CASE A has agreement to share the collections with other institutions. Even though the digital collections only can be accessed via intranet, they allow outside institutions to access their digital contents and the process of cooperation need to follow their rules and regulations to avoid risks. Other cases in this study are not allowed outsiders to access their digital contents. The outsiders should come physically to their institutions for accessing the resources. 4.1.6 Storage All of the cultural institutions store their digital resources in house. They are could not afford to keep the digital collections in service providers’ servers. This is a risk for the cultural institution since they do not have

Zuraidah Abd Manaf et al. other backup server outside the institutions if there is anything happened to their building and existence servers. The finding also reveals that digital resources in one of the institutions have been cracked due to there was no firewall installed in the server. Firewalls are the cornerstones of corporate intranet security. Once a firewall is acquired, a security or systems administrator has to configure and manage it to realize an appropriate security policy for the particular needs of the company. 4.1.7 Scalability According to Mayesti, Rachman, and Yayan (2011), the computer storage capacity will determine how many data and files could be stored inside. Findings of this study found that only storage in one cultural institution is not scalable. Other storages in another three (3) institutions are scalable. However, a research that has been done in Indonesia also found that their capacity of storage is also not enough. Mayesti, Rachman and Yayan (2011) report that if the storage has only small capacity, the digital files that could be stored will be limited. Webb (2003) explains that the virus attack has become a crucial problem in the digital preservation. Viruses may cause damage the integrity of digital collections (file corrupt). Kahn (2004) stated that the best protection to solve computer problems (which has been caused by virus) can be done by installing antivirus software at each computer and networks. The findings reveal that most of cultural institutions in this study are concerned about antivirus and always update it from time to time. 4.1.8 Metadata The role of metadata in ensuring long‐term access and management is analyzed, described, and commented upon by other researchers, including Alemneh et al. (2002). Maintaining usable and sustainable digital collections necessitates maintaining high‐quality metadata about those digital objects. The two aspects of cultural institutions data quality are the quality of the data in the objects themselves, and the quality of the metadata associated with the objects (Diallo, 2011). Diallo (2011) adds that because poor metadata quality can result in ambiguity, poor recall and inconsistent search results, the existence of robust quality assurance mechanisms is a necessary feature of well‐functioning cultural institutions which provide digital collections. In order for end users to benefit fully from the development of digital libraries, responsible and viable service providers need to address metadata quality issues. Low quality metadata can render an information institution almost unusable, while high metadata quality can lead to higher user satisfaction and increased use (Stvilia1 et al., n. d.). A quality metadata also can be measured by metadata scheme that an organisation is using. The finding of this study reveals that one of the cultural institutions has a metadata scheme. However, the metadata scheme that is used is not user friendly in term of managing metadata of their digital resources as the scheme is more suitable for profit making organisation, and not for non‐profit organisation such as cultural institutions. This situation contributes risks for the digital resources since it will lead to loss of data and other consequences. If compared to other institutions, the risk that they have is much minimum rather than other cultural institutions which do not have metadata scheme. All of cases in this study include the descriptive metadata for their digital resources. As stated in Chapter Two, descriptive metadata encompass a range of information from basic elements such as title and subject to more advanced elements such as geographic or temporal coverage and relationships (UMDM, 2011). There are three (3) of the cultural institutions include the technical metadata for their digital resources. According to IBM (2011), technical metadata is data about the processes, the tool sets, the repositories, the physical layers of data under the covers. It covers data about run‐times, performance averages, table structures, indexes, constraints; data about relationships, sources and targets, up‐time, system failure ratios, system resource utilisation ratios, performance numbers. However, findings shows that most of cultural institutions do not consider the technical metadata is important for them. In term of putting IPR, only one of the cases does not include it. Besides that, not all the cultural institutions consider that history metadata is an important part to be included. According to Atlassian Confluence (2011), history is of importance to the present because the current situation for metadata management within the institutions, and some of the challenges to be faced in the future, reflect past strategies. In addition, IPR is an important element of metadata that need to be considered by cultural institutions. Most of the institutions are aware about the important of putting element of this. Since all of the cases are not allowed outsiders to access their digital resources, they automatically they do not open the resources to be integrated. The metadata elements also do not provide multi‐language and multi‐national network.

Zuraidah Abd Manaf et al. 4.1.9 Copyright Copyright assessments play a defining role in digitising projects and must be addressed early in the selection process. Therefore, a watermarking technique is to insert copyright information into digital images that the ownerships can be declared. A watermarking technique is to prevent digital images that belong to rightful owners from being illegally commercialised or used, and it can verify the intellectual property right (Lee, Lin, Su & Lin, 2008). The findings reveal that most of cultural institutions embedded visible watermarking but only one of the institution embedded invisible watermarking. Invisible watermarking is more effective than visible watermarking as even though digital watermarking cannot by itself prevent copying, modification, and re‐ distribution of documents, if encryption and copy protection file, watermarking allows the document to be traced back to its rightful owner and to the point of unauthorised use (Su, Hartung & Girod, 1998).

4.2 Sustainability factors adopted In considering the suitability of particular digital formats for the purposes of preserving digital information as an authentic resource for future generations, it is useful to articulate important factors that affect choices. The sustainability factors that are taken as important consideration planned by Library of Congress are very systematic. The factors that are considered by them are applied in this study which consists of disclosure, adoption, transparency, self‐documentation, external dependencies, impact of patents, and technical protection mechanisms. The findings show that Malaysian cultural institutions do not take the sustainability factors seriously. However, they are aware of the importance of technical protection mechanisms factor. Sustainability in the institutions maybe the factors have been applied by them without their awareness since the processes are in embedded in their routines processes. Besides that, they are not familiar with these specific factors.

5. Conclusion Digital collections are rapidly becoming an integral facet of any type of institutions around the globe. The volume and heterogeneity of digital resources grows daily, presenting challenges across cultural heritage institutions and other repositories of digital resources. It is reported in finding that in term of selecting criteria and storage, the digital cultural information in Malaysia are somewhat saved from hazardous risks. Selecting criteria that are considered by all the cultural institutions are totally base on their cultural and historical values of the resources. It can be concluded that digital cultural information in Malaysia is exposed to risks since most of the cultural institutions producing the digital resources are not managing digitization projects according to standard digitisation policy. Digitisation of resources is done based on their general skills and knowledge. Apart from that, the lack of skilled workers who produce the digital product is another risk which could not be ignored. This is a hazardous situation since digitisation process is extremely complex, difficult in manner and need highly qualified and experienced professional to do the job efficiently. Moreover, inappropriate metadata management of digital contents is another risk of Malaysian digital cultural resources management. The metadata associated with digital objects could affect the access to and usability of items in the collection. The process of digitisation in the cultural institutions do not take sustainability factors as important consideration due to the lack of understanding among them towards these factors. Therefore, the management of digitisation process of the cultural information in Malaysia will not be in an excellent condition until the issues of the followings are resolved. One of the major issues is allocated budgets for digitization projects. Projects with sufficient budget will cover the installation of quality digital resources management system, recruiting sufficient number of skilled and knowledgeable staffs, having enough and sophisticated facilities, having a systematic and efficient metadata scheme and embedded both visible and invisible watermarking to the digital resources. In a nut shell, the requirement of a framework that show the potential risks that might be exposed to the digital cultural information is really necessary for cultural institutions so that they always aware on it to avoid deterioration of the valuable information sources.

References Alavi, M. and Leidner, D. E. (2001). Knowledge management and knowledge management systems: conceptual foundations and research issues. MIS Quarterly, 25 (1), 107‐36. Alemneh, D. G. (2002). Metadata quality assessment: a phased approach to ensuring Aliza, I. (2010). Assessing the practice of trusted electronic records management in Malaysian government‐controlled companies. PhD Dissertation. Argyris, C. (2000). Flawed advice and the management trap. New York: Oxford.

Zuraidah Abd Manaf et al. Atlassian Confluence (2011). A brief history of metadata (in the ABS). Retrieved December 1, 2011, from: http://www1.unece.org Betts, M. (1999). Business worry about long‐term data losses. Computerworld, 33(38), 22‐24. Chowdhury, G. G and Chowdhury, Sudatta. (2003). Introduction to digital libraries. London: Facet Publishing. Community Mental Health Oakland Country Authority. (2011). Definition best practice. Retrieved December 15, 2011 from: http://www.occmha.org/index.php?option=com_content&view=article&id=117&Itemid=177 Coombs, P. (1999). The crisis in electronic government record keeping: a strategy for long‐term storage. Library Computing, 18 (3), 196‐202. Perpustakaan Negara Malaysia (2010). Dasar pendigitalan koleksi Perpustakaan Negara Malaysia. Retrieved October 1, 2011 from: http://www.pnm.gov.my/upload_documents/DasarPendigitalanKoleksiPerpustakaan.pdf ‐ Hazen, D., Horrell, J., and Merill‐Oldham (1998). Selecting research collections for digitisation. Washington: Council on Library and Information Resources. Hirtle, P. B., Hudson, E. & Kenyon, A. T. (2009). Copyright and cultural institutions: guidelines for digitization for U.S. libraries, archives, and museums. Ithaca, New York: Cornell University Library. JICC (2006). Digital preservation briefing paper. Retrieved December 5, 2011 from: http://www.jisc.ac.uk/publications/publications/pub_digi preservationbp.aspx Kahn, M. B. (2004). Protecting your library’s digital sources: the essential guide to planning and preservation. Retrieved from: http://203.128.31.71/articles/protecting_your_librarys_digital_sources.pdf Kaufman, P. & Watstein, S. B. (2008). Library value (return on investment, ROI) and the challenge of placing a value on public services. Reference Services Review. 36 (3), 226‐231. Retrieved from: http://www.emeraldinsight.com Levi, Y. (2008). Digital preservation: an ever‐growing challenge. Information Today, 25 (8), 22‐22. Retrieved December 6, 2011, from: http://search.proquest.com/docview/214811375?accountid=42518 Lim, S. L., Ramaiah, C. K., & Pitt, K. W. (2003). Problems in the preservation of electronic records. Library Review, 52 (3), 117‐125. Retrieved February 14, 2011 from: http://www.emeraldinsight.com Lusenet, Y. de (March 2002). Preservation of digital heritage. Discussion paper prepared for UNESCO. European Commission on Preservation and Access. Lynch, C. (2002). Digital collections, digital libraries and the digitization of cultural heritage information. Microform and Imaging Review, 31 (4), 131‐145. Retrieved January 19, 2011 from: http://www.emeraldinsight.com Malaysia Vacation Guide (2011). Malaysia national heritage. Retrieved February 6, 2011 from: http://www.malaysiavacationguide.com/nationalheritage.html Mason, D. & Pauleen, D. J. (2003). Perceptions of knowledge management: a qualitative analysis. Journal of Knowledge Management, 7 (4), 38‐48. Puglia, S. (1999). The costs of digital imaging projects. RLG Digi‐ News 3, 20‐23. Sanwal, A. (2008). The myth of best practices. The Journal of Corporate Accounting and Finance. Shariful I. (2011). Toward digitization: problems and prospects for the libraries of Bangladesh. Paper presented in World Congress Muslim Librarians and Information Scientists 2011. Gombak, Kuala Lumpur: International Islamic University Malaysia. Su, J. K., Hartung, F. & Girod, B. (1998). Digital watermarking of text, image, and video documents. Comput & Graphics, 22 (6), 687‐695. Todero, J. B. (2002). Change for the right reason: What is a best practice? Community & Junior College Libraries, 11 (1). UMDM (2011). University of Maryland Descriptive Metadata. Retrieved December 1, 2011 from: www.lib.umd.edu/dcr/publications/taglibrary/umdm.html Webb, C. (2003). Guidelines for the preservation of digital heritage. Retrieved from http://unesdoc.unesco.org/images/0013/001300/130071e.pdf Wiig, K. M. (2000). Knowledge management: an emerging discipline rooted in a long history, in Despres, C. and Chauvel, D. (Eds), Knowledge Horizons: The Present and the Promise of Knowledge Management, Butterworth‐Heinemann, Boston, MA, 4‐26. WIU Libraries Digitization Policies Committee (2009). WIU libraries digitization policies recommendations. Retrieved December 1, 2011 from: http://www.wiu.edu/libraries/administration/policies/DigitizationPolicies_0809.pdf Yin, R. K. (2004). Applications of case study research. USA: Sage Publications Inc. Zach, L. (2006). Using a multiple‐case studies design to investigate the information‐seeking behaviours of arts administrators. Library Trends, 55 (1), 4‐21. Zhou, L. (2004). A digital resource management system based on open source systems. Gelato Meeting. Tsinghua University. Zuraidah, A. M. & Aliza I. (2009). Malaysian cultural heritage at risk? A case study of digitisation projects. Library Review, 59 (2), 107‐116. Retrieved January 21, 2011 from: http://www.emeraldinsight.com Zuraidah, A. M. (2006). The state of digitisation initiatives by cultural institutions in Malaysia. Library Review, 56 (1), 45‐60. Retrieved January 21, 2011 from: http://www.emeraldinsight.com

Investigating the Quality and Effectiveness of the Picture Archiving Communication System (PACS) at an Academic Hospital in South Africa Bronwyn Abrahams and Michael Kyobe Department of Information Systems, University of Cape Town, Cape Town, South Africa michael.kyobe@uct.ac.za Bronwyn.Abrahams@westerncape.gov.za Abstract: PACS is a valuable workflow tool within a hospital environment and specifically in radiology departments. PACS is increasingly adopted in the developing countries like South Africa and India. While there has been much enthusiasm about the potential for PACS, major challenges persists in PACS implementation which calls for continuous evaluation of this system. However a review of previous PACS research indicates fragmented evaluation attempts and as such lack of comprehensive assessment of PACS systems. In addition, due to this fragmented work, the significance of factors influencing PACS quality has not been established. The present study investigated more comprehensively the quality of PACS in an academic hospital in South Africa and determined the extent to which the initial objectives for its development were met. A single case study involving 52 participants involved in PACS implementation and utilisation was conducted and mixed methods were used to collect and analyse the data. The findings suggest that many of the project objectives were achieved. In particular, PACS supports work processes within the Radiology department; Clinicians obtain images immediately and their interpretations of the results were perceived to be fairly accurate. However major challenges relating to security and compliance, maintainability, reliability of the data still exist and require urgent attention. This is also confirmed by the regression analysis which reveals that security and compliance are the most significant factors influencing the quality of PACS implementation in this academic hospital. Keywords: picture archiving communication systems, academic hospital, evaluation, South Africa

1. Introduction PACS is a valuable workflow tool within a hospital environment and specifically in radiology departments. PACS is increasingly adopted in the developing world like South Africa and India. While there has been much enthusiasm about the potential for PACS, major challenges persists in its implementation (You, Yun and Tu, 2008). The recurrence of these challenges calls for continuous assessment and monitoring of PACS projects. The present study investigates the quality of PACS in an academic hospital in South Africa. It determines whether the initial objectives for its development were met and the factors that significantly influence the quality of PACS implementation. Understanding such significance would enable focusing resources to the appropriate interventions for enhancing PACS effectiveness. Background to the Academic Hospital Tygerberg Hospital is a tertiary institution with approximately 1647 beds. The radiology department performs 150000 exams per year (Philips Healthcare report, 2010). The hospital system has gone through a transition from analogous to digital technology. Prior to this transition, images could not be printed effectively due to high costs; folders were misplaced; comparison of results was almost impossible; and patients were often re‐ examined which over‐exposed them to the risk of radiation (Schulze, Ackerman, Greyling, Viljoen and Andronikou, 2007). The Department of Health invested R25 million in the implementation of PACS and the radiology information system (RIS) at the academic hospital (Hawkins and Schulze, 2008; Philips Healthcare report, 2010). The key objectives of this PACS project were: elimination of film with a view to reduce costs; ensure effective distribution of images and reporting throughout the enterprise; guarantee speedy access to medical data by clinicians and radiologists; ensure efficient patient flow within radiology department; ensure efficient radiology reporting; and improvement to the quality and quantity of statistical data (Philips Healthcare report, 2010). Research Objectives In the light of the recurrent challenges involved in the implementation of PACS and the need for accountability for the large investment of funds by the government into these projects, it was imperative to assess the

Bronwyn Abrahams and Michael Kyobe effectiveness of PACS, the extent to which its objectives have been achieved and to understand the significance of the factors influencing its implementation. Therefore the objectives of the present study were:

To evaluate the extent to which the intended benefits have been realized by examining the quality and effectiveness of PACS.

To determine the significant factors influencing the quality of PACS implementation.

Research Questions The following research questions formed the basis of this study: Does PACS support work processes within the radiology Department? Do Clinicians obtain results sooner? Is the Clinicians’ interpretation of PACS results accurate? Has the use of PACS improved patient outcomes? Is the implementation and use of PACS cost‐effective? And which of the factors influence the Quality of PACS implementation the most? In the following sections the researchers review literature on IS evaluation methods and health information systems in particular. Challenges involved in such evaluation are discussed and the key issues needed to be considered during the evaluation processes are identified. A conceptual model indicating the factors influencing the quality and effectiveness of PACS is presented. This model is then tested in an academic hospital and the results are presented. These results are then discussed, conclusions are drawn and recommendations made.

2. Literature review According to Faggioni et al. (2011) the Picture Archiving and Communication System (PACS) has been tagged as a very valuable workflow tool designed to assist radiologists with efficient image interpretation. Adoption of PACS is growing in South Africa. Irrespective of all the excitement regarding what PACS can do, it has been reported that major challenges in its implementation still persist (You, Yun & Tu, 2008). Earlier studies have raised concerns such as: physicians not obtaining imaging results any sooner when using PACS than with film (Anderson & Flynn, 1997); Clinicians hard pressed with time resort to interpreting images themselves rather than consulting a radiologist; difficulties in identification of right vendors; integration of PACS with existing health information system (HIS); lack of standardization and proper archiving of medical data (Kalyanpur, Singh and Bedi, 2010). In addition, there are high costs involved in the implementation and maintenance of PACS; ergonomics issues faced by radiologist using the system (Prabhu, Ghandi, and Goddard, 2005) and potential legal liabilities (e.g. in event of lost results, misdiagnosis, and privacy abuse (Pattynama, 2009). These challenges continue to occur in many PACS implementation and it is necessary to carry out continuous evaluation of these systems especially in the developing world where such systems have recently been deployed (Van Heerden, Lockhat and Barn, 2011). Costello et al. (2007) contend that it is through effective IT assessment that organizations can reach maturity in the use of their IT investments.

3. Evaluation of information systems (IS) Evaluation is the assessment of the strengths and weaknesses of programs, systems, policies, personnel or organizations to improve their effectiveness. Ammenwerth et al. (2003) define evaluation as the decisive assessment of defined objects, based on a set of criteria, to solve a given problem. It is a multidisciplinary concept and many different theoretical approaches to evaluation have been employed in the IS field (Berghout and Remenyi, 2005). The economics/accounting theory for instance has been used in the assessment of the value and risks of IS investment (e.g. value of IS capital investment, economies of scale and scope, benefits to individual users, uncertainty of IS and improvement in IS utility) Bakos and Kemerer (1992). Interpretive approaches and grounded theory have been used to match IS development with the dynamics of organizational culture and politics (Serafeimidis and Smithson, 2000). Traditional evaluation models (e.g. accounting, economic and technological) have been criticised for seeing evaluation as an external judgement of IS. They treat IS as if it exists in isolation from its human and organisational components or effects. Interpretive approaches on the other hand aim to involve a wide variety of stakeholders to understand and assess the meanings they assign to the phenomena. Recent studies emphasise that the methods employed in IS evaluation should be more holistic in nature and appropriate to the situation investigated. Hallikainen and Chen (2005) for instance recommend that an IS evaluation should examine i) the outset situations

Bronwyn Abrahams and Michael Kyobe (organizational norms and values, IS project contingencies and IS project resources), (ii) the business development process (e.g. IS development and procurement) and (iii) the outcomes (e.g. success of IS implementation, success of IS investment and success of IS functionality).

4. Evaluation of PACs In their study of the benefit of PACs to the clinical users, Van Heerden, Lockhat and Barn (2011) emphasise the importance of evaluating PACS systems. They state that digitalised hospital imaging archiving systems are still in their infancy in the South African (and African) setting and need to be monitored carefully. Evaluation of IT projects is also a key requirement in ensuring accountability for the large investment of public funds (Marion, 2008). There is however little agreement on the essential role of evaluation, on what and how to evaluate, whom to involve and within what paradigm to proceed (Klecun and Cornford, 2005). Health care systems usually have to meet many political imperatives. Given their innovative and evolving nature, it may take time before any effects of their introduction are measured effectively (Williams et al., 2003; Ammenwerth et al.,2003). Anderson and Flynn (1997) also argue that while some technical aspects of system performance can be measured directly, it is difficult to design studies that show relationships between the installation of a PACS and measurable changes in organizational process or clinical outcomes.

5. Aspects of PACS implementation to be evaluated Many studies on evaluation of health information systems have identified some specific focus areas for evaluation. According to Oosterwijk (2004) there must be clear objectives for implementation of PACS and these must be defined prior to procurement of the system. Chen, Bradshaw and Nagy (2011) emphasise careful consideration of PACS’s architecture (which vary widely) and its interoperability. Kalyanpur, Singh and Bedi (2010) also consider maintenance, troubleshooting, training, and integration to be major considerations. The efficiency and performance of the system need to be measured regularly and performance metrics should mirror the architecture, the program as well as the implementation policies (Khaddaj et al., 2004). Scalability is another important issue which relates to extent to which the system can cope with a significant amount of processes and lengthy functioning programs (Khaddaj et al., 2004). Cost‐Benefit analysis is also of utmost importance. If the perceived costs of developing the system exceed the benefits, then it may not be worthwhile going ahead with the implementation (Khaddaj et al., 2004). Usability is also critical – it has been described as the capability of a developed system to be accepted for the anticipated objective. Chen, Bradshaw and Nagy (2011) report the difficulties involved in using navigation features, mouse movements and their impact on user productivity and satisfaction with the product. Useful systems usually contain adequate documentation, support features and straightforward interface (Khaddaj et al., 2004). It is also expected that the system should be Portable, Reliable and robust and meet ergonomic requirements (Naul and Sinclear, 2001). Compliance with legislation and standards, legal issues, misdiagnosis or missed diagnosis, security, confidentiality and disclosure of records could all present major challenges (Michalson 2005; White, 2002). In the present study, the researchers examined the impact of the following factors on the quality of PACS implementation (Quality of PACS): These include: Maintainability; Efficiency; Scalability; Cost/Benefit; Usability; Portability; Security confidentiality and compliance; Robustness; Correctness; and Reliability (Khaddaj and Horgan, 2004). The conceptual model representing the relationship between these variables and the quality of PACS is presented in Figure 1.

6. Methodology The study adopted a positivist philosophy and the research strategy involved a variety of methods (i.e. a survey, interviews and analysis of secondary data). Use of a variety of data collection methods has been recommended in previous studies (Goertz and Mahoney, 2012). Secondary data sources comprised a Business Case, Vendor (Philips) Healthcare Report, and Budget Speech by the Premier, Government Gazettes regarding the State Information Technology (SITA – 1998, Amended Act 2002) and the Public Finance Management Amendment Act – 1999. These secondary sources provided useful information for the development of the survey instruments. The survey questionnaire was devised with the assistance of the hospital Quality Department. It consisted of a combination of responses to statements and questions, measured on a 5 point

Bronwyn Abrahams and Michael Kyobe Likert scale (1 = lowest and 5 = highest option). Free‐response questions were also used to obtain a broader perspective on the respondent’s view about the quality of PACS in the hospital. These questions assessed: the quality of PACS; its maintainability; scalability and integration; costs and benefits derived from its implementation and usage; usability; portability; security and compliance; robustness and reliability.

Figure 1: Factors influencing the quality of PACS In order to refine further our questionnaire instruments, a pre‐test was administered to a relatively small number of potential respondents. Given that different stakeholders may have different needs and interests and may attribute different outcomes to the PACS, interviews were conducted with 16 representatives from various groups of users (e.g. Radiologists, Clinicians and Technologists and PACS Administrators and Consultants). The sample was selected using the non‐probability and judgement sampling approaches. The survey instrument was then administered by one of the researchers to 52 staff members and later semi‐ structured interviews were conducted.

7. Findings and analysis Demographics The participants in this study were drawn from various department of the hospital. These included the Department of Health representatives (Local); Medical community of healthcare workers – Clinicians, Radiographers, Radiologists, PACS/RIS administrators and Clerks; Hospital Management Staff (Involved with the adoption of PACS); Interested 3rd party medical practitioners; Centre of e‐Innovation staff(I.T); Software Implementation Vendors (PHILLIPS, AGFA) and some patients. There were 31 Radiographers; 2 PACS Administrators; 2 Medical officers; 3 Clinicians; 5 Radiologists; 3 Registrars; 3 Consultants and 3 Technologists. Those interviewed were 6 Radiographers, 2 Clinicians and 2 PACS Administrators.

Bronwyn Abrahams and Michael Kyobe Qualitative Analysis Thematic analysis was conducted to analyse the qualitative data (Braun& Clarke, 2006). We adopted the steps suggested by Braun and Clarke in this analysis. First the researchers listened carefully to the recordings. Codes were used to highlight pertinent issues and these were transcribed into MS‐word. Themes that emerged in this analysis are outlined below: Table 1: Main themes and responses Key Themes Quality and Efficiency

No 19

Usability

Cost effectiveness Security and compliance

3 14

Maintenance

Web Access & PACS integration

responses The technology used is satisfactory; Good quality images are generated; the system is fit‐for‐purpose; Patient data and images are updated; Quality of service is satisfactory; there are improvement of PACS diagnosis; the system and workflows are effective; Network is fast Insufficient staff to capture patient data; Inadequate training of staff; there is need for greater responsibility; effective communication needed to improve patient care; there is need to understand roles and responsibilities to improve workflow; proper procedures need to be followed in shutting down of the system. Costs of operation reduced Evaluation of users of PACS system done regularly; PACS diagnosis sometimes evaluated; Auditor general requirements are met; there are reporting monitors in place; Compatibility issues exist; Data integrity issues exist; Department does not compliance with some standards; Data quality is maintained; Need to fix system errors, Proactive maintenance of network necessary; 24/7 Accessibility crucial; Contract between department and vendor need scrutiny; Backup power needs regular maintenance PACS integration with other systems not effective; limited budget available for this; challenges with licence of integrated systems; and also the image quality being limited by monitors used

Quantitative Analysis Reliability Assessment The reliability test was conducted using the Cronbach’s alpha (Saunders, et al., 2009, p. 374). The Cronbach alpha coefficient above the recommended 0.70 would confirm the reliability of the questions used (Nunnally, 1978). Table 2 below shows the reliability test results for the constructs in the conceptual model. Table 2: Assessment of reliability Variable Overall Quality of PACS Maintainability Efficiency (in performance) Scalability Cost/Benefit analysis Usability Portability Security, confidentiality & compliance Robustness Correctness Reliability

No of Items 4 2 4 2 3 5 2 2 3 2 3

Alpha 0.88 0.72 0.77 0.87 0.88 0.89 0.67 0.68 0.75 0.70 0.81

The Cronbach alpha coefficients for most variables are above the recommended 0.70 (Numally, 1978), thereby confirming reliability or internal consistency of the variables used in the present study. Most variables as stated above were adapted from prior studies. This also assisted in ensuring construct and content validity. However variables Portability and Security are slightly below the threshold. This could perhaps be explained by the few items used to measure the variable.

Bronwyn Abrahams and Michael Kyobe Table 3 shows that the overall quality of PACS is considered to be fairly good. The mean results suggest however that many of the responses did not provide strong agreement with the questions asked. Firm agreement was provided in response to scalability, usability and correctness. Further analysis by respondent group indicates that Clinicians and consultants were most uncertain about the quality of PACS. Table 3: Descriptive statistics – all responses Variable Overall Quality of PACS Maintainability Efficiency (in performance) Scalability Cost/Benefit analysis Usability Portability Security, conf. & compliance Robustness Correctness Reliability

Number 52 52 52 52 52 52 52 52 52 52 52

Mean 3.80 3.65 3.86 4.13 3.86 4.05 3.96 3.53 3.38 4.00 3.30

Min 3.00 2.00 2.00 3.00 2.00 3.00 2.00 1.00 1.00 3.00 1.00

Max 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00

Std. Dev 0.52 0.65 0.65 0.48 0.65 0.57 0.86 0.87 0.84 0.56 1.00

Table 4 Analysis of results by respondent’s occupation

Overall quality of PACS: The overall quality was measured by determining the up‐to‐datedness of data; image quality; diagnosis quality; frequency of evaluation of PACS and reporting. These dimensions have been used by many researchers to measure PACS quality (Kalyanpur et al. (2010). Most respondents, (except for the consultants) agreed that patient data was up‐to‐date; the images were of diagnostic quality and that the quality of the diagnosis had improved. They were however divided over the issue of the frequency of PACS evaluation. The reports were also considered by many to meet user needs except for the technologist. Maintainability: Maintainability was measured by determining if PACS software, server and network are easy to maintain. Most of the respondents did not agree firmly that these are maintained efficiently. Only the Registrars and Technologists seem to agree. Efficiency: With regard to Efficiency, most respondents felt the system is fairly efficient since queries are addressed promptly; pertinent image data was displayed in time and less time is wasted on interpreting images. Consultants were the only respondents uncertain about this. Scalability: Scalability was determined by the usefulness of PACS interface with RIS and HIS. Medical officers and radiologists were the only respondents uncertain about the scalability of the system. The rest indicated that the interface with RIS and HIS were useful. Cost effectiveness: Except for the radiographers and consultants, the rest of the respondents perceived the system to be cost effective and enjoyed some benefits. For instance they indicated that efficiency increased with PACS and more studies could be interpreted per session. However, there appears to be some areas where full benefits are yet to be realized. Radiographers, PAC Administrators, medical officers and consultants did not agree that costly examinations were things of the past.

Bronwyn Abrahams and Michael Kyobe Usability: Usability was determined by the perceived user friendliness of the patient data interface system; usefulness of web access; the ease with which images could be saved and the usefulness of the training provided on image display interface. Medical officers and clinicians experience problems with patient data interface and training for patient data interface. In addition, medical officers were not happy with the way images were stored. Portability: Portability was measured by the ease with which data could be accessed from anywhere and anytime. Except for the medical officers, radiologists and consultants who had issues with accessibility of data, the rest of the respondents agreed that PACS is portable. Security, confidentiality and compliance: The quality of security, confidentiality and compliance was determined by the extent to which PACS digital images were kept secure and whether electronic patient records were not tampered with. Radiographers, PACS Admin, clinicians, radiologists and consultants were concerned about the security and confidentiality of images and data. They were not entirely certain that patient records are never tampered with. Robustness: The robustness of the PACS was measured by the resilience of PACS in event of power failure and cable theft and also the robustness of the multi‐media server. Majority of the respondents are not certain that the system is resilient enough. Correctness: Medical officers, clinicians and consultants were not firmly certain that patient information is correct and whether staff use PACS correctly. Reliability: With the exception of the Technologists and clinicians, all other respondents seem to suggest that system crashes do exist while accessing data and images and that there are challenges interfacing with other systems. Interfacing with RIS is considered essential for proper vetting of access requests and compliance with reporting requirements (Saywell, 2010). This therefore suggests lack of PACS reliability to some extent.

8. Regression analysis In order to meet the second objective of this study, we conducted a regression analysis to determine which of the above variables has the greatest influence on the quality of PACS implementation. Table 5: Regression summary for dependent variable: Quality and effectiveness of PACS R = 0.80757 ; R Sq = 0.65218; Adjusted R Sq = 0.42778; df(20,31) F=2.9063; p < 0.003753; Std.Error of estimate: 0.813488

Beta

Std Err of B

T(31)

P=level

0.6881 0.6446

Std Err of Beta 0.30586 0.29172

Intercept Scalability Security, conf. and compliance Reliability Maintainability Efficiency Portability Cost effectiveness Usability Robustness Correctness

7.0040 1.28243 0.75229

1.37117 0.57003 0.340449

5.10903 2.24975 2.20969

0.000016 0.031708 0.034643

0.4366 0.4937 0.39412 0.13693 ‐0.35489 0.344630 0.356107 0.006710

0.15614 0.22728 0.166323 0.330719 0.198963 0.191729 0.200149 0.155619

0.87552 0.79605 0.806740 0.22540 ‐0.58033 0.65229 0.45388 0.01169

0.31304 0.366421 0.340451 0.544369 0.325347 0.332449 0.255100 0.271047

2.79675 2.17250 2.36963 0.41406 ‐1.78373 2.10969 1.77921 0.04312

0.008788 0.037583 0.022719 0.681678 0.084230 0.044643 0.085014 0.965883

Results of the regression analysis (Table 5) suggest that in the South African context, Security, confidentiality and compliance most influence PACS implementation quality. Surprisingly, portability, cost effectiveness, robustness and correctness did not have significant influence on the quality of PACS implementation.

Bronwyn Abrahams and Michael Kyobe

9. Discussion There appear to be mixed feelings about the overall quality of PACS at this academic hospital. Answers to the research questions are outlined below: (Q1)Does PACS support work processes within the Radiology department? While radiologists agree that PACS is a quality system and is efficient, cost effective, portable and usable. They were not certain about its security, maintainability, reliability and robustness. These challenges were also found to be significant influencers of the quality of PACS implementation in South Africa. These issues have also been raised in many other studies today (Kalyanpur et al (2010; van Heerden et al (2011). Most interviewees however considered the workflow to be more efficient and effective. In general we can conclude that PACS supports work processes in the radiology department but its security and maintenance require further attention. PACS are convenient in that they allow for radiology studies to be interpreted in any location where a computer and connectivity can be found without the need for physical films. PACS also allow for increased efficiency in the drafting, sharing, and archiving of radiology reports. However, these benefits are not without potential drawbacks‐‐especially in the legal arena. As attorneys for plaintiffs injured by alleged medical malpractice become more accustomed to EMR across medical disciplines, they are developing strategies for using EMR to their advantage during the course of litigation. Radiologists and radiologic technologists should be aware of how metadata and the copying and pasting of information can be used to their detriment in medical malpractice actions (Q2) Do Clinicians obtain results much sooner? The findings appears to suggest they do since they consider queries to be dealt with rapidly; pertinent image data is displayed in time; images are displayed quickly and less time is wasted interpreting images. They also agree that data is easily accessible from anywhere and anytime. However as one interviewee indicated the quality is limited by the monitors used. This finding is somehow different from that of van Heerden et al (2011) where Clinicians effort was frustrated by lack of adequate hardware, sub‐optimal personal IT proficiency, and system downtime. Tis finding is also confirmed by the regression analysis results which show that reliability of PACS is a significant quality influencer. (Q3)Is the Clinicians’ interpretation accurate? Some interviewees claimed they have not come across any patient miss‐diagnosed, and maintained the images were of good quality and their interpretation was more accurate. However Medical officers, Clinicians, Radiologists and Consultants do not firmly agree that the data was entirely correct although they were more certain that queries are answered quickly and less time is wasted in interpreting images. Therefore answers to this question are mixed and inconclusive. The regression analysis also shows that the influence of correctness on the quality of PACS implementation was not significant. (Q4) Has the use of PACS improved patient outcomes? Some interviewees felt that there are shorter waiting times, shorter hospital stays, less exposure to radiation, and missing folder challenges have been eliminated. In addition, the findings also confirm improvements in handling patient queries; timely display of pertinent data; lesser time in interpreting the images; and improvements in the quality of the diagnosis. These findings reasonably suggest improvements in patient outcomes. Van Heerden et al (2011) also observed benefits from PACS implementation resulting from better patient follow‐up; easy access to previous radiological imaging, etc. Chen, Bradshaw and Nagy (2011) also maintain that user satisfaction and interoperability are major PACS success factors. (Q5) What is the result of the cost‐benefit assessment of PACS? There were noticeable benefits reported by interviewee and survey respondents. For instance, PACS is easy to use; it is perceived to be cost effective and fast and the costs of its utilisation were generally perceived to be

Bronwyn Abrahams and Michael Kyobe dropping. However, cost effectiveness was not found to be a significant influencer of the quality of PACS implementation in this study.

10. Conclusion The present study offers a more comprehensive assessment of PACS in a hospital environment and identifies the key factors influencing PACS implementation in South Africa. The findings show that respondents were general satisfied with the quality of PACS implementation. PACS appears to have improved the radiologists’ work practices, clinicians obtain results much sooner and observed improvements in clinical decision‐making and in patients outcome. However, there are still major issues that require urgent attention. The analysis reveals that ensuring security, compliance, maintainability and reliability are not yet realities. Security and potential legal implications are major concern today and could present costly liabilities for the hospital. Radiologists and technologists should therefore be aware of how to manage data for such information can be used to their detriment in medical malpractice actions. Negative comments about the system, PACS infrastructure and maintenance and the call by respondents to scrutinize the contract between the department and the vendors clearly suggest that vendors need to be more responsive to their client’s needs especially in areas relating to ergonomics, screen size, bug fixing, and licensing, training and systems integration. The paper proposes a conceptual model for evaluating PACS implementation which has been tested in the present study. However, further validation will be necessary as this model was only tested in one hospital and involved only 52 respondents. More case studies in other regional hospitals should be conducted and probably involve more patients in order to compare their views on the success of the system.

References Ammenwerth, E., Graber, S., Herrmann, G., Burkle, T., and Konig, J. (2003). Evaluation of health information systems – problems and challenges, Int. J. Med. Inf., 71, 125‐135. Anderson, D., and Flynn, K. (1997). Technology Assessment ProgramPicture Archiving and Communication Systems: A Systematic Review of Published Studies of Diagnostic Accuracy, Radiology Work Processes, Outcomes of Care, and Cost. 1‐98. Bakos, J.Y., and Kemerer, C.F. (1992). Recent applications of economic theory in Information Technology Research, Decision Support Systems, 8(5): 365‐386. Berghout, E., and Remenyi, D. (2005). The eleven years of the European conference on IT evaluation: Retrospectives and Perspectives for possible future research. The electronic journal of information systems evaluation. 8(2): 81‐98. Chen, J., Bradshaw, J. and Nagy, P. (2011). Has the Picture Archiving and Communication system (PACS) become a commodity? Journal of digital imaging, 24(1), 6‐11. Costello, P., Sloane, A., and Moreton, R. (2007). IT Evaluation Frameworks – Do they make a valuable contribution? A critique of some of the classic models for use by SMEs. Electronic Journal of Information systems Evaluation, 10(1): 57‐64. Hallikainen, P. and Chen, L. (2005). A holistic framework on information systems evaluation with a case analysis. The electronic journal of information systems evaluation. 9(2):57‐64. Marion, C.C. (2008) Kwazulu Natal Anti Corruption Strategy, [Online] Available: http://www.kwazulunatal.gov.za/premier/anti_fraud/Presentation_To_Kzn_Provincial_Anti_Corruption_Summit.pdf [20 August 2010]. Michalson, L. and Hughes, B. (2005) Guide to the ECT Act, [Online] Available : http://www.michalson.com [20 August, 2009]. Klecun, E.,and Cornford, T. (2005). A critical approach to evaluation. European Journal of Information Systems, 14, 229‐243. th doi: 10.1057/palgrave.ejis.3000540 Retrieved on the 20 August 2012 from http://www.palgrave‐ journals.com/ejis/journal/v14/n3/abs/3000540a.html Oosterwijk H. (2004). PACS Fundamentals. OTech Publications; 2004. Herr G. (2004). The PACS initiative: What does it take? Biomed Instrum Technol 2004‐05. Suppl 39‐43. Kalyanpur, A., Singh, J., and Bedi, R. (2010). Practical issues in picture archiving and communication systems and networking. Indian Journal of Radiology Imaging, 20(1), 2–5. Naul L.G., and Sincleair S.T. (2001),Radiology goes filmless. What does this mean for primary care physicians? Postgrad Med, 2001, 109: 107‐110. Faggioni L, Neri E, Castellana C, Caramella D, and Bartolozzi C. (2011). European Journal of Radiology,78(2):253‐8. Khaddaj, S., and Horgan, G. (2004). The evaluation of software quality factors in very large information systems. Electronic Journal of Information Systems Evaluation, 7(1), 43‐48.

Bronwyn Abrahams and Michael Kyobe Pattynama, P. (2009, February). Practice and legal reality of telemedicine / teleradiology [Legally Telemedicine Workshop]. UEMS e‐Health working group. 1‐29. Retrieved on the 27th September 2012 from http://ec.europa.eu/information_society/activities/health/docs/events/2009/ehealth2009/legally‐ tm_eh2009/eH2009‐pattynama‐uems.pdf Philips Healthcare South Africa. (2010). Statement of work: Deployment of RIS at Tygerberg Hospital (Document Version Number 3.0). (Unpublished report). Prabhu, S.P., Gandhi, S., and Goddard, P.R. (2005). Ergonomics of digital imaging. British Journal of Radiology, 78, 582‐586. th doi 10.1259/bjr/51650129 Retrieved on the 27 September 2012 from http://bjr.birjournals.org/content/78/931/582.long Public Finance Management Amendment Act. (1999). Government Gazette: [No. 19978]. Insertion of Chapter 3 in Act 1 of 1999: Provincial Treasuries and Provincial Revenue Funds. 406, 1‐38. Schulze,O.,C, Ackermann,C., Greyling,J., Viljoen,H., and Andronikou,S (2007). Talking PACS: Part 2 ‐ why should we change to PACS? South African Journal of Radiology, 11(4), 86‐90. Serafeimidis, A., and Smithson, S. (2000). Information systems evaluation in practice: a case study of organizational change. Journal of Information Technology, 15:93‐105. Van Heerden, J., Lockhat, Z., Bam, D., Fletcher, J., and Sommerville, J. (2011). PACS: Do Clinical users benefit from it as a training adjunct? South African Jounral of Radiology, 15(2), 38‐41. White, P. (2002). Legal issues in teleradiology – distant thoughts! British Journal of Radiology, 75(891), 201‐206. Retrieved th on the 27 September 2012 from http://bjr.birjournals.org/content/75/891/201.long Williams, T., May, C., Mair, F., Mort, M. and Gask, L. (2003) Normative models of health technology assessment and the social production of evidence about telehealthcare, Health Policy, 64, 39–54. You JJ, Yun L, and Tu JV (2008). Impact of picture archiving communication systems on rates of duplicate imaging: a before‐ after study. BMC Health Serv Res 2008;8:234.

Assessment of Knowledge Sharing Readiness: An Institute of Higher Learning Case Study Nur Syufiza Ahmad Shukor1, Haslinda Sutan Ahmad Nawi1, Suzana Basaruddin1 and Nasrudin Md Rahim2 1 Information Systems Department, Faculty of Computer Science and Information Technology, Universiti Selangor, Bestari Jaya, Malaysia 2 Faculty of Science and Biotechnology, Universiti Selangor, Bestari Jaya, Malaysia nur_syufiza@unisel.edu.my haslindasan@unisel.edu.my suzana_b@unisel.edu.my nasrudin@unisel.edu.my Abstract: In 2002, the Knowledge‐based Economy Master Plan was formulated to provide strategic direction for the nation to transform Malaysia from a Production‐based economy to a knowledge‐based economy. Institutes of Higher Learning (IHLs) play a major role in making the K‐based economy achieving its targets. As an academic centre, IHL is a place where pools of knowledge‐rich human resources are best acquired, codified and disseminated. This study is a survey on the sharing/transfer of knowledge activities in an institute of higher learning. To achieve an in‐depth study, an institute of higher learning was chosen for a case study. The findings are based on replies to a questionnaire survey done from May 2010 to August 2010. The result from 206 respondents showed that the IHL is ready for knowledge sharing; and the knowledge sharing processes that comprise of knowledge capture, organize, refine and transfer activities do exist and have been practiced by the academician. This finding is significant to the university if it choose to embark on the knowledge management initiatives, such as implementing the Knowledge Portal (K‐Portal). Keywords: knowledge sharing readiness, knowledge sharing process, institute of higher learning

1. Introduction Knowledge can be categorized into two major types, tacit that refers to knowledge resides with the expert and explicit that refers to knowledge that had been documented into manual, reports and such (Nonaka & Takeuchi, 1995). According to (Debowski, 2006), knowledge management is the process of identifying, capturing, organizing and disseminating the intellectual assets that are critical to the organization’s long‐term performance. This will enable the organization’s intellectual property (recorded or expert of its members) to turn into a greater productivity, new values and increased competitiveness. It also enables the creation, distribution, and utilization of knowledge to produce and continue greater value from core business competencies through the knowledge capturing; organizing; refining; and transferring process (Awad & Ghaziri, 2004). Pools of knowledge can be easily noted in any institute of higher learning (IHL). The teaching and learning and research activities done at the IHLs are the intellectual properties that help the IHL in achieving their goals. If the knowledge is not managed and catered for, that knowledge will either leave the IHL or left unutilized. Thus it is important for the IHL to manage their knowledge and having a portal for knowledge sharing activities is of the KM initiatives that can be taken by the IHL. This research focuses on the knowledge sharing readiness at one of the private IHL in Malaysia. As opined by many researchers, the knowledge sharing readiness is important for any IHL before embarking into KM initiatives such as knowledge portal (K‐Portal)( Mohayidin et al. (2007), Nassuora & Hassan (2010). It is important for the IHL to assess the knowledge sharing readiness among its staff as the recurring of these listed problem; difficulties in finding resources for teaching and learning especially for new lecturer (e.g.: syllabus, lecture notes, slides, exercises, assignments etc.); difficulties in maintaining physical subject file; and no formal and centralized knowledge sharing platform existed. The main objective of the study is to investigate the readiness of knowledge sharing at a private IHL. The IHL will then be able to assess their academic staff readiness thus it will enable them to embark on the K‐Portal development.

1.1 Knowledge management (KM) initiatives in Malaysian IHLs Knowledge is the process of translating information and past experience including believed and valued into a meaningful set of relationships which are understood and applied by an individual. Academia create countless of knowledge during research and teaching processes which needs to be managed. Thus knowledge is the

Nur Syufiza Ahmad Shukor et al. essence in high learning institution where knowledge is transfered and shared in producing quality graduates. It is point out that, KM would help instituition to capture tacit knowledge and experience of their academia and student that usable by larger community over a a longer period of time (Kumar & Gupta, 2010). Previous research by Ismail et al. (2008) cited a few study on how IHL in Malaysia found the importance to manage their knowledge for future use by the organization. In IHL sectors, collaboration tools of KM System explore the opportunity to create, gather, access, distribute and disseminate the knowledge to their community of practice for many purposes (Rusli et al., 2004). Most IHL in Malaysia have the vision to be the leading university that pioneers innovation, research, and education to achieve the aspiration of producing knowledge society (Juhana et al., 2006). Therefore knowledge must be managed effectively to make sure it is used at optimum level. Staff in university must actively involve and plays their part to make sure knowledge is shared across departments and faculty towards achieving the vision. Butler (2009), in his research before illustrates how public sector organizations can avoid the ‘‘great trap in KM” by focusing on designing IT artifacts to make explicit the tacit knowledge in people’s heads, and not in the information contained in document repositories. IHL now are must no longer just providing knowledge to the students, but should also manage and collaborate the existing knowledge for future reference (Ismail and Chue, 2004). A systematic framework grounded in the sociology of knowledge and based on the view of organizations as social collectives and "knowledge systems" comprised a four "knowledge process": creation, organization, transfer, and application (Holzner & Marx, 1979). While other researchers have their own components about KM processes; capturing, organizing, refining, and transferring (Awad & Ghaziri, 2004). A framework of knowledge management process developed by Zaim et al. (2007), composed into four main processes, which are namely: knowledge generation and development; knowledge codification and storage; knowledge transfer and sharing; and knowledge utilization. Ramachandran et al. (2009), identified six common KM processes and to define these processes in the IHL setting as a systematic process of knowledge creation, capture, organisation, storage, dissemination, and application by the academics of various faculties sharing similar areas of interest in advancing theirs as well as the endeavours of the IHLs. While Alavi et al. (2010) opined that KM process consists of knowledge acquisition, creation, sharing, utilization, storage and refinement. A survey of KM practices proved that the KM processes (capturing, organizing, refining and transferring) does exist among the academicians at one faculty of a private IHL in Malaysia (Shukor et al., 2009). While Norrini et al. (2010), reported four KM processes that composed of knowledge generation, knowledge codification, knowledge sharing and knowledge utilization were emphasized in one of the public IHL in Malaysia. Another study by Nassuora & Hassan (2010) revealed Malaysian academia were convinced that knowledge sharing activities were beneficial to them and the main motivator for them to share their knowledge were the intention to learn from each other.

1.2 Knowledge sharing activities among academia in IHLs Due the fact that almost all of the employees are knowledge workers, sharing of knowledge is very essential in knowledge‐based organizations like IHLs. Knowledge sharing in an educational system ensures that academia is updated timely with the most recent knowledge. Mohayidin et al., (2007) pointed out that IHL are suitable for the application of knowledge sharing by reasons of; a) IHL generally possess new information infrastructure, b) sharing of knowledge with others is natural for academia, c) the willingness of students is to obtain knowledge from available sources as fast as possible, and d) normally a trusting atmosphere at IHL, no neither hesitation or fear for publishing or otherwise disseminate her or his knowledge. According to Nassuora & Hassan (2010), the main stakeholders involved in knowledge sharing activites at IHL can be classified into three groups; academics (tutors, lecturers, assistant lecturers, associate professors and professors), non‐academics (administrators and technicians), and students. These groups of people with different backgrounds, skills, knowledge and experience will collaborate their efforts to fulfill the tasks which will then create a new environment; KM. IHLs is critically associated with the preservation of knowledge and ideas through these processes; teaching, research, publication, extension and services and interpretation (Ratcliffe‐Martin et al., 2000). As a result, knowledge is a must to be promoted as a business in the IHL and should remain as the focus of higher education instituitions.

Nur Syufiza Ahmad Shukor et al.

2. Knowledge sharing readiness Implementing KM projects or knowledge‐sharing philosophies in organizations often requires significant organization change (Davenport and Prusak, 1998). Most leaders are aware that successfully introducing changes, of any kind, are difficult where resistance to change is often dramatic and immediate. Because of this, leaders have been encouraged to proactively prepare their organizations as they begin one of these initiatives. As the first step in this process, managers should comprehensively examine their organization’s underlying readiness to embrace these initiatives. For this research, the readiness for knowledge sharing can be defined as ‘the intention to be involved in the KM process by the academia within the existing IHL context’. Through this assessment, the IHL leaders can identify the gaps that exist between their own beliefs about the KM efforts and the academia’s’ beliefs. In essence, an assessment of an organization’s readiness could serve as a guide to these leaders as they plan and implement KM initiatives for a future of respectful IHL. In 2009, it was reported that although the majority of respondents in both the types of IHLs (public and private) demonstrate some knowledge of KM and that those from the private IHLs show higher awareness of a formal KM programme in their institutions, the KM processes are only moderately practiced by both types of IHLs (Ramachandran et al., 2009),. Furthermore, Mohayidin et al., (2007) pointed out Malaysian IHLs were not utilizing knowledge to the fullest to improve their performance and this is because the data, information and knowledge available in the IHLs were not properly managed and such they could be efficiently shared and reused to generate new knowledge. This again implies a need to assess knowledge sharing readiness in order to guarantee it smooth implementation. It is suggested that KM readiness is influenced by people and process (Cho et al., 2000); culture (O'Dell et al., 1998); and content (Havens and Knapp, 1999). This is translated into: who is involved; how KM is occurring; where KM is occurring and what is involved. In all, this suggested that a measurement of KM readiness should assess individual attributes, the internal context, the KM initiative content, and the process used to diffuse KM. While Holt et al. (2004), suggested to assess readiness from one of these four perspectives; (a) the individual, (b) the organizational culture and climate, (c) the specific change, and (d) the process. The whole questions in the study were catered all the four perspective but this paper will only report and l focus on the readiness of the KM process among the academicians at the IHL.

3. Methodology The main objective of the study is to explore the existence of KM processes at one private institute of higher learning that consists of capturing, organizing, refining and transferring. The other objectives are to investigate the awareness and readiness of KM implementation in the IHL. The questions were constructed to reflect the practice of KM processes in the IHL. There are total of 37 questions to cover all the six hypothesis in this study. Six sections are namely A, B, C, D, E and F. Sections are : Data will be gathered through 5 point Likert scale questionnaires. Pilot test was done in April 2010. Respondent in pilot test is selected randomly from three faculties namely Information Technology, Biotechnology & Life Sciences & UGSM/CGS. Total of thirteen survey question returned in the test. The result from respondent help researcher adjusts the questions for readability and understanding. Cronbach Alpha for pilot test set of questions is .82. Real data collection has improvement on the questions as the result of Cronbach Alpha is .85. Data for this study was collected through questionnaires distributed to all academicians. The population of academicians on May ‐ August 2010 was 399. Survey was distributed to respondents batch by batch, faculty by faculty. Returned questionnaire is 206 from ten faculties (refer Figure 1 for respondents’ background). Analysis done using interval, mean was determined by the research and for both objectives, the independent t‐Test will be used to examine the existence of the KM processes. Data collected are analyzed to test the hypotheses. There were six hypotheses to reflect the study. If the majority of the respondents regard these processes to be of particular importance, then it would confirm their validity. The hypotheses are categorized in terms of hypothesis sets. The following are the sets of hypotheses: H1 0 : Knowledge capturing process does not exist. H20 : Knowledge organizing process does not exist. H30 : Knowledge refining process does not exist.

Nur Syufiza Ahmad Shukor et al. H40 : Knowledge transferring process does not exist. H50 : Knowledge sharing willingness does not exist. H60 : Management support on knowledge management does not exist.

Figure 1: Background of respondents by faculty For statistical purpose, this is translated into: H0 is rejected if p‐value ≥ 0.05 while H0 is failed to be rejected if p‐value < 0.05. Test value = 1, meaning there is existence if value >3. These hypotheses are tested using t‐Test to get the pvalue. t‐Test is used in this study because:

the sample size is small

the population standard deviation is unknown

4. Analysis result and discussion 4.1 Respondents profile A total number of 206 respondents participated in this study. Majority of the respondents (72%/149) were master degree holders, where by the rest were bachelor degree holders (18%/38), PhD holders (8%/17) and one (1%) respondent was a higher diploma holder. Most of the respondents had 5 to 10 years of teaching experience with 39%/81 respondents. Also high percentage of 35%/71 of the respondents had between 2‐4 years of teaching experience as compared to 20%/41 of the respondents who had more than 10 years of teaching experience. Only 6%/13 respondents had 1 year or less of teaching experience. Testing the reliability of the questions that used Likert Scale shows that data from this study is highly reliable with Cronbach’s alpha value of 0.85 as depicted in Table 1. In general, reliabilities less than 0.60 are considered to be poor, those in the 0.70 range, acceptable, and those over 0.80 are considered good. Therefore we can conclude that the data collected in this study is reliable. Table 2: Reliability test Cronbach’s Alpha

N of Items

.850

4.2 Knowledge sharing readiness knowledge acquisition method A total of 179(87%) respondents got their soft skills knowledge through formal learning such hands‐on training, seminar and workshop. Besides through formal learning they also gathered their soft skills knowledge through self discovery or working experienced (175/85%). This follows by using documented material (170/82%) such as books, manual journals and video streaming. The least method of obtaining such soft skills

Nur Syufiza Ahmad Shukor et al. knowledge was through personal coaching where 122(59%) respondents indicated this method. Meanwhile, majority of the respondents (170/85%) believed that written knowledge could be obtained through formal learning such as hands‐on training, seminar and workshop; follows by through documented material such as manual, books, journals and video streaming (170/85%); and self discovery (145/70%). Personal coaching (106/52%) on the other hand were not the common methods of obtaining written knowledge by the respondents.

4.3 Knowledge repository type Due to the lack of central repository availability, majority of the respondents preferred to keep their knowledge through local repository system. A total of 172(84%) respondents kept their knowledge through digital files in local repository but 155(75%) respondents also keep it via printed files in local repository. Only 77(37%) respondents kept their knowledge through either digital files in central repository or printed files in central repository which was made available for others to share.

4.4 Knowledge type kept in the repository Respondents did keep their lecture notes (180/87%), assignments (161/78%), tests (159/77%) and quizzes (153/74%) in their knowledge repositories. Some respondents also kept other items such as final exam, articles from website/journal/text book and lab reports in their repositories.

4.5 Knowledge sharing method Since the online facilities availability is limited, majority of the respondents only shared their knowledge manually (188/91%) such as photocopying of notes. A number of respondents managed to share their knowledge through online facilities such as e‐groups (68/33%), personal website (56/27%) or e‐learning tools (20/10%). Some respondents (20/10%) shared their knowledge through other methods such as email or giving softcopy to students.

4.6 Awareness to knowledge sharing On average, respondents agreed that the knowledge sharing processes in terms of knowledge capturing, organizing, refining and transferring exist in the IHL. The highest level of existence were knowledge organizing and transferring with a mean score of 4.3 respectively. This follows by knowledge capturing with 4.1 mean score and knowledge refining with 3.8 mean score. To test whether there is significance existence of the knowledge sharing processes awareness, the following hypotheses were conducted, H10 : Knowledge capturing process does not exist. H20 : Knowledge organizing process does not exist. H30 : Knowledge refining process does not exist. H40 : Knowledge transferring process does not exist. (Test value ≤3, meaning there is existence if value >3). Using one sample t‐test on the above hypotheses at significance value .05 concluded that each process does exist. This is because all processes gathered p‐value of <.001 which is less than .05 therefore all H0 are rejected. Table 2: t‐Test on knowledge sharing awareness (Likert scale used = 1 to 5) Knowledge Sharing Awareness

Mean

Std. Deviation

t‐statistic

p‐value

Knowledge Capture

205

4.1

0.5

30.28

<.001

Knowledge Organize

205

4.3

0.6

31.01

<.001

Knowledge Refine

204

3.8

0.6

18.74

<.001

Knowledge Transfer

206

4.3

0.6

34.32

<.001

Nur Syufiza Ahmad Shukor et al.

4.7 Willingness to knowledge sharing Respondents agreed that they were willing to share their knowledge with others with a mean score of 4.1 and a standard deviation of 0.6. The following hypothesis is used to test the significance existence of knowledge sharing willingness, H50 : Knowledge sharing willingness does not exist. (Test value ≤3, meaning there is existence if value >3). Result from one sample t‐test indicates that hypothesis 5 is rejected at significance value .05. Since p‐value is <.001 which is less than .05, we can conclude that there is a significance existence of knowledge sharing. Table 3: t‐Test on Knowledge Sharing Willingness (Likert scale used = 1 to 5) Knowledge Sharing Willingness

Mean

Std. Deviation

t‐statistic

p‐value

Knowledge Sharing

206

4.1

0.6

25.28

<.001

5. Conclusions This report represented a fragment of a bigger research on knowledge sharing process at IHL. There are four perspectives in assessing the readiness of organization in knowledge sharing namely individual; organizational culture and climate; specific change, and process. The analysis and results from the respondents showed that the IHL is ready for knowledge sharing; and the knowledge sharing processes that comprise of knowledge capture, organize, refine and transfer activities do exist and have been practiced by the academician. The academicians are willing to share their knowledge and this is supported by the management. By looking at the process perspective alone, evidently they are ready for knowledge sharing activities. It is significant to the university to measure all the four perspectives, before embarking on any knowledge management initiatives, such as implementing the Knowledge Portal (K‐Portal).The next step is to analyse on the other perspectives in measuring the readiness of knowledge sharing.

Acknowledgements The research team would like to express their sincerely thank to Centre for Research & Innovation, Universiti Selangor (Project No.: G‐UNISEL/ST‐0005) for the financial support of the research work.

References Abdullah, R., Shahabudin, S., Alias, R.A. and Selamat, M.H. (2004) “The Development of Knowledge Management System for Public Higher Learning in Collaborative Environment Using Lotus Notes Software”, Knowledge Management International Conference and Exhibition, Penang, Malaysia, February. Alavi, S. Abd. Wahab, Z. and Muhamad, N. (2010) “Exploring the relation between Organizational Learning and Knowledge Management for Improving Performance”, 2010 International conference on information retrieval and knowledge management (CAMP10), Shah Alam, Malaysia, March, pp 296‐299. Awad, E.M., Ghaziri, H.M.(2004) Knowledge Management, Prentice Hall, New Jersey. Butler T. (2009) “Designing a core IT artifact for Knowledge Management System using participatory action research in a government and a non‐government organization”, Journal of Strategic Information Systems, Vol 17 No.4, pp 249‐267. Cho,G., Jerrell,H. and Landay, W. (2000) Program Management 2000: Know the Way How Knowledge Management Can Improve DoD Acquisition, Defense Systems Management College, Fort Belvoir, January. Davenport, T. H. and Prusak, L.(1998) Working Knowledge. Boston: Harvard Business School Press,. Debowski, S. (2006) Knowledge Management, John Wiley & Sons, Milton Old 4064. Havens, C. and Knapp, E.(1999) "Easing Into Knowledge Management," Strategy and Leadership, Vol. 27, pp. 4‐9. Holt, D.T., Bartezak, H. E., Clark, S. W. and Trent, M. R. (2004) “The Development of an Instrument to Measure Readiness for Knowledge Managemen”, Proceedings of the 37th Hawaii International Conference on System Sciences. Holzner, B. and Marx, J. (1979).The Knowledge Application: The Knowledge System in Society, Aliyn‐Bacon, Boston. Ismail, M. A., Yaacob, M. and Abdul Kareem, S. (2008) “Institutional Repositories: Review and Knowledge Management Perspective”, International Conference on Knowledge Management 2008, Langkawi, Malaysia, June. Ismail, M. A. and Chue, L.Y.(2004) “Implication of Knowledge Management in Higher Learning Institution”, Faculty of Computer Science and Information Technology. Kumar, M. and Gupta, A. (2010) “Knowledge Management in Academia A Case Study”, IIT Delhi. 5th Knowledge Management International Conference, Delhi, India. Mohayidin, M.G., Azirawani, N., Kamaruddin, M.N. and IdawatiMargono, M. (2007) “The Application of Knowledge Management in Enhancing the Performance of Malaysian Universities”, The Electronic Journal of Knowledge Management, Vol 5, No.3, 301‐312.

Nur Syufiza Ahmad Shukor et al. Nassuora, A. B. and Hassan, S. (2010) ”Knowledge Sharing among Academics in Instituitions of Higher Learning”, 5th Knowledge Management International Conference, Terengganu, Malaysia, May. Nonaka, I., Takeuchi, H. (1995). The Knowledge Creating Company – How The Japanese Companies Create the Dynamic Innovations, Oxford Univ. Press. Norrini, M., Baharom, A.R., and Yau’Mee Hayati, M.Y. (2010) “The Relationship between Knowledge Management Practices (KMP) and Academic Performance: A Case Study in Universiti Teknologi MARA (UiTM) Terengganu”, 5th Knowledge Management International Conference, Terengganu, Malaysia, May. O'Dell,C., Grayson Jr.,C. J. and Essaides,.N. (1998) If We Only Knew What We Know. New York, The Free Press. Ramachandran, S. D., Siong C. C. and Ismail H. (2009) ”The practice of knowledge management processes: A comparative study of public and private higher education institutions in Malaysia”, VINE, Vol 39, No.3, pp. 203‐222. Ratcliffe‐Martin, V., Coakes, E., & Sugden, G. (2000) ”Enhancing Knowledge Acquisition and Transfer in the University Sector”, Reviewed Jan. 20, 2011, from http://users.wmin.ac.uk/~coakese/knowledge/bit2000.htm Salim, J., Yahya, Y., Othman, M. S. and Rashid, N.R.M. (2006) “The use of Holistic Approach to Knowledge Management Initiative in Managing Information in Higher Learning Institution : A Perspective”, 6th WSEAS International Conference on Education and Educational Technology, Italy. Shukor, N.S.A., Nawi, H.S.A., Basaruddin, S. and Rahim, N. M. (2009) “Investigation of Knowledge Management Processes Among Academicians at Faculty of Industrial Information Technology, UNISEL: A Case Study”, International Conference of Information Management & Engineering, Kuala Lumpur, Malaysia, February, pp.732 – 735. Zaim, H., Tatoglu, E., & Zaim, S. (2007) “Performance of Knowledge Management Practices: A Causal Analysis”, Journal of Knowledge Management, Vol 11, No.6, 54 – 67.

Ishikawa Diagram of Critical Factors for Information Technology Investment Success: A Conceptual Model Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad Universiti Teknologi MARA, Selangor, Malaysia fauziaha@tmsk.uitm.edu.my aryati@tmsk.uitm.edu.my habibah@tmsk.uitm.edu.my Abstract: The reliance on information systems, Internet, and networking technology and infrastructure has made Information technology (IT) become the fundamental elements in many areas. That’s why organizations that pursue success are highly anticipated to invest tremendously in attaining flawless IT implementations. In ensuring the IT spending made a wise decision, necessary steps were taken to perform vital evaluations of the IT investments beforehand. Varieties of evaluations available, with different approaches adapted to different style of organizational preferences. Nonetheless, the essential part of it is the expectations of returns or values. Once the investment has been made, the achievement of the returns and values indicates the successfulness of the IT investment spending. But not all cases of IT investments end with success stories. Many have ended contrary to what organizations have expected. Lesson learned, studies have recommended to couple the evaluation aspects with enablers of IT investment success or success factors. This paper reviews the current literature regarding the IT investment evaluation approaches and proposes an Ishikawa diagram that describes the factors that influence the success of the IT investment. In this paper, six factors of IT investments enablers are discussed which are Financial, Internal, External, Strategic, Technological, and Risk.

Keywords: Information technology; investment; success factors; enablers; evaluation

1. Introduction The reliance on information systems, Internet, and networking technology and infrastructure has made Information technology (IT) become the fundamental elements in many areas. It seems that all individuals, organizations, and governmental bodies are heavily dependent on the IT in accomplishing processes and activities (Gunasekaran, Love, Rahimi, & Miele, 2001). That’s why organizations that pursue success are highly anticipated to invest tremendously in attaining flawless IT implementations. Usually the investments are enormous with the expectation of huge returns. Organizations, be it private or public, are making huge spending on IT. In USA, the investment in IT has reached 4% of organization’s revenue and taken up almost half of the corporate America capital budget (Mccormack, 2000). European government IT spending was to increase from US$110 billion to US$119 from 2005 to 2007 as predicted (Cresswell, Burke, & Pardo, 2006) . Within the same period, spending on IT by American government has also increased from US$144 billion to US$174 billion (US Census, 2011). Asia Pasific region, excluding Japan, has spent US$22.7 billion in 2006 alone in attaining IT (Cresswell et al., 2006). In ensuring the IT spending made a wise decision, necessary steps were taken to perform vital evaluations of the IT investments beforehand. Varieties of evaluations available, with different approaches adapted to different style of organizational preferences. Nonetheless, the fundamental of all the evaluations were based on the expectations of returns or values, either returns to the organizations (Davern & Kauffman, 2000; Dehning, Richardson, & Stratopoulos, 2005) or returns to the public (Cresswell, 2006; Lech, 2007; Mooney, Gurbaxani, & Kraemer, 2001). Once the investment has been made, the achievement of the returns and values indicates the successfulness of the IT investment spending. But not all cases of IT investments end with success stories. Many have ended contrary to what organizations have expected. Lesson learned, studies have recommended to couple the evaluation aspects with determinants of IT investment success or success factors (Gunasekaran, Ngai, & McGaughey, 2006; Roztocki & Weistroffer, 2011). Using an Ishikawa diagram a causal relationship to show the cause and effect of IT investment success based on a conceptual model is developed. The discussion of the elements was based on the common ground shared among several models of IT evaluation models. The review has resulted in a proposed IT investment success factors model.

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad

2. Review of literature Despite the anticipations of the benefits and returns, some statistics have shown some unfavorable success figures. The daunting figures seems to suggest that some investments made on IT technology or projects since was not promising and the measure of returns cannot be justified with the sum of the IT investment made (Arshad, Idrus, & Ahmad, 2012). There were many reasons to explain the deficiency of project success rate. Many factors can become the contributors to the failure such as ineffective project management and governance, technology failures, funding discontinuity, unrealistic political demand, cost and schedule overrun (Ahmad & Othman, 2006; Frisk, 2009; Sharif, Irani, & Weerakkoddy, 2010), and etc. Somehow, bad project or investment can be avoided if organization performed evaluations in determining investment or project worth (Arshad et al., 2012; Sharif et al., 2010). Commenced from the need of a structured evaluation and justification, many proposed approaches varied from financials to other method of value calculations. Financial approaches of measuring ICT investments are the most quantitatively relied and prioritized (Chambers, 2007; Joshi & Pant, 2008) such as Return on Investment, Net Present Value, Break-Even Analysis, Payback Period, and Internal Rate of Return. However, these traditional measures are argued to be weak in truly measuring the potential of IT. Many researchers have criticized on the sole use of traditional measure of measuring IT investment as deemed inappropriate (Bannister, 2005; Clemons & Weber, 1990; Irani & Love, 2001; Sharif et al., 2010) and many have suggested the combination of financial and nonfinancial measures. Financial measure alone is no longer sufficient since the nature of information technology has evolved from operational to transformational. The role of IT in the organization is no more merely increase the efficiency of business operation, but it has made it way to assisting the CEOs strategic decision making and has become the agent of change in organization. There are many approaches that use non-financial measure on top of financial measures proven effective such as Balanced Scorecard (BSC) (Arshad et al., 2012; Strasser, 2004), Information economics (Joshi & Pant, 2008; Martinsons, Davison, & Tse, 1999), and Multi-Criteria Decision Making approaches (Chou, Chou, & Tzeng, 2006; Gunasekaran et al., 2006). Balanced scorecard approach evaluates IT investment benefits by supplementing financial data with goalrelated measures. It incorporates four elements of assessment of IT investments which are financial, user responses, internal processes, and learning and growth. Similarly, Information economics (or some called it Enhanced Return on investment) also measures the IT investments values using financial criteria, but covering also business and technological criteria. Multi-Criteria approach is another approach that combined financial with operational and strategic aspects of IT investment in the evaluation. It seems that financial aspect alone is insufficient in evaluating IT investment, but the evaluation becomes incomplete if it is to be abandoned. Therefore, most evaluations methods developed to include financial elements and other variables or factors deemed appropriate to suit the nature of the organization, the technology, or the purpose of the investment itself. Nonetheless, the variety of measurements available sometimes does not provide true measure of the investment being made. Some investments collapse after implementation and some are being withdrawn halfway through. Therefore, before commencing any project, a full consideration of critical success factors of an organization need to be identified and addressed (Gunasekaran et al., 2001). Then only the identifications of benefits and values perceived from the IT investment worth valuing. Abundance of research have been discussing the benefits and value assessment of IT investment in different perspectives and concepts. This paper is not pursuing the discussion of those, but deliberating the determinants or enablers that will determine the success of an IT investment. Few studies have suggested that in evaluating IT investment there are few considerations that organization need to pay attention. The following paragraphs will discuss the determinants as found in the literature. These determinants are the enablers or the success factors to ensure the IT investment initiatives are fruitful spending. Clemons and Weber (1990) wrote a paper that describe the IT investment decision problem and proposed principles for evaluating and selecting the IT investment. In their paper, they urged the executives of organizations to consider some important aspects in the evaluation of IT investment. Similar to Clemons and Weber, Gunasekaran et al. (2001), extensively, proposed 30 factors under four main categories of

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad considerations which are strategic, tactical, operational, and tangible considerations. These considerations were put forward in justifying the IT investment to be made by organizations. Chou et al. (2006), has also identified 17 criteria for IT investment evaluation which grouped into four main categories; External, Internal, Risk, and Cost criteria. A few studies, however, addressed the determinants through risk definition. Wu and Ong (2008) claimed that IT investments involved some uncertainties that must be carefully managed. The identified external and internal uncertainties explained several risk items that argued to be important addressed when evaluation on IT investment is performed. Similarly, Azadeh et al. (2009) used several criteria in evaluating the IT investment. Two of the criteria, costs and risks, defined the determinants of IT investment. Some studies use different platform of introducing success factors of IT investment. Dameri (2005) used the concept of balanced scorecard elements to describe the determinants of IT investment success. Dameri proposed several objectives to measures based on the four perspectives of balanced scorecard which are Stakeholder, Action/Services, Processes, and Resources criteria. On the other hand, Braaksma et al. (2006) introduced several value drivers that influence the benefits and burdens of the intended IT investment. Hussein et al. (2007) study was more focused and proposed six organizational factors that they believed to be significant in influencing the success of IT investment. Another different approach is Sharif et al. (2010) study that has elaborated in lengthy all factors that can influence the decision making in IT investment. These factors are classified under four themes which are Decision making, Practitioner concerns, Evaluation methods, and Performance assessment. These nine studies become the source of reference for the conceptual model of the IT success factors. The conceptual model is built to explain the relationship between the organizations’ decisive factors with the success of the intended IT investment. Ishikawa diagram is chosen in illustrating the relationship. Next section will further discuss on the conceptual model.

3. Methodology In developing the model that describe the important success factors of IT investment, this study has performed analysis of nine pertinent studies that deliberately construct elements that explain the determinants of IT investment success. These studies are chosen since they propose models to evaluate investments and the subject of evaluation is IT. Every variable or metric presented by the studies are compared and pooled together in terms of terminology. Similarities between identified variables are grouped together. Some metrics that share similar terminology but using different names are combined and an appropriate new name that distinguishes those metrics is identified and given. For example, referring to Table 1, Effectiveness in resource allocation metric as named by Dameri (2005) also represents resource allocation metric identified by Hussein et al. (2007). Once all the identified metric are grouped, a proper categorical name to identify the grouped factor is given based on the description of the group factor. For example two metrics, Revenue and Budget, fall under the same group. Based on both metrics’ attributes, grouped factor is defined as “the economic aspect of IT investment including projected profitability and anticipation of cost incurred.” Undeniably, the naming of the grouped factors somehow is influenced by the categorical names used by previous studies such as Gunasekaran et al. (2001), Chou et al. (2006), and etc. Therefore, based on classifications of the metrics, six factors are formed to explain the determinants of IT investment success. These six factors are Financial, Internal, External, Strategic, Technological, and Risk. Next section further discusses the model. IT investment success factors model using fishbone diagram Based on the literature review, a conceptual model to describe IT investment success factors has been developed. Figure 1 below illustrates the six factors. Table 1 (on the next page) will further summarize each of the factors’ conceptual definitions, important metrics to describe the IT investment factors, and the literature sources that presented the metrics. Ishikawa diagram is being chosen to represent the relationship of the

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad identified factors towards the success of IT investment. Though this diagram has been used extensively in the management area, the diagram is also useful in the planning and execution of science and technology diagram (Li & Lee, 2011).

Figure 1: Ishikawa diagram of IT investment success factors

3.1 Financial Every spending requires resources of fund. IT investment made by organizations involved huge amount of spending. The decision for such massive actions usually requires budgeting and serious justification and measure of returns. Researchers agree that sufficient financial resources are important in determining success of IT investment (Bowns, Rotherham, & Paisley, 1999; Milis & Mercken, 2002; Walter, 2010). It is important to ensure that the sum of money invested will yield an excess of the cost of the investment (Frisk, 2011; Gunasekaran et al., 2001). Financially, the most important aspects of IT investment that need to be scrutinized before decision to invest is made are on the justifications of revenue stream generated and the budgeting of the investment. The promise of income flow directly or indirectly from the IT investment will have a higher weight and a sufficient allocation of budget also need to be put into consideration.

3.2 Technological Researchers believe that existing IT infrastructure should be in place before additional IT supports can be introduced (Hamaker, 2009; Peppard, Ward, & Daniel, 2007). The facilitation of the current IT will help to boost the effectiveness of the future IT implementation. Therefore, organization needs to address current technological issues that can impede the development of the intended IT projects. It is to ensure the investment of the project is assured. For example, an integrated technology environment in an organization is highly important considerations in providing business improvement initiatives (Hussein et al., 2007). Among other things, organization needs to ensure that there is no issue of compatibility among existing IT and systems with the anticipated IT investment. Besides that, the decision for IT investment should also consider the maturity and requirements of the new technology to match with existing ones.

3.3 Strategic Decisions in IT investment should be designed to impact favorably the key factors of an organization. Organization need to strategically assess how the intended IT investment will support the overall aims and interests of the organizations and provides competitive advantage (Clemons & Weber, 1990). In doing so, the flexibility and support of the management is crucial leading to successful implementation of IT investment (Gunasekaran et al., 2001; Sharif et al., 2010). Therefore, organization needs to assess (i) the strategic impact of the IT investment in terms of competitive advantage that position the organization in the market, and (ii) the alignment of the IT investment objectives with the overall mission and vision of the organization. Besides that, the positivity of the existing top

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad management supports of IT investments and optimistic management style in upheld the IT investment in the organization are also important criteria in IT investment decision making. Table 1: The Conceptual Definitions and Metrics of the Success Factors of IT Investment Factors Financial

Internal

Conceptual definition The economic aspect, including projected profitability and anticipation of cost incurred Organizational resources that include human, assets, infrastructure that influence the organization's behavior and decision making

Metrics Revenue Budget Effectiveness in resource allocation Organizational restructuring Quality of execution Business efficiency

External

Strategic

The requirement of the environment, including the citizen, the private sector, the benchmark of other countries

Overall aims and interest of organizations in achieving corporate missions and visions

Timing value Customer Focus Ally with partner Law enforcement User perception Replicability and competitor response Competitive advantage Strategic/goal alignment Top management support Management style

Technological

The effective management of technology by the organization.

Risk

The exposures to the organization in terms of business risks and technology risks

Compatibility with existing IT portfolio Maturity of new technology Technical/System requirement Market environment risk Organizational risk Project risk

Literature Sources (Gunasekaran et al., 2001; Wu & Ong, 2008) (Ibid) (Dameri; 2005; Hussein et al, 2007) (Chou et al., 2006; Clemons & Weber, 1990) (Braaksma, Commandeur, & Berghout, 2006; Dameri, 2005; Gunasekaran et al., 2001) (Azadeh et al., 2009; Braaksma et al., 2006; Dameri, 2005) (Clemons & Weber, 1990; Sharif et al., 2010; Wu & Ong, 2008) (Azadeh et al., 2009; Braaksma et al., 2006; Dameri, 2005) (Braaksma et al., 2006; Chou, Chou, & Tzeng, 2006) (ibid) (Dameri, 2005; Gunasekaran et al., 2001) (Chou et al., 2006; Clemons & Weber, 1990; Wu & Ong, 2008) (Azadeh et al., 2009; Gunasekaran et al., 2001) (Azadeh et al., 2009; Gunasekaran et al., 2001; Hussein, Karim, & Selamat, 2007) (Clemons & Weber, 1990; Gunasekaran et al., 2001; Hussein et al., 2007; Wu & Ong, 2008) (Clemons & Weber, 1990; Hussein et al., 2007; Sharif et al., 2010) (Chou et al., 2006; Gunasekaran et al., 2001; Sharif et al., 2010) (Braaksma et al., 2006; Chou et al., 2006; Wu & Ong, 2008) (Wu & Ong, 2008) (Clemons & Weber, 1990; Wu & Ong, 2008) (Azadeh et al., 2009; Chen, Zhang, & Lai, 2009) (Azadeh et al., 2009; Chen et al., 2009; Chou et al., 2006; Sharif et al., 2010; Stewart & Mohamed, 2002)

3.4 Risk There are always possibilities of downsides involving decision-making. Likewise, in making IT investment successful, exposure towards uncertainties is likely to demote the decision. Risks are exposure of uncertainties and usually positively increase with the potential business value (Stewart & Mohamed, 2002). As potential business value of an IT investment gets higher, the associated risk to the project would also increase. Organization need to be able to identify the associative risks of the IT investment, in order to possibly contain the expected returns with possibly minimum expected risks (ibid).

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad

3.5 External Environmental factors such as customers, legal authority, allies and competitors should well be addressed and examined before anticipation of IT investment takes place (Clemons & Weber, 1990; Dameri, 2005). The requirements and expectations of those parties will determine how the organization will be able to exploit the existing market (Dameri, 2005; Gunasekaran et al., 2001). The anticipation of enhancements from the IT investment should give indications to the organization how it will reach new segments of demand, and growing the business in ways of unforeseeable at the time of the investment decision (Clemons & Weber, 1990). Organizations need to pay attention to customers, partners, legal authorities, and competitors in making IT investment decision. Some requirements from the customers and legal authorities need to be reflected on the decision made. The impact of the IT investment made to the competitors need also be analyzed hence sustainability of advantage would still be on the organization side instead of the opponents.

3.6 Internal Organizational resources that include human, assets, and infrastructure that influence the organization's behavior and decision making are believed to be substantial in determining successful implementation of IT investment(Irani & Love, 2001). Many of the operational activities of organizations depend on the use of IT. Vise versa, the significant usage of the IT also depend on how it can provide functionality for the business operations. These factors are internal and controllable and if any benefits are to be realized, understanding of organizational or internal impact and change is required (Frisk, 2009). Among the important criteria of internal factors that addressed by literature includes the effectiveness of organizational resources allocation such as manpower, assets and skills. Also, many literature have emphasized on the current capacity and efficiency of the organization to handle the anticipated IT investment as critical as well. Assessment of potential threat from the environment, internal exposures, and project vulnerabilities need to be scrutinized as to ensure lowest possibility of risks occurred.

4. Conclusion In making IT investment decisions, identified benefits and values alone are not sufficient at deriving a conclusive assessment. Some literatures neglect the other important component of IT investment evaluation, which is the determinants or success factors of IT investments and limit the discussions to only cover the benefits and values achieved once IT investment have been implemented. The measure of important criteria that determine the successfulness of the IT investment is equally important, and inclusion of these success factors (financial, internal, external, strategic, technological and risk) into the evaluation part of IT investment will guarantee a thorough assessment. Thus, this conceptual model provides a better insight in IT investment decision making. However, this conceptual model is not conclusive yet. Further research need to be undertaken to expand the conceptual model to include the assessment of benefit/value delivery that the intended IT investment will offer to the organization. This part of assessment also needs critical review of existing models from the literature before the model can be built. Combination of these two parts of assessment, determinants and value delivery, will provide a comprehensive conceptual model that can be further utilized to develop an evaluation methodology of an IT investment.

5. Limitation of research In pursuing this research several difficulties are encountered. Although they did not affect the outcome of the research somehow this study may be well executed without them. One example of difficulty that we found most inconvenience when performing comparisons of metrics was the unavailability of metricsâ&#x20AC;&#x2122; definitions by some of the authors of the models that we were analyzing. We resolved the problem by putting the undefined

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad metric into context (as the author described in the paper), and reassigned the definition as close as to other common authors’ definition. Apart from that, we also found that limited research availability on this area has posed some restrictions to our study. That’s why many articles that we have gone through seemed to include literature past decades ago. It is common in the area of IT investment evaluation to contain literature as early as 1990s being cited and discussed with recent ones.

6. Acknowledgment Researchers of this study would like to extend the gratitude to the Universiti Teknologi MARA for the establishment of Research Intensive Faculty (RIF) Grant Scheme of which this study has benefited.

References Arshad, N. H., Idrus, H. M., & Ahmad, F. (2012). A Balanced Scorecard Approach in Evaluating IT projects in the Public Sector. 2012 IEEE Business, Engineering & Industrial Applications Colloquium (BEIAC), 162–166. doi:10.1109/BEIAC.2012.6226043 Azadeh, A., Keramati, A., & Jafary Songhori, M. (2009). An integrated Delphi/VAHP/DEA framework for evaluation of information technology/information system (IT/IS) investments. The International Journal of Advanced Manufacturing Technology, 45, 1233–1251. doi:10.1007/s00170-009-2047-2 Bannister, F. (2005). When Paradigms Shift : IT Evaluation in a Brave New World. Electronic Journal of Information System Evaluation, 8(1), 21–30. Bowns, I. R., Rotherham, G., & Paisley, S. (1999). Factors associated with success in the implementation of information management and technology in the NHS. Health Informatics Journal, 5(3), 136–145. doi:10.1177/146045829900500305 Braaksma, J., Commandeur, A., & Berghout, E. W. (2006). The Business Case for ICT Investment Evaluation in Nonprofit Organisations. Sprouts: Working Papers in Information Systems2. Retrieved from http://sprouts.aisnet.org/6-60 Chambers, M. (2007). Book Reviews - The Effective Measurement and Management of ICT Costs & Benefits. Credit Control ; 28, 3, 68–70. Chen, T., Zhang, J., & Lai, K.-K. (2009). An integrated real options evaluating model for information technology projects under multiple risks. International Journal of Project Management, 27, 776–786. doi:10.1016/j.ijproman.2009.01.001 Chou, T.-Y., Chou, S. T., & Tzeng, G.-H. (2006). Evaluating IT/IS investments: A fuzzy multi-criteria decision model approach. European Journal of Operational Research, 173(3), 1026–1046. doi:10.1016/j.ejor.2005.07.003 Clemons, E. K., & Weber, B. W. (1990). Strategic Information Technology Investments : Guidelines for Decision Making *. Journal of Management Information Systems, 7(2), 9–28. Cresswell, A. M. (2006). The Role of Public Return on Investment Assessment in Government IT Projects. Proceedings of the 2006 national conference on Digital government research - dg.o ’06, 2, 379 – 380. doi:10.1145/1146598.1146711 Cresswell, A. M., Burke, G. B., & Pardo, T. A. (2006). Advancing Return on Investment Analysis for Government IT A Public Value Framework (p. 50). Dameri, R. P. (2005). Using the Balanced Scorecard to Evaluate ICT Investments in Non profit Organisations. The Electronic Journal of Information Systems Evaluation, 8(2), 107–114. Davern, M. J., & Kauffman, R. J. (2000). Discovering Potential and Realizing Value from Information Technology Investments. Journal of Management Information Systems, 16(4), 121–143. Dehning, B., Richardson, V. J., & Stratopoulos, T. (2005). Information technology investments and firm value. Information & Management, 42(7), 989–1008. doi:10.1016/j.im.2004.11.003 Frisk, E. J. (2009). From Business Case to Value Case -Assessing the Organizational Value of IT Investments. 17th European Conference on Information System (pp. 1–13). Frisk, E. J. (2011). Evaluating as Designing - Towards a Balanced IT Investment Approach (p. 76). Gothenburg: Chalmers Reproservice. Gunasekaran, a., Love, P. E. D., Rahimi, F., & Miele, R. (2001). A model for investment justification in information technology projects. International Journal of Information Management, 21(5), 349–364. doi:10.1016/S02684012(01)00024-X Gunasekaran, a., Ngai, E. W. T., & McGaughey, R. E. (2006). Information technology and systems justification: A review for research and applications. European Journal of Operational Research, 173(3), 957–983. doi:10.1016/j.ejor.2005.06.002 Hamaker, J. L. (2009). Information Technology Investment Methodologies: An Investigation into Method / Technology Fit. University of Nebraska. Hussein, R., Karim, N. S. A., & Selamat, M. H. (2007). The impact of technological factors on information systems success in the electronic-government context. Business Process Management Journal, 13(5), 613–627. doi:10.1108/14637150710823110 Irani, Z., & Love, P. E. D. (2001). The Propagation of Technology Management Taxonomies for Evaluating Investments in Information Systems. Journal of Management Information Systems, 17(3), 161–177.

Fauziah Ahmad, Wan Aryati Abd Ghani and Noor Habibah Arshad Joshi, K., & Pant, S. (2008). Development of a framework to assess and guide IT investments: An analysis based on a discretionary–mandatory classification. International Journal of Information Management, 28(3), 181–193. doi:10.1016/j.ijinfomgt.2007.09.002 Lech, P. (2007). Proposal of a Compact IT Value Assessment Method. The Electronic Journal Information System Evaluation, 10(1), 73–82. Li, S.-S., & Lee, L.-C. (2011). Using fishbone analysis to improve the quality of proposals for science and technology programs. Research Evaluation, 20(4), 275–282. doi:10.3152/095820211X13176484436050 Mccormack, K. (2000). The Information Technology Investment Decision Process : Why isn ’ t it working ? (pp. 1–7). Milis, K., & Mercken, R. (2002). Success factors regarding the implementation of ICT investment projects. International Journal of Production Economics, 80(1), 105–117. doi:10.1016/S0925-5273(02)00246-3 Mooney, J. G., Gurbaxani, V., & Kraemer, K. L. (2001). A Process Oriented Framework for Assessing the Business Value of Information Technology. Forthcoming in the Proceedings of the Sixteenth Annual International Conference on Information Systems (pp. 1–26). Özogul, C. O., Karsak, E. E., & Tolga, E. (2009). A real options approach for evaluation and justification of a hospital information system. Journal of Systems and Software, 82, 2091–2102. doi:10.1016/j.jss.2009.07.005 Peppard, J., Ward, J., & Daniel, E. (2007). Managing the Realization of Business Benefits from IT Investment. MIS QUARTERLY EXECUTIVE, 6(1), 1–11. Renkema, T. J. W., & Berghout, E. W. (1995). Methodologies for Information Systems Investment Evaluation at the Proposal Stage (pp. 1–15). Eindhoven. Roztocki, N., & Weistroffer, H. R. (2011). Information technology success factors and models in developing and emerging economies. Information Technology for Development, 17(3), 163–167. doi:10.1080/02681102.2011.568220 Sharif, A. M., Irani, Z., & Weerakkoddy, V. (2010). Evaluating and modelling constructs for e-government decision making. Journal of the Operational Research Society, 61(6), 929–952. doi:10.1057/jors.2010.11 Stewart, R., & Mohamed, S. (2002). IT/IS projects selection using multi-criteria utility theory. Logistics Information Management, 15(4), 254–270. doi:10.1108/09576050210436101 Walter, J. T. (2010). Prioritizing Information Technology Investment: Assessing the Correlations among Technological Readiness, Information Technology Flexibility, and Information Technology Effectiveness. Capella University. Wu, L.-C., & Ong, C.-S. (2008). Management of information technology investment: A framework based on a Real Options and Mean–Variance theory perspective. Technovation, 28, 122–134. doi:10.1016/j.technovation.2007.05.011

Persona Ontology for User Centred Design Professionals Farshid Anvari1 and Hien Minh Thi Tran2 1 Department of Computing, Macquarie University, Australia 2 Office of Financial Services, Macquarie University, Australia farshid.anvari@students.mq.edu.au Abstract: The design of software applications is a multistage creative process. User Centred Design (UCD) is a methodology used to develop applications that consider the goals of the users as a primary requirement. The use of personas, archetypical users, in UCD is well established in the software industry. Personas may take various forms: personas, mash‐ up personas, incomplete personas and unspoken personas. The use of personas increases as the design progresses through stages. Personas are used to facilitate the design of applications focusing on target users and to communicate with stakeholders. Users’ tasks of consuming the information from an application or web browser are cognitively demanding. It is expected that if the users’ profile matches the application’s personas, the users will find the application cognitively less demanding. Different techniques have been used to author personas with alternative media and varying information content, to such an extent that there is now a need to define the ontology of personas. This ontology would allow personas with consistent meanings to be shared across the design, development, test, marketing and sales teams; and would allow persona attributes and properties to be mapped to other design and development tools, such as actors, use cases and domain models. In this paper we explore the research question: how can we evaluate the usefulness of personas in software design, development, testing, communication and training? Through five case studies in various Australian industries, we provide insights into the use of personas in various phases of design, development, testing and training. We present a conceptual design of persona ontology. We propose the design of a persona ontology framework which represents personas in a hierarchical, multidimensional structure. We present a Holistic Persona with five dimensions: Factual, Personality, Intelligent, Knowledge and Cognitive Process Dimensions. The Holistic Persona at the top level is an abstract persona with generic properties and attributes. Keywords: holistic persona, scenarios, user centred design, usability, persona ontology

1. Introduction UCD is a methodology where the applications are developed considering the goals of the users as a primary requirement. Users actively participate in the design and development of the product. This methodology has its origins with the seminal work of Norman and Draper (1986). Personas ‐ archetypical users ‐ are fictional characters that represent the needs of typical users of the system and the way they interact with it; scenarios are the actions carried out by the personas interacting with the system (Goodwin, 2009 p 11). The use of personas in design and development of applications is well established in software industry (Miaskiewicz and Kozar, 2011). Personas support the design of applications by focusing on target users, and facilitating communication with stakeholders (Goodwin, 2009 pp 231‐234). Some authors have criticised the contribution that personas can make to the development work. For example, Chapman and Milham (2006) and Massanari (2010) argued that personas may not be determined using real data but are more likely to be developed by teams in political conflict. Mathews, Judge and Whittaker (2012) in a study of a global corporation, found that designers do not use personas as they find personas to be abstract, impersonal, misleading and distracting. It seems that poorly constructed personas can be counterproductive. In this paper, we explore the research question: how can we evaluate the usefulness of personas in software design, development, testing, communication and training? This introduction is followed by literature review on cognitive processes during design activities, various methods of authoring personas, personality traits, intelligences, knowledge, cognitive process dimensions, usability criteria and software ontology. The case studies section highlights the roles of personas in design, development and training of software applications. We propose the design of a persona ontology framework with five dimensions: Factual, Personality, Intelligent, Knowledge and Cognitive Process Dimensions. The paper is concluded with research outline to test our empirical findings.

Farshid Anvari and Hien Minh Thi Tran

2. Literature review Scholars from an architectural background divided design activity into three segments: problem understanding, design and retrospection; they further divided the design process into three sub‐segments: conception, development and representation (Akin and Lin, 1995). In the design segment, ideas are externalised in quick succession in the form of sketches; this is attributed to the limitation on working memory and the resultant difficulty in creating a new schema for commitment to long term memory (Kavakli and Gero, 2001). The amount of working memory is limited to hold about seven items (Miller, 1956). To overcome the limitations of working memory, schemas which help to systematically store and access information are created during the learning process (Sweller, Merrienboer and Paas, 1998). Schemas help us perform complex tasks many of which would otherwise be impossible such as computer programming (Jeffries et al., 1981). Visualisation helps designers in the creative design process (Park, Kim, and Cho, 2006). Park and Kim (2007) suggested that, for the creative design process, visual reasoning ability is needed which comprises perception, analysis, interpretation, generation, transformation, maintenance, internal representation and external representation. Different techniques have been used to author personas with alternative media and varying information content. Cooper (1999) proposed the idea of personas described with text and photographs. Goodwin (2009, pp 229‐297) provided methodologies to author personas and scenarios based on market research. Switzky (2012) developed personas by identifying use cases, whereas Schneidewind et al. (2012) identified personas from project scope definitions. Guðjónsdóttir (2001) used posters and cardboard cut‐outs to acquaint team members with personas. Long (2009) reported that designers showed a higher level of empathy towards personas with photos of real people compared with illustrated personas. Chang and Stolteman (2008) found that personas may take various forms: personas, mash‐up personas, incomplete personas and unspoken personas (personas that exist, undocumented). Personas represent end users of applications and the users have human qualities. The Five Factor Model of personality comprises Openness to experience, Conscientiousness, Extraversion, Agreeableness and Neuroticism ‐ OCEAN (McCrae and Costa, 1987). Dehghanzade and Moradi (2011) through survey questionaries based on the Big Five model of personality found that the personality traits of the users were influential in their perceptions that the information available by the system meets their requirements. Based on neuroscience and cognitive theory, the nature of people and psychological observations, Gardner (1993, pp. 7‐9) concluded that there are seven intelligences: linguistic, logical‐mathematical, spatial, musical, bodily‐ kinaesthetic, interpersonal and intrapersonal. The Knowledge Dimension is grouped into four categories: Factual, Conceptual, Procedural and Meta‐cognitive (Anderson and Krathwohl, 2001). The Cognitive Process Dimension is grouped into six categories: remember, understand, apply, analyse, evaluate, and create (Krathwohl, 2002). To solve problems, both the Knowledge and the Cognitive Process dimensions are essential. Abstract reasoning and knowledge of a domain provides a person with ability to transfer their skills from one domain to other similar domains (Anvari, Tran and Kavakli, 2013). It can be deduced that users with domain knowledge can use the software applications in similar domains efficiently. Testing and training are essential components of the software application development life cycle. Two sets of usability criteria that are commonly used to assess a website are the “Ten Heuristics” constructed by Nielsen and Molich (1990) and the “Eight Golden Rules” constructed by Shneiderman (1998). However, this kind of website assessment is not sufficient for a well‐designed system. Cronholm and Bruno (2009) found that there is a need for usability criteria on the communication and business process level as the users of computing system wish to achieve higher goals than just interact with the system. Personas need ontology to have consistent meanings. Ontology provides models of the real world for software designers by explicitly specifying domain concepts, terms, assumptions, definitions, relations, constraints, and other semantic contents (Chandrasekaran, Josephson and Benjamins, 1999). To overcome the limitation placed on the preparation, presentation and use of personas, one of our objectives is to define an ontology of personas that is accessible, personal, informative and focused, and therefore more useful to designers.

Farshid Anvari and Hien Minh Thi Tran

3. Case studies We present five case studies in various Australian industries spanning over a decade in which the authors used personas or users in various stages of design, development, communication, testing and training. Data presented here are from project diaries, field studies, observations and reflective journals. A summary of the characteristics of personas or users are presented in figure 2. The key themes that emerge from our case studies are presented in the section ‘General Findings from the Case Studies’. Our findings provide background to the conceptual design of a persona ontology.

3.1 Case 1: Electric plant request and outage management application The Plant Request and Outage Management System (PROMS) at the Hydro Electric Corporation, Tasmania, Australia, was developed to manage the planning of plant outages for maintenance work. The lead author, Anvari, as a software engineer, performed field studies with the users. While he was conceiving a solution, the work procedures, along with the organisation’s IT architecture were foremost in his mind. However, after the initial segments of the design, the users became more influential in his thinking. He had their knowledge, skills and requirements in mind while he was examining his sketches and preparing the design details. The users continued to be influential throughout the software development, communication with stakeholders and testing of the application. The development team and the authors came to know personally the majority of users. The team collaborated to meet the users’ needs. The authors considered users’ knowledge and personality while designing User Interfaces (UI) to meet users’ specific requirements for ease of information access. The users’ personalities, knowledge, intelligence and cognitive processes were incorporated in the test scripts. The tests verified that the application adopted itself to the users’ locations, functions and needs. When the application was put into production, the users needed minimal training and they used all features of the application.

3.2 Case 2: Digital TV extras application Anvari, as a software engineer, worked on a project at the Special Broadcasting Services (SBS) in Australia, called Digital TV extras (DEXTRAS). DEXTRAS required the web publications such as news, program guides, weather and feature films be available on TV channels (figure 1).

Figure 1: Two samples of DEXTRAS outputs on SBS digital TV Channel The personas were unspoken personas (Chang et al., 2008): the operators of the application, managers and viewers. The unspoken personas were conceived by interviewing SBS’s managers, journalists and operators and field studies with the operators. The method of interview was unstructured qualitative with the managers and journalists and, semi‐structured qualitative with the operators. Figure 2 presents a summary of samples of personas’ attributes. During the conception sub‐segment of design (Akin et al., 1995), Anvari’s thoughts were centred on meeting the requirements of television and IT systems to produce technical solutions. After the conception sub‐segment, unspoken personas and their needs assisted in producing the detailed design. For example, an unspoken persona representing the elderly viewers made management aware of two factors: their musical ability and desire to purchase the music album. Anvari incorporated these requirements in the

Farshid Anvari and Hien Minh Thi Tran

system at the design stage by incorporating functions to mix selected music track with the video and to provide album’s relevant textual and image information in a video slide (figure 1). This application was highly commended in the Commonwealth Broadcasting Association Award for Cost Effective Engineering (SBS, 2005).

3.3 Case 3: e‐health application Healthy.me, an online research platform to support consumer health decision making was developed at an Australian University. There are multiple elements to Healthy.me: Authoritative explanations of health topics; Personal Space for personal information; Collections such as pillbox, schedules, reports; Sharing information with team members; and Social Space to interact with others (Anvari, 2009). Journey was a feature of Healthy.me that provided specific information on medications, procedures, etc. associated with each stage of a health condition (Coiera et al. 2010). Anvari, a software engineer, designed and led the team to implement Healthy.me in stages. During the conception sub‐segment of the design, he had in mind only the requirements of the peripheral software that would interact with the core modules. During this phase of the design, no personas entered into his thinking. While assessing the design feasibility and preparing the details of the system, unspoken personas were prominent in his thoughts. These unspoken personas were in fact the authors themselves who had incidentally undergone health journeys of the kind anticipated by the application. The unspoken personas helped to develop data modules and were used during the functional testing.

3.4 Case 4: Application to disseminate tools An application to disseminate research tools was developed at an Australian research organisation by Anvari. This application would allow scientists to disseminate tools to a community of registered users. Anvari authored personas based on data collected from unstructured qualitative interview with stakeholders and literature search. The personas were posted on the organisation’s Wiki for all members to see and comment. Valuable suggestions were received from management and senior developers which helped during the development of the application. Personas and scenarios assisted in selecting the software frameworks, design data modules and UI layouts. Senior developers commented that by reading the personas and scenarios, they have gained full knowledge of the application, how it operates, what to expect from it and how to test it.

3.5 Case 5: Financial information management systems (FIMS) At an Australian University, Tran was responsible for developing procedures, designing training materials and testing FIMS. FIMS had a diverse group of users with different needs: administration and finance staff, department and faculty managers, post‐graduate students etc (Tran and Anvari, 2013). Personas representing each group of users were included in FIMS documents to help users to remember the relevant functions and gain quick access to relevant materials. The cognitive load on the memory and the learn‐ability of users of the software application depend on the training materials (Sweller et al. 1998). Cognitive load on users were considered e.g. users’ expectations and prior knowledge; worked‐examples and scenarios were used as the modelling technique; FIMS documents were illustrated with pictures and descriptions of real‐world scenarios.

3.6 General findings from the case studies The key themes that emerge from our case studies are:

In the conception sub‐segment (Akin et al., 1995) the focus of our attention was on the requirements of the applications without any consideration of personas or the users. The personas or users became influential during the development and representation of the design.

Personas conceived or the users selected for case studies had some of the following attributes: demographics, personalities, intelligence, knowledge and cognitive processes. We present samples of the attributes of the personas and users in three of our case studies in figure 2. The attributes represent the five dimensions: Factual, Personality (McCrae and Costa, 1987), Intelligent (Gardner, 1993), Knowledge and Cognitive Process (Anderson and Krathwohl, 2001) dimensions.

Visual processing is an essential part of creating interconnected software (Park et al., 2006). Personas provided visual aids and help software designers’ creativity by converting a number of design parameters to visual concepts.

We have observed that if applications are built meeting the needs of personas, target users would be able to use the application intuitively with little training. PROMS and DEXTRAS went into production smoothly

Farshid Anvari and Hien Minh Thi Tran as the applications were designed with users in mind. FIMS documents contained brief descriptions of personas and scenarios that matched the users and these provided the users with the schemas to remember the relevant sections of the documents.

To author suitable personas, we developed a five‐step framework: Describe (describe a persona for the application), Analyse (analyse the persona’s needs), Transform (transform the needs of the persona into an action plan; turn their needs into scenarios), Act (act out the persona) and Evaluate (evaluate the closeness of the personas to the users; refine the personas by reflecting on the results of the worked examples, tests and survey feedbacks).

Personas are useful tools for testing applications (Grudin and Pruitt, 2002). By adapting Nielsen’s (1993) ten heuristics, we have developed the E‐USABLE framework to meet objectives at the business processes levels. E‐USABLE has seven categories: Effective to use (effectiveness) e.g. how effective the application documents can convey the business processes to the users; have good Utility (utility) e.g. do the application documents contain worked‐examples that assist users to achieve business goals; Safe to use/prevent possible errors (safety); can the data entries be Audited (audit‐ability); do User Interfaces provide feedBack (feedback); is the application easy to Learn (learn‐ability) and Efficient to use (efficiency). We devised the three‐dimensional table (see figure 3), which we used as a framework to author suitable personas for testing and training purposes. We also use the E‐USABLE framework to create questionaries to evaluate software applications and prepare application documents using personas. It is out of scope of this paper to discuss the details of E‐USABLE.

Figure 2: Sample of the personas’ attributes derived from three case studies

4. Persona ontology Based on our case studies we can infer that to engineer an idea into software, UCD professionals require personas that are not only factually accurate but are emotionally resonant so that it can assist in design thinking and design communication. All team members interpret personas similarly. Personas have been widely used. However, there is no defined framework to construct personas. We have used the E‐USABLE framework to author suitable personas for testing and training purposes. Literature review shows that personas have been created based on facts e.g. demographics, hobbies, profession, family details etc. Figure 4 is a persona in Vietnam for Vietnamese UCD professionals; Figure 5 is a persona in Australia for Australian UCD professionals. The confusion arises if an Australian developer, while developing an application in Australia, reads a persona description: Chi is a farmer. The persona description lacks details which can be overcome using the persona ontology shown in figure 6. In this design of a persona‐ ontology framework, a persona is represented in a hierarchical, multidimensional structure. The persona at the

Farshid Anvari and Hien Minh Thi Tran

top level is an abstract persona with generic properties and attributes. The sub‐derivatives of the abstract persona are personas with generic data that represent many types of users in various cultural settings, disciplines and demographics. When persona attributes and properties are populated with specific data, they may be used effectively in design and development of an application. The persona descriptions in figures 4 and 5 have factual details but they do not convey any information about the personality traits, knowledge or cognitive processes of the personas.

Figure 3: Example of E‐USABLE framework used to author suitable personas for testing and training

Farshid Anvari and Hien Minh Thi Tran

Figure 4: Chi is a farmer Figure 5: Karl is a farmer Following is an example of changes in design decisions that could have been the result of considerations of each of the persona’s dimensions in the ontology. At a broadcasting company, if the UCD professionals had full access to audience research data, they could test assumptions about viewers' facts (demographics), personalities (McCrae and Costa, 1987), intelligences (Gardner, 1993), knowledge and cognitive processes (Anderson and Krathwohl, 2001). The data would have then revealed unfulfilled needs for programs such as short feature films to be included in the broadcast schedule to meet the audiences’ artistic intelligences. This would have affected the persona building process and would have influenced changes in the design of the application such as DEXTRAS.

Figure 6: Ontology of a famer persona

Farshid Anvari and Hien Minh Thi Tran

5. Holistic personas We propose Holistic Personas which have multi‐dimensional facets that designers can emotionally resonate with and more realistically represent the human users of the system. Holistic Personas consist of five dimensions: Factual, Personality, Intelligent, Knowledge, Cognitive Process dimensions (figure 7). Holistic Personas are at the root of persona ontology (figure 6) with all the attributes available to be used as needed further. The Factual Dimension: Factual dimension covers details such as demographics, interests etc. The Personality Dimension: openness to experience, conscientiousness, extraversion, agreeableness and neuroticism (McCrae and Costa, 1987). UCD professionals can use the personality traits to design a matching application. The personality of the operators (conscientious, disagreeable and high‐strung) influenced the design of DEXTRAS: while broadcasting live events, the application required minimum number of keyboard entries. The Intelligent Dimension: linguistic, logical‐mathematical, spatial, musical, bodily‐kinaesthetic, interpersonal and intrapersonal (Gardner, 1993). The application should cater for persona’s intelligence. To retain the attention of users who are talented in one of the intelligences, the applications should be challenging. UCD professionals can use personas’ intelligent dimension to design applications that cater for users’ intelligence profiles. DEXTRAS highlighted the importance of musical intelligence: elderly viewers had musical intelligence; DEXTRAS was designed to cater for their needs. In FIMS, the documents were tailored according to presumed personas' intelligence. The Knowledge Dimension: Factual knowledge, Conceptual knowledge, Procedural knowledge and Meta‐ cognitive knowledge (Anderson and Krathwohl, 2001). UCD professionals can use personas’ knowledge dimension to design applications suit personas’ domain knowledge. In PROMS, the operators had procedural knowledge; the managers had meta‐cognitive knowledge. The Cognitive Process Dimension refers to the learning process: remember, understand, apply, analyse, evaluate, and create (Krathwohl, 2002). Novices (e.g. patients, students) remember the facts but the experts (e.g. doctors, professors) evaluate existing methodologies and create new ones as the information presents meta‐cognitive knowledge. UCD professionals can use personas’ cognitive process dimension to develop applications that match users’ cognitive processes. In FIMS, a general manager used FIMS documents to analyse and evaluate FIMS procedures to effectively streamline business processes.

Figure 7: The holistic persona with five persona dimensions In summary, all dimensions of the Holistic Personas are concepts that can be modelled. Holistic persona ontology would allow personas with consistent meanings be shared across the teams. Holistic Personas do not act like actors in UML but they present persona ontology for the UCD professionals. Holistic Personas are hierarchical, multidimensional structure and have consistent meanings; they can be shared across the design, development, testing, marketing and sales teams.

6. Conclusion and future work Our case studies support the importance and utility of personas in software design and confirm the previous findings that personas have to feel realistic for designers to use them in their design work (Matthews et al.

Farshid Anvari and Hien Minh Thi Tran

2012; Long, 2009; Blomquist and Arvola, 2002). In our case studies, while conceiving the design we did not use personas, but soon after this stage, personas or users became prominent in our thinking. We present Holistic Personas with five dimensions: Factual, Personality, Intelligent, Knowledge and Cognitive Process dimensions. Our case studies indicate that Holistic Personas have the potential to present a persona ontology for UCD professionals. Holistic Personas are accessible, personal, informative and focussed, and therefore useful to designers. The persona ontology would allow personas with consistent meanings to be shared across the design, development, test, marketing and sales teams. Persona attributes and properties can be mapped to other design and development tools, such as actors, use cases and domain models. We plan to investigate each of the five dimensions separately to establish their contributions to software development. In the first stage we will test the personality dimension of Holistic Personas. Using ontological engineering principles, we will develop a model for the personality dimension of the personas. We will investigate the effect of the personas authored with the personality dimension on developers and the users of software applications.

Acknowledgements We thank Paul Vincent, A/Prof Manolya Kavakli, A/Prof Mark Wiggins and the anonymous reviewer for their helpful comments. We also thank Will Berryman, Paul Vincent, Greg Koen, Nigel Milan, Katherine McConnell, Daniel Collins, Dr David Miron, Dr Susie Hester and Alf Clark for their support during the design and development of the applications.

References Akin, O., and Lin, C. (1995) “Design protocol data and novel design decisions”, Design Studies, Vol. 16, No. 2, pp 211‐236. Anderson, L. W., and Krathwol, D. R. (2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Anvari, F. (2009), “Healthy.me: an online research platform to support consumer health decision making”, in eResearch Australasia, Novotel Sydney Manly Pacific, 9‐13 November 2009, viewed 26 Jan 2013, http://espace.library.uq.edu.au/view/UQ:188376 Anvari, F., Tran, H.M.T., and Kavakli, M. (2013) “Using Cognitive Load Measurement and Spatial Ability Test to Identify Talented Students in Three‐Dimensional Computer Graphics Programming”. International Journal of Information and Education Technology, Vol. 3, No. 1, pp. 94‐99. Blomquist, Å., and Arvola, M. (2002) “Personas in action: ethnography in an interaction design team”, in Proceedings of the second Nordic conference on Human‐computer interaction, pp 197‐200. ACM. Chandrasekaran, B., Josephson, J.R., Benjamins, V.R., (1999) “What are ontologies, and why do we need them?” IEEE Intelligent, 14, 1, pp 20–26. Chang, Y., Lim, Y., Stolterman, E., (2008) “Personas: From Theory to Practices”, Proceedings: Nordi CHI. Chapman, C.N., Milham, R. P. (2006) “The persona’s new clothes: methodological and practical arguments against a popular method”, Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, pp 634–636. Coiera, E., Lau, A. Y., Anvari, F., and Sacks, G. (2010) “Healthy. me: An Online Research Platform to Support Consumer Health Management”. In Melbourne: Health Informatics Conference (HIC). Cooper, A., (1999), The Inmates Are Running the Asylum, Sams. Cronholm, S., and Bruno, V. (2009) “Usability of IT‐systems is more than interaction quality‐the need of communication and business process criteria”, in 17th European Conference on Information Systems. Dehghanzade, H. and Moradi, M. A. (2011) “A Survey of Human Factors’ Impacts on the Effectiveness of Accounting Information Systems”, International Journal of Business Administration, Vol 2, No. 4, pp 166‐174. Gardner, H. (1993), Multiple Intelligences: The Theory in Practice. Basic Books. A Division of HaperCollins Publishers, pp 7‐9. Goodwin, K., (2009) Designing for the digital age: How to create human‐centered products and services, Wiley publishing. Grudin, J., and Pruitt, J. (2002), “Personas, participatory design and product development: An infrastructure for engagement”. In Proceedings of the Participatory Design Conference, pp. 144‐152. Guðjónsdóttir, R. (2001) “Life‐Size Personas”. In Proceedings of the Usability Professionals Conference, Las Vegas, USA. Jeffries, R., Turner, A., Polson, P. & Atwood, M. (1981) “The processes involved in designing software”, Cognitive Skills and Their Acquisition, pp. 255‐283. Kavakli, M., Gero, J. S. (2001) “Strategic knowledge differences between an expert and a novice”, in Gero JS, Hori K (eds) Strategic knowledge and concept formation III. Key Centre of Design Computing and Cognition, University of Sydney, Australia, pp 55–68. Krathwohl, D, (2002) “A revision of Bloom’s Taxonomy: An Overview”. Theory Into Practice, Volume 41, No. 4. Long, F. (2009) “Real or Imaginary: The effectiveness of using personas in product design”, in Irish Ergonomics Society Annual Conference, Dublin, May 2009.

Farshid Anvari and Hien Minh Thi Tran

Massanari, A. L. (2010) “Designing for imaginary friends: information architecture, personas and the politics of user‐ centered design”, New Media & Society, Vol 12, No. 3, pp 401‐416. Matthews, T., Judge, T., & Whittaker, S. (2012) “How do designers and user experience professionals actually perceive and use personas?” In Proceedings of the ACM annual conference on Human Factors in Computing Systems, pp 1219‐ 1228. McCrae, R. R., & Costa, P. T. (1987) “Validation of the five‐factor model of personality across instruments and observers”, Journal of personality and social psychology, Vol 52, No 1, pp. 81. Miaskiewicz T., Kozar K. A. (2011) “Personas and User‐Centered Design: How Can Personas Benefit Product Design Processes?” Design Studies, Vol 32, No. 5, pp 417‐430. Miller, G. A. (1956) “The magical number seven, plus or minus two: Some limits on our capacity for processing information” Psychological Review, Vol 63, pp. 81‐97. Nielsen, J. (1993) Usability Engineering Academic Press, Boston, xiv, 362. Nielsen, J., and Molich, R. (1990) "Heuristic evaluation of user interfaces," Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, ACM, Seattle, Washington, United States, pp 249 ‐ 256. Norman, D. A., Draper, S.W. (1986) User‐Centered System Design: New Perspectives on Human Computer Interaction, Erlbaum, Hillsdale, NJ Park J. A., and Kim Y. S. (2007) “Visual Reasoning and design processes, International conference on engineering design”, ICED’07 28 ‐ 31, cite des sciences et de l'industrie, Paris, France. Park, J. A., Kim, Y. S., & Cho, J. Y. (2006). “Visual reasoning as a critical attribute in design creativity”. In Proceedings of International Design Research Symposium. Petre, M., & Blackwell, A. F. (1999) “Mental imagery in program design and visual programming”. International Journal of Human‐Computer Studies, Vol 51, No. 1, pp 7‐30. SBS (2005) “Special Broadcasting Service (SBS) Annual Report”, viewed 23 Jan 2013, http://media.sbs.com.au/sbscorporate/documents/230408_rep_of_ops.pdf, p. 87. Schneidewind, L., Horold, S., Mayas, C., Kromker, H., Falke, S., & Pucklitsch, T. (2012) “How personas support requirements engineering”. In Usability and Accessibility Focused Requirements Engineering (UsARE), First International Workshop on 1 May. IEEE. Shneiderman, B. (1998), Designing the user interface: Strategies for effective human‐computer‐interaction, (3rd ed) Addison Wesley Longman, Reading, Mass, xiv, 639. Sweller, J., Chandler, P., Tierney, P., Cooper, M. (1990) “Cognitive load as a factor in the structuring of technical material”, Journal of Experimental Psychology: General, Vol. 119, No. 2, pp 176‐192. Sweller, J., Merrienboer J.J. and Paas F.G. (1998) “Cognitive Architecture and Instructional Design”, Educational Psy‐chology Review, Vol 10, No 3, pp. 251–296. Switzky, A. (2012) “Incorporating UCD into the software development lifecycle: a case study”. In Proceedings of the ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts, pp 469‐484. Tran, H. M. T. and Anvari F. (2013) “How Reflective Professionals Design and Evaluate Financial Information Management Systems Courses”. In Proceedings of the ICIME 4th International Conference on Information Management and Evaluation, Ho Chi Minh City, Vietnam.

Evaluating Strategic Information Systems Planning (Sisp) Performance Among Malaysian Government Agencies Using Organizational Learning‐Based Model Fazidah Abu Bakar1, Mohd Adam Suhaimi2 and Husnayati Hussin2 1 Tax Analysis Division, Ministry of Finance, Putrajaya, Malaysia 2 Kulliyyah of Information & Communication Technology (KICT), International Islamic University, Kuala Lumpur, Malaysia fazidah.abubakar@treasury.gov.my adam@iium.edu.my husnayati@iium.edu.my Abstract: Previous strategic information systems planning (SISP) studies have highlighted that an ideal SISP practice should emphasise on organisational learning (OL) in their SISP approach. Surprisingly, studies focusing on the extent of OL and its contextual factors are lacking. This study aims to develop and validate a model for evaluating SISP performance in the Malaysian government agencies from OL perspective. For this purpose, an SISP performance model was developed based on the absorptive capacity theory where SISP is viewed as a learning process instead of planning process. The theoretical assumption is that the quality of SISP contextual factors will influence the extent of SISP learning which eventually will influence the SISP success. The contextual factors are SISP climate, consultant expertise and CIO capability. Meanwhile, the SISP learning factors comprise of shared ICT/business knowledge and SISP process effectiveness with SISP success as the SISP performance measurement. 706 questionnaires were sent to 234 selected government agencies in Malaysia. Only 27% of the government officers responded to the questionnaire meets the criteria of this study. Measurements were initially validated using SPSS. Subsequent confirmation on the measurements and structural validity were done by adopting the Structural Equation Modeling (SEM) analyses using AMOS. The results of this study demonstrated that the hypothesised SISP performance model adequately fits the sample data which assumed the model is acceptable. Generally, the findings of this study indicated that higher level of SISP climate, consultant expertise and CIO capability positively influence the extent of SISP learning factors and later influence the SISP success. In determining the SISP success, this study revealed that it is crucial for the Malaysian government agencies to identify appropriate SISP team members based on their knowledge, skill and attitude to promote conducive SISP learning climate for effective SISP decision making. Keywords: SISP, government agencies, SISP performance, organizational learning, absorptive capacity model

1. Introduction The advancement in Information Technology (IT) and complexity of Internet technology have influenced the Strategic Information System Planning (SISP) to consider ‘fitting’ the identified strategies with the environmental uncertainty, culture, experience and skill of the organization (Doherty, Marples, & Suhaimi, 1999; Clark, Clark, Gambill, & Fielder, 2000; Lee & Bai, 2003; Duhan, 2007). Newkirk & Lederer (2006) describe uncertainty as the lack of information on how to develop the IS plans. In order to reduce uncertainty, Otim et al. (2009) suggest that through organisational learning (OL), organisation can acquire more information from the environment. Few researchers like Lee & Bai (2003), Duhan (2007) and Otim et al. (2009) recognise the applicability of OL in the current environment and call for more research on SISP from OL perspective. In addition, difficulties of adapting to these considerations explain why SISP remains the top managerial concerns among practitioners (Teo & Ang, 2000; Luftman & Ben‐Zvi, 2010). Based on the current scenario, some studies suggested organisation should view SISP as a learning process rather than a planning process (Huysman, Fischer, & Heng, 1994; Reponen, 1998; Wang & Tai, 2003). Surprisingly, empirical studies on the extent of OL and its antecedents in SISP are lacking. Therefore, a new SISP performance model need to be developed where the focus is more on the capability of the organisation to adapt and learn from any environmental uncertainty for continuous improvement. The main objective of this study is to investigate whether the SISP performance model can be evaluated using organisational learning‐based model. The research question is how the SISP contextual factors, the extent of OL and SISP performance inter‐related with each other in SISP? The paper is organised to give background on SISP from OL perspective, Absorptive Capacity Theory, the research model and hypotheses, research methodology, reliability and validity, model and hypotheses testing, discussion of findings and conclusion.

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin

2. SISP from OL perspective OL can be defined as the capacity or process within an organisation to maintain or improve performance based on experience, knowledge and understanding (Fiol & Lyles, 1985; Dibella, Nevis, & Gould, 1996). SISP process when viewed as a learning process such that past experiences, knowledge, procedures and routines were referred by the organisation for decision making. Both Huysman et al (1994) and Baker (1995) view that the implications of SISP from OL perspective will impact on the various actors in planning process and the organisation of the process itself, in addition to the SISP performance. The organisation that had greater experience with SISP and in a more mature IT/IS stage had better SISP success (Grover & Segars, 2005). Earl (1993) describes the occurrence of OL in SISP is reflected from the IS strategies that seemed to emerge from organisational activities, such as “sense and response” or “rational and adapt” to changes in business strategies in interactive manner and broad organisational participation. Therefore, the theoretical perspective of OL provides an appropriate underlying description of the current SISP scenario that enables the investigation of the linkage between the extent of what has been learned from SISP experience and SISP success. By doing so, it will assist the organisation to evaluate the capability of its organisational knowledge (mean) in order to sustain competitive advantage (end).

3. Absorptive capacity theory In their conceptual paper, Huysman et al. (1994) suggested the ideas on learning experiences can best be explained by the notion of organisation’s absorptive capacity developed by Cohen & Levinthal (1990). It defines as the ability of the organisation to identify and value, assimilate and exploit external information for commercial use (Cohen & Levinthal, 1990). However, based on the notion of absorptive capacity from Cohen & Levinthal (1990), Zahra & George (2002) reconceptualised it as a set of organisational routines and processes by which organisations acquire, assimilate, transform and exploit knowledge to produce a dynamic organisational capability which reflected from Potential absorptive capacity (PACAP) and Realized absorptive capacity (RACAP). Activation triggers, social integration mechanism and regimes of appropriability are the moderators that have potential influence on the level of absorptive capacity.

4. Research model and hypotheses

Figure 1: Research model and hypotheses The conceptual framework based on Absorptive Capacity Model as presented in Figure 1 provides the underlying assumption that the quality of SISP context influences the extent of the SISP learning which later affects the SISP performance. It is usually assumed that the quality of organizational climate affects the level of knowledge capacity absorbed which later influenced organizational performance (Mikkelsen & Grønhaug, 1999). Absorptive capacity theory when applied to the domain of SISP suggests that the experiences during SISP which involves social interactions, policies, procedures and structures have affected the organisation cognitively and behaviourally; which later influence the SISP performance. In other words, the organisation’s ability to effectively carry out SISP (OL has occurred) is dependent on the development of a variety of SISP context related to knowledge and processes that bind together various SISP team members namely the top management, the business managers, the IT manager (Mentzas, 1997; Pearson & Saunders, 2006) and external consultants in some agencies (Teo et al., 1997; Chi et al, 2005). Implications of SISP from OL will impact the

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin actors with shared ICT/business knowledge and will also impact the SISP process itself in terms of SISP process effectiveness. Therefore, H1: Higher level of shared ICT/business knowledge will positively influence the SISP process effectiveness; H2: Higher level of SISP climate will positively influence the shared ICT/business knowledge; H3: Higher level of SISP climate will positively influence the SISP process effectiveness; and H4: Higher level of consultant expertise will positively mediate the effect of SISP climate on the shared ICT/business knowledge Prior studies also have indicated that the Chief Information Officer’s (CIO) role in SISP also affects the SISP process (Bai & Lee, 2003; Pai, 2006). Good ICT/business relationships with free communication flow will develop trust and promote knowledge sharing behaviour among SISP members for better SISP process (Teo & Ang, 2001; Bai & Lee, 2003; Lee & Pai, 2003; Pai, 2006). Thus, H5: Higher level of CIO capability will positively mediate the effect of shared ICT/business knowledge on SISP process effectiveness. Evidence from the works of Kearns & Lederer (2004) indicated that SISP helps the knowledge creation for both business and ICT managers where Reich & Benbasat (2000) study concluded that only shared ICT/business knowledge will influence SISP performance in terms of attaining long term alignment of IS and business objectives. Similarly, SISP also assists the knowledge utilisation reflected from SISP process effectiveness. Related SISP studies from Segars & Grover (1998), Kunnathur & Shi (2001), Lee & Pai (2003) have concluded that SISP process effectiveness is vital to SISP success. Therefore, H6: Higher level of shared ICT/business knowledge will positively influence the SISP success; and H7: Higher level of SISP process effectiveness will positively influence the SISP success Empirical studies from Mikkelsen & Grønhaug (1999), Droge et al.(2003), Tippins & Sohi (2003) and Jantunen (2005) have consistently highlighted the mediating role of knowledge elements in influencing the performance of the organisation especially to sustain competitive advantage over other organisations. Similarly, in the context of SISP with the rapid change in business and ICT environment, it is hypothesized that H8: SISP learning fully mediates the effect of SISP context on SISP performance

5. Research methodology To empirically test the research model, a sample is selected from 382 Malaysian government agencies which comprises of federal agencies, secretary of state offices and local authorities as listed under Public Service Department (JPA, 2008) and also the recipients of the government circulars. Quota sampling approach based on the sample size decision table from Krejcie & Morgan (1970) is applied to determine the appropriate representation of the various agency levels. In addition, the multiple respondents approach is used where both views from business and ICT managers are gathered in order to reduce sampling bias. The questionnaires were pretested and piloted before the execution of the actual survey via mail. . Measurement items used in this study were adapted from previously validated measure or were developed on the basis of a literature review mainly from SISP, IT management and IT outsourcing literature as listed in the Appendix. All items were measured using a seven‐point Likert‐type scale. Of the 706 distributed, 309 completed and usable questionnaires were returned, representing a response rate of 43.8 percent from 160 government agencies. All respondents were business managers, ICT managers, CIOs where more than 50 per cent have working experience of more than 5 years in current agency. 92.3 percent of the respondents from federal agencies, while both respondents from secretary of state office and local authorities are at 3.9 percent each. Overall, 60.8 percent respondents are from ICT division and 39.2 percent respondents from other than ICT division. A comparative analysis of two descriptive variables (current position grade, and years of service in current agency) was conducted in order to see if respondents have significantly different characteristics from non‐respondents. The Mann‐Whitney test results provide evidence that there was no response bias problem in the sample ( p‐values are 0.517 and 0.449). However, only 27 percent (153 responses) from 43.8 percent responses met the criteria of this study which indicated involvement of CIO and external consultant in their SISP activities.

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin

6. Reliability and validity To empirically assess the construct reliability and validity, this study conducted the reliability analysis and exploratory factor analysis (EFA) using SPSS v16. The result of items performance using EFA by applying ‘principal factor analysis’ with ‘direct oblimin’ rotation indicated that 75 percent of the total variance explained with all items showed factor loadings greater than 0.35. The reliability analysis on all constructs indicated the items measuring respective construct clustered together as a set with reliability estimate value (Cronbach alpha) ranging from 0.81 to 0.96 which is greater than the recommended threshold of 0.70 (Hair et al., 2006). In order to determine further construct validity and goodness fit of the specified research model against the sample data, Structural Equation Modeling (SEM) technique is applied by using confirmatory factor analysis (CFA) in AMOS v16 as the statistical tool. By using statistical models in SEM, the main objective is to determine whether the specified model is valid rather than to find a suitable model (Gefen et al., 2000; Byrne, 2001; Hair et al., 2006). Following the two steps approach as recommended by Anderson & Gerbing (1988), the measurement model is evaluated first then followed by structural model given that the measurement model are acceptably fit the data. For a measurement model to have sufficiently good model fit, the chi‐square value 2 normalized by degrees of freedom ( /df) should not exceed 3 (Bagozzi and Yi, 1988), Comparative Fit Index (CFI) and Tucker Lewis Fit Index (TLI) (Hair et al., 2006) should exceed 0.9, and the Root Mean Square Error of Approximation (RMSEA) should not exceed 0.10 (Hair et al., 2006). The measurement properties resulted from CFA are reported in Table 1. The composite reliability assessed the internal consistency of the scales. From Table 1, the composite reliability of all constructs exceeded the benchmark of 0.6 as recommended by Bagozzi and Yi (1988). Additionally, convergent validity refers to the extent to which multiple measures of a construct agree with one another. Bagozzi and Yi (1988) suggest that weak evidence of convergent validity exists when item factor loading is significant. Factor loading exceeds 0.5 is acceptable with value exceeding 0.7 shows strong evidence of convergent validity. From AMOS outputs, parameter estimates for convergent validity namely standardized factor loadings should exceed 0.5, t‐value should exceed ±1.96 at p‐level < 0.05 and composite reliability should exceed 0.6 (Hair et al., 2006). From Table 1, the factor loading for all items exceeds the recommended level of 0.5, factor loadings are statistically significant at p 0.05 and composite reliability exceeds 0.6. Therefore, all scales for construct reliability and validity were met. Table 1: CFA results Construct

Standardised factor loading

Critical ratio (t‐value)

Squared multiple correlation

SISP Climate

Average variance extracted

0.74

0.54

TMI

0.71

7.07

0.51

IntraAC

0.90

‐‐‐

0.82

InterAC

0.54

6.61

0.30

0.79

0.66

Shared ICT/Business Knowledge MIK

0.81

‐‐‐

0.66

MBK

0.81

7.49

0.65

0.95

0.78

Consultant expertise CCR3

0.83

‐‐‐

0.69

CCR4

0.85

13.15

0.72

CCO1

0.92

15.08

0.85

CCO2

0.94

15.08

0.88

CCO3

0.87

13.78

0.76

CCO4

0.89

14.32

0.80

0.94

0.75

CIO capability

Composite reliability

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin Construct

Standardised factor loading

Critical ratio (t‐value)

Squared multiple correlation

Composite reliability

Average variance extracted

CIOK1

0.82

15.33

0.68

CIOK3

0.81

14.85

0.66

CIOS1

0.94

‐‐‐

0.89

CIOS2

0.93

21.65

0.86

CIOS3

0.83

15.50

0.68

0.88

18.02

0.77

CIOS4

SISP Process Effectiveness

0.90

0.66

FOR

0.91

12.29

0.82

PAR

0.79

‐‐‐

0.62

COM

0.67

8.58

0.45

CON

0.85

12.89

0.72

0.94

0.69

SS1

0.81

18.50

0.65

SS2

0.84

19.78

0.70

SS3

0.87

21.58

0.76

SS4

0.88

‐‐‐

0.77

SS5

0.87

21.64

0.76

SS6

0.81

18.48

0.65

SS7

0.81

18.76

0.66

SS8

0.76

16.71

0.58

SISP Success

7. Model and hypotheses testing Similar to measurement model, the evaluation of structural model depends on the model adequacy based on the model fit indices and parameter estimates. However, the structural model is interested on how constructs relate to each other according to the hypothesized model. The model fit indices where normed 2 was 1.707, CFI was 0.91, TLI was 0.91 and RMSEA was 0.068 suggesting adequate model fit. In SEM, the direct effects are the relationships linking two constructs with a single arrow. Meanwhile, the indirect effects are the relationships that involve a sequence of relationships with at least one mediating construct involved (Kline, 2005; Hair et l., 2006). The size of indirect effect can be determined by multiplying the direct effects by each other where if exceeds 0.08 indicate the mediating effect exist (Hair et al, 2006). The summarized results on the hypothesized structural estimates are presented in Table 2. From the structural model evaluation, six hypothesised paths were statistically significant while two were non‐significant. Unlike other hypotheses results are determined from the significance of their respective parameter estimates; the result of H8 depends on the outcome of all the hypotheses results. In addition, a new significant linkage is discovered from consultancy expertise to CIO capability. Table 2: Summarised results on the hypothesized structural estimates

Hypothesis tested

Standardised path coefficient (sign)

H1: Higher level of shared ICT/business knowledge will positively influence the SISP process effectiveness H2: Higher level of SISP climate will positively influence the shared ICT/business knowledge H3: Higher level of SISP climate will positively influence the SISP process effectiveness

0.05 (+)

Critical ratio (t‐value)

Effect (size)

Result

0.29 ns

Direct (0.05)

Not supported

0.61 (+)

4.09***

Supported

0.66 (+)

3.55***

Direct (0.61) Direct (0.66)

Supported

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin Critical ratio (t‐value)

Hypothesis tested

Standardised path coefficient (sign)

Effect (size)

Result

H4: Higher level of consultant expertise will positively mediate the effect of SISP climate on the shared ICT/business knowledge

0.68 (+)

5.76***

Indirect (0.14)

Supported

0.21 (+)

1.98*

H5: Higher level of CIO capability will positively mediate the effect of Shared ICT/business knowledge on SISP process effectiveness

0.52 (+)

Indirect (0.10)

Supported

0.20 (+)

4.72*** 2.00*

H6: Higher level of shared ICT/business knowledge will positively influence the SISP success

0.15 (+)

1.59 ns

Direct (0.15)

Not supported

H7: Higher level of SISP process effectiveness will positively influence the SISP success

0.73 (+)

4.88***

Direct (0.73)

Supported

Significant H2, H3, H4, H5 and H7 linkages

Supported

H8: SISP learning fully mediates the effect of SISP context on SISP performance Consultant Expertise positively influence the CIO Capability

0.26 (+)

2.69**

Direct (0.26)

New significant linkage

Note: *** Significant at p< 0.001 (t > ± 3.29) * Significant at p< 0.05 (t > ± 1.96) ** Significant at p< 0.01 (t > ± 2.58) ns Non‐significant

8. Discussion of findings The underlying assumption of this study is that a condusive learning environment will positively influence the extent of learning which later improves performance. This study found strong empirical support for the extent of SISP learning factors fully mediate the effect of SISP contextual factors on SISP success . The outcome of this study is consistent with other knowledge‐based view studies like Boynton et al. (1994), Droge et al. (2003). Tippins & Sohi (2003) and Andrawaina & Govindaraju (2008). In terms of role participation, this study is consistent with Pita et al.’s (2009) findings where consultants’ involvement did not directly influence SISP performance. These findings imply that SISP process can be viewed as an interactive learning process where the past experience, knowledge, procedures, routines were referred by any agency for decision making are most likely to benefit its capabilities and SISP process itself in order to attain the agency’s performance in a long run. In this case, a condusive SISP climate that promotes knowledge sharing among the various SISP team members either within the agency or between agencies is important including the top management involvement to encourage participation from relevant parties. Some agencies seek new insights and expertise from outside knowledge especially the external consultants to facilitate the ISP development. New knowledge is acquired based on the social interaction among the SISP team members which is assimilate with past knowledge and experience for better exploitation on the SISP decision making. The involvement of CIO facilitates the knowledge flow and in finalising the decisions in accordance with Lee & Pai (2003) and Pai (2006) study. Consequently the organisational performance is reflected from this experience. Moreover, from the structural model evaluation on the hypothesised model based on the absorptive capacity model by Zahra & George (2002) has demonstrated that the model is adequately fit the sample data. Therefore, it can be concluded that SISP performance model from OL based is successfully established.

9. Conclusion Theoretically, this study adds new insights and understanding to existing body of SISP knowledge especially in SISP performance evaluation in the public sector. In addition, the establishment of the SISP performance model based on the absorptive capacity approach implies that the conceptualization of such theory can be further validated and applied in SISP literature.

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin Practically, using the questionnaires instrument developed in this study assists the organisations in assessing the status of SISP practice in their organisations for continuous improvement. It integrates the elements drawn from SISP environment, structure, process and performance. The policy makers may review these observations and incorporate them into forms of guidelines, standard procedures and directives for other agencies to emulate. Generally, this study has empirically showed how the extent of OL influences the overall performance of SISP in organisations. By focusing the extent of the capabilities among SISP team members, accurate decision makings on SISP can be assured and eventually help the organisations to continuously improve SISP. In this case, the study revealed that it is crucial for the Malaysian government agencies to identify appropriate SISP team members based on their knowledge, skill and attitude to promote condusive learning climate for better knowledge exchange towards deriving effective SISP decision making in order to sustain strategic or competitive advantage.

Acknowledgements We are grateful to the anonymous reviewers for their constructive comments.

Appendix: List of variables Variable

Description

No of items

Reference

Shared ICT/business knowledge

Ability of ICT & business managers to understand and be able to participate in the other’s key processes and to respect each others’ unique contribution and challenges

Boynton et al. (1994), Bassellier et al. (2003), Bassellier & Benbasat (2004)

SISP process effectiveness

Emergent pattern of planning process characteristics that organises activities for planning accomplishment

Segars & Grover (1998), Grover & Segars (2005)

SISP climate

Shared, enduring perception by SISP team members on salient aspects of SISP environment associated with SISP activities

Byrd et al. (1995), Nelson & Cooprider (1996), Basu et al. (2002), Bai & Lee (2003), Lee & Pai (2003), Nieminen (2005), Kearns (2006), Pai (2006)

Consultant expertise

The consultant’s capability to understand the client’s problem, identify and impart solution to client

Ko et al. (2005), Kakabadse & Louchart (2006)

CIO capability

The knowledge and interpersonal skill of the CIO in terms of the degree of strategic business and ICT knowledge and political savvy

Armstrong & Sambamurthy (1999), Smaltz et al. (2006)

SISP success

The extent of improvements in the organisation’s capabilities to support IS strategic management combined with degree of fulfillment of SISP objectives and degree of SISP contribution to overall organisation performance

Tang & Tang (1996), Lederer & Sethi (1998), Segars & Grover (1998), Doherty et al. (1999), Seneviratne (1999)

References Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two‐step approach. Psychological Bulletin, 103(3), 411‐423.

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin Andrawina, L., & Govindaraju, R. (2008, 10‐12 June). Knowledge sharing capability, absorptive capacity, and innovation: an empirical study of Indonesia's information and communication technology industries. Paper presented at the Knowledge Management International Conference, Langkawi, Malaysia. Armstrong, C. P., & Sambamurthy, V. (1999). Information technology assimilation in firms: The influence of senior leadership and IT infrastructure. Information Systems Research, 10(4), 304‐327. Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation model. Journal of Academy of Marketing Science, 16(1), 74‐94. Bai, R. J., & Lee, G.‐G. (2003). Organizational factors influencing the quality of the IS/IT strategic planning process. Industrial Management & Data Systems, 103(8), 622‐632. Baker, B. (1995). The role of feedback in assessing information systems planning effectiveness. Journal of Strategic Information Systems, 4(1), 61‐80. Bassellier, G., & Benbasat, I. (2004). Business competence of information technology professionals: Conceptual development and influence on IT‐business partnerships. MIS Quarterly, 28(4), 673‐694. Bassellier, G., Benbasat, I., & Reich, B. H. (2003). The influence of business manager's IT competence on championing IT Information Systems Research, 14(4), 317‐336. Basu, V., Hartono, E., Lederer, A. L., & Sethi, V. (2002). The impact of organizational commitment, senior management involvement, and team involvement on strategic information systems planning. Information & Management, 39(6), 513‐524. Boynton, A. C., Zmud, R. W., & Jacobs, G. C. (1994). The influence of IT management practice on IT use in large organizations. MIS Quarterly, 18, 299‐320. Byrd, T. A., Sambamurthy, V., & Zmud, R. W. (1995). An Examination of IT Planning in a Large, Diversified Public Organization. Decision Sciences, 26(1), 49‐73. Byrne, B. M. (2001). Structural Equation Modeling with AMOS: Basic concepts, applications and programming. Mahwah, New Jersey, USA: Lawrence Erlbaum Associates, Publishers. Chi, L., Jones, K. G., Lederer, A. L., Li, P., Newkirk, H. E., & Sethi, V. (2005). Environmental assessment in strategic information systems planning. International Journal of Information Management, 25(3), 253‐269. Clark, C., Clark, J., Gambill, S., & Fielder, B. (2000). Strategic Information Systems Planning Paradoxes. Information Strategy: The Executive's Journal(Fall), 27‐31. Cohen, W. M., & Levinthal, D. A. (1990). Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly(35), 128‐152. Dibella, A., Nevis, E., & Gould, J. (1996). Understanding organizational learning capability. Journal of Management Studies, 33(3), 361‐379. Doherty, N. F., Marples, C. G., & Suhaimi, M. A. (1999). The relative success of alternative approaches to strategic information systems planning: an empirical analysis. The Journal of Strategic Information Systems, 8(3), 263‐283. Droge, C., Claycomb, C., & Germain, R. (2003). Does knowledge mediate the effect of context on performance? Some initial evidence. Decision Sciences, 34(3), 541‐568. Duhan, S. (2007). A capabilities based toolkit for strategic information systems planning in SMEs. International Journal of Information Management, 27, 352‐367. Earl, M. J. (1993). Experience in Strategic Information Systems Planning. MIS Quarterly, 17(1), 1‐24. Fiol, M. C., & Lyles, M. A. (1985). Organizational learning. Academy of Management Review, 10(4), 803‐813. Gefen, D., Straub, D., & Boudreau, M.‐C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communication of AIS, 4(7), 1‐79. Grover, V., & Segars, A. H. (2005). An empirical evaluation of stages of strategic information systems planning: patterns of process design and effectiveness. Information & Management, 42(5), 421‐431. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate Data Analysis (6th ed.). Upper Saddle River, NJ: Pearson Prentice Hall. Huysman, M. H., Fischer, S. J., & Heng, M. S. H. (1994). An organizational learning perspective on information systems planning. 1994, 3(3), 165‐177. JPA. (2008). Senarai Agensi Perkhidmatan Awam pada 1 Januari 2008. Retrieved 16 January 2009. from www.jpa.gov.my. Jantunen, A. (2005). Knowledge processing capabilities and innovative performance: an empirical study. European Journal of Innovation Management, 8(3), 336‐349. Kakabadse, N. K., & Louchart, E. (2006). Consultant's role: a qualitative inquiry from the consultant's perspective. Journal of Management Development, 25(5), 416‐500. Kearns, G. S., & Sabherwal, R. (2006). Strategic alignment between business and information technology: A knowledge‐ based of behaviors, outcome, and consequences. Journal of Management Information Systems, 23(3), 129‐162. Kearns, G. S., & Lederer, A. L. (2004). The impact of industry contextual factors on IT focus and the use of IT for competitive advantage. Information & Management, 41(7), 899‐919. Ko, D.‐G., Kirsch, L. J., & King, W. R. (2005). Antecedents of knowledge transfer from consultants to clients in enterprise system implementations. MIS Quarterly, 29(1), 59‐85. Krejcie, R., & Morgan, D. (1970). Determining sample size for research activities. Educational and Psychological Measurement 30, 607‐610. Kunnathur, A. S., & Shi, Z. (2001). An investigation of strategic information systems planning success in Chinese publicly traded firm. International Journal of Information Management, 21( 6), 423‐439.

Fazidah Abu Bakar, Mohd Adam Suhaimi and Husnayati Hussin Lederer, A. L., & Salmela, H. (1996). Toward a theory of strategic information systems planning. Journal of Strategic Information Systems, 5(3), 237‐253. Lederer, A. L., & Sethi, V. (1996). Key prescriptions of strategic information systems planning. Journal of Management Information Systems., 13(2), 35‐62. Lee, G. G., & Bai, R. J. (2003). Orgnizational mechanisms for successful IS/IT strategic planning in the digital era. Management Decision, 41(1), 32‐42. Lee, G. G., Lin, H.‐F., & Pai, J.‐C. (2005). Influence of environmental and organizational factors on the success of internet‐ based interorganizational systems planning. Internet Research, 15(5), 527‐543. Lee, G. G., & Pai, J. C. (2003). Effect of organization context and inter‐group behaviour on the success of strategic information systems planning: an empirical study. Behaviour & Information Technology, 22(4), 263‐280. Lin, H.‐F. (2006). Interorganizational and organizational determinants of planning effectiveness for Internet‐based interorganizational systems. Information & Management, 43, 423‐433. Luftman, J., & Ben‐Zvi, T. (2010). Key Issues for IT Executives 2009: Difficult Economy's Impact on IT. MIS Quarterly Executive, 9(1), 49‐59. Mentzas, G. (1997). Implementing IS strategy ‐ a team approach. Long Range Planning, 30(1), 84‐95. Mikkelsen, A., & Grønhaug, K. (1999). Measuring Organizational Learning Climate: A Cross‐National Replication and Instrument Validation Study Among Public Sector Employees. Review of Public Personnel Administration, 19(31), 31‐ 44. Nieminen, H. (2005). Organizational receptivity ‐ Understanding the interorganizational learning ability. The Electronic Journal of Knowledge Management 3(2), 107‐118. Nelson, K. M., & Cooprider, J. G. (1996). The contribution of shared knowledge to IS group performance. MIS Quarterly, 409‐429. Newkirk, H. E., & Lederer, A. L. (2006). The effectiveness of strategic information systems planning under environmental uncertainty. Information & Management, 43(4), 481‐501. Otim, S., Grover, V., & Segars, A. H. (2009). The role of Organizational Learning in Strategic Information Systems Planning in Uncertain Environments. In Advances in Managemeni Information Systems (Vol. 12, pp. 233‐256). Pai, J. C. (2006). An empirical study of the relationship between knowledge sharing and IS/IT strategic planning (ISSP). Management Decision, 44(1), 105‐122. Pearlson, K. E., & Saunders, C. S. (2006). Managing & Using Information Systems: A strategic Approach (3rd ed.). New Jersey, USA: John Wiley & Sons. Peppard, J., & Ward, J. (2004). Beyond strategic information systems: towards an IS capability. Journal of Strategic Information Systems, 13(2), 167‐194. Pita, Z., Cheong, F., & Corbitt, B. (2009). Major Issues in SISP: Insights into the main reasons of SISP failure. Paper presented at the 17th European Conference on Information Systems, Verona, Italy. Reponen, T. (1998). The role of learning in information systems planning and implementation. In R. D. Galliers & W. R. J. Baets (Eds.), Information Technology and Organizational Transformation. England: John Wiley & Sons Ltd. Segars, A. H., & Grover, V. (1998). Strategic Information Systems Planning Success: An Investigation of the Construct and its Measurement. MISQ, 22(2), 139‐163. Segars, A. H., Grover, V., & Teng, J. T. C. (1998). Strategic information systems planning: Planning systems dimension, internal coalignment, and implications for planning effectiveness. Decision Science, 29(2), 303‐344. Seneviratne, S. J. (1999). IT and organizational change in the public sector: Idea Group Inc (IGI). Smaltz, D. H., Sambarmuthy, V., & Agarwal, R. (2006). The antecedents of CIO role effectiveness in organizations: An empirical study in the healthcare sector. IEEE Transactions of Engineering Management, 53(2), 207‐222. Teo, T. S. H., & Ang, J. A. K. (2000). How useful are strategic plans for information systems. Behaviour & Information Technology, 19(4), 275‐282. Teo, T. S. H., & Ang, J. A. K. (2001). An examination of major IS planning problem. International Journal of Information Management, 21(6), 457‐470. Teo, T. S. H., Ang, J. A. K., & Pavri, F. N. (1997). The state of strategic IS Planning practice in Singapore. Information & Management, 33(1), 13‐23. Tippins, M. J., & Sohi, R. S. (2003). IT competency and firm performance: Is organizational learning a missing link? Strategic Management Journal, 24, 745‐761. Wang, E. T. G., & Tai, J. C. F. (2003). Factor affecting information systems planning effectiveness: organizational contexts and planning systems dimensions. Information & Management, 40(4), 287‐303. Ward, J., & Peppard, J. (2002). Strategic Planning For Information Systems (3rd Edition ed.). London: John Wiley & Sons. Zahra, S. A., & George, G. (2002). Absorptive Capacity; A review, reconceptualization, and extension. Academy of Management Review, 27(2), 185‐203.

Accessibility and Human Computing Interaction: Engaging and Motivating Students With Disabilities Through More Computer Empowerment Bob Barrett American Public University, Charles Town, USA Docjob00@msn.com Abstract: While many instructors need to focus on the syllabus, subject content, administrative tasks, and students with varying learning styles, they may also need to address various learning style of students with disabilities. Many universities offer teacher training, but their training may not be detailed enough to help instructors be prepared to work with classroom accommodations for students with disabilities. In particular, online (virtual) instructors have another factor to work with in this situation, they have to work with students with disabilities virtually and offer similar or comparable accommodations. Many educational institutions are seeing that more students are enrolling in online programs and courses, and they realize that there may be some additional barriers to learning in terms of this learning environment’s technical process and structure. In particular, students with disabilities are enrolling even more with online courses with the hopes of a barrier‐free environment. Thus, there are still some barriers still present in the learning environment in terms of technical/software application or interaction/communication problems. The purpose of this paper will be to look at how a university can address such problems and develop/create virtual solutions to these barriers by incorporating the help of others in the online community to brainstorm methods of inquiry and build virtual strategies. In particular, there needs to be a special emphasis given to online instructors to become better prepared and trained with technology in terms of structure and how to motivate all types of students, especially students with disabilities, to become more interactive online. Thus, there is a growing need for more human computer interaction, rather than just selecting and clicking single choices. Consequently, universities need to design and develop training programs to help educate and train current and potential instructors in the areas of disability awareness, virtual classroom accommodations for students with disabilities, as well as to create strategies for better learning opportunities for these students. The key focus with this training program would be to create better strategies to help increase better human computing opportunities for all types of students. While many instructors may not be facilitating learning for this particular population at this time, the virtual learning environment is starting to attract more students with disabilities with a growing need to help them become more successful in their educational endeavors. Finally, this presentation will help to offer more insight as to the role and function of instructors in meeting the needs of students with disabilities. Keywords: accessibility, disability, virtual learning, interaction, teacher training, human computing

1. Introduction People with disabilities have long been underrepresented in the workplace and the educational system due to various barriers they have encountered in their lives. While organizations and companies have made strides in employing workers with disabilities as an act of social responsibility, other entities have started to realize the need and value of this untapped human resource (Thakker, 1997). In addition, research has shown that employees with disabilities have low turnover rates, low absenteeism, and high motivation to prove themselves (Fersh & Thomas, 1993). Thus, executives will need to rethink their employment practices in order to compete for employees from the current, shrinking workforce. As the workforce has become smaller due to the retirement of the Baby Boomers, there is a growing need for a more educated workforce to operate the growing amount of technology used in the workforce. While many people have rallied for changes in the way computer software and hardware operate for all people, especially people with disabilities, there are still some concerns for this group’s ability to have more accessibility and need for more human interaction. Traditionally, many employers have relied on the assistance of academic institutions to help provide for them well‐qualified job candidates. However, we need to reexamine whether educational institutions are still able today to enable, educate, and motivate all learners for the many changes in the workforce. In particular, are they able to equip all learners with the necessary technological skills and education to best prepare them for this new workforce outlook? Currently, 54 million Americans with disabilities use information technology (IT) at colleges and universities, which accounts for 20 percent of the population (Oblinger & Ruby, 2004). Consequently, this will help to set the backdrop for this paper in understanding what has been problems for students with disabilities and whether or not if technology can help to break down some of these barriers (whether visible or invisible). For many centuries, education has been focused on the learning of course

Bob Barrett content, but the learning styles of the students have been ignored . As Knowles (1987) noted that “everyone [learners] should be able to participate and control their own learning process.” While most of the academic approaches have been centered on the mastery of course content knowledge, not all learners learn in the same way. As a result, different teaching techniques, strategies, and tools may be needed to help all students acquire, understand, and apply learning gained from the course content. Thus, this leads to a larger question as to whether or not e‐Learning can fill this void in helping to focus on the learning style and skills needed by learners, especially students with disabilities (Barrett, 1999). There are some obstacles or barriers in education that do prevent students with disabilities from succeeding in both the face‐to‐face and online learning environments, especially in certain courses. The focus of this paper will be on how accessibility to e‐Learning, as a method of new computer teaching technology, can be used as a strategic tool to help overcome some of these barriers. In particular, this paper will approach the main research question of “how accessibility and human computer interaction can help to engage and motivate students with disabilities through more computer empowerment” in today’s learning environments?” This newer form of engagement can be used as one way to linking modern day technology to learning content knowledge, as well as offering a more barrier‐free learning environment. As a result, this focus will look at the strategies used at one major U.S. virtual university system and how they are strengthening their use of human computing interaction and best practices to help motivate all students, especially addressing the needs of students with disabilities. Further, this paper will the literature to determine how the need to use e‐Learning as a strategic tool to help engage and enrich the learning opportunities for students with disabilities. It is important that this particular student population be engaged with a variety of e‐ Learning applications to help students with disabilities to become more familiar and empowered to interact more with the computer technology and participate at a higher level in the online learning environment. In particular, the paper will center its focus on the current need to use e‐Learning, or virtual learning, as another mode of instruction. It will also look at this type human computing interaction as a strategic tool for breaking down current educational barriers faced by students with disabilities in educational institutions. In the following section, there will be an overview of the current statistics of people with disabilities in the United States, along with an overview of distance education, which will also be referred to as e‐Learning in this paper.

2. Disability statistics overview Statistics helps one to understand what the current picture of a situation in order to consider what might need to be done in order to change the current outlook. This will provide a baseline for the reader to understand why instructors need to consider why students with disabilities are fighting not only barriers in the workplace, but also in the realm of academia. Before looking at the statistics in the U.S., we need to look at the global perspective. According to United Nations Enable, “The United Nations estimates that 650 million people around the world are disabled, about 10 percent of the world's population.” (para.1). Besides looking at the U.S. statistics, we need to consider a larger global perspectives since many online learning courses enroll students from many global locations. Hudson (2005) reported the following: The incidence of disabilities (and limiting illness) that restricts a person’s ability to function in everyday life, as recorded by government agencies in some countries:

United Kingdom, 18% of the population (National Statistics, 2001).

Australia, 17% of the population (Australian Bureau of Statistics, 2003).

United States, 19.3% of the population (US Census Bureau, 2000).

Canada, 12.1% of the population (Statistics Canada, 2001).

New Zealand, 20% of the population (Statistics New Zealand, 2001).

European Union, across the 15 EU countries in 2001, 19.3% of the population was hampered by physical or mental health problem, illness or disability, with 9.3% severely hampered. (Eurostat, 2003) (Hudson, 2005).

Considering the above statistics, we can see that there are a certain percentage of these global populations that have people with disabilities that can benefit from the use of more accessibility to online learning and computer technology/interaction. According to several resources, the following statistics about the

Bob Barrett employment of people of disabilities was noted: “The total number of people with disabilities aged 16‐64 is 33,153,211, [of which], the total number employed is 18,525,862. The percent of people with disabilities aged 16‐64 employed is 55.8” (Census, 2000, para. 4‐6). These reports show that “18.6 million people disabilities employed aged 16‐64, 60.1% of men with disabilities are employed, and 51.4% of women with disabilities are employed” (Census, 2000, para. 7). Thus, these statistics demonstrate that there is still a disparity between the employment rates of people with disabilities versus their able counterparts. Consequently, one way of changing these employment figures may be the use of technology and how it is offered and facilitated with people with disabilities (Disability Status 2000, Census 2000 Summary File #3; and Census 2000 Brief (March 2003) (C2KBR‐17)). While there is still a disparity between the employment rates of people with disabilities versus their able counterparts, one way of changing these employment figures may be the use of technology and how it is offered and facilitated with people with disabilities. If more students with disabilities focus on obtaining higher education, they may start to make a movement towards online education in light of physical classrooms. In this transition to a new learning venue (environment), they still anticipate the change with some level of apprehension, along with a certain amount of anxiety. Therefore, it is important for online administrative and instructional staff to focus on building virtual trust for these learners.

3. Distance/virtual education in today’s learning environment For the purposes of this paper, the term distance education and virtual learning may be used interchangeably for the same referencing of learning with computer technology in the context of understanding more about the use of technology and human computing interaction. Neal and Miller (2006) defined distance education as “education that takes place independent of location, in contrast to education delivered solely in the classroom, and that may be independent of time as well (para. 4). ASTD, an education/training & development professional organization, noted that “distance education can be characterized as an educational situation in which the instructor and students are separated by time, location, or both. Education or training courses can be delivered to remote locations via synchronous or asynchronous means of instruction (Neal & Miller, 2006, para. 5). Thus, we need to consider what is the role of technology for changing learning and promoting diversity and including more students with disabilities, as well as whether or not this this new technology offer a change for them?

4. Learning and technology advancements in virtual learning environments Technology has made a major emphasis on learning in today’s adult learners. As a result, adult learners must have a different type of skills sets in order to compete in today’s online learning environment and workplace. Academic institutions must design, develop, implement different practices and procedures in order to prepare all students, especially students with disabilities, to compete for quality and meaningful employment. Thus, this paper helps to provide an open forum for the reader and others to determine if there is a need for change. If so, it also provides a chance for further research to be conducted to help examine how various schools of business, both traditional and online, can approach the career development segment of their course offerings and programs. As more technology has become available in many parts of the globe, a new type of student population has emerged. While the traditional student image of higher learning has been somewhat limited in many countries, but given the impact of the Internet, this traditional “student body” has transitioned over to online learning environments (online learning communities). According to Preece (2000), these online communities “consists of people who interact socially as they strive to satisfy their own needs or perform special roles; a shared purpose that provides a reason for the community; policies that guide people’s interactions; and computer systems to support and mediate social interaction and facilitate a sense of togetherness” (p. 10). In the fields of education, business, and management, many educators have recognized the demographical changes of the student population. As a result, technology today has provided a powerful infrastructure, the emerging technologies have allowed educational institutions, educators, and students to achieve education on a much higher playing field – in a virtual learning environment. Thus, more students with disabilities are enrolling in online courses. One reason for this increase in online enrollment is due to online learning environment allowing for more barrier‐free opportunities to education for students with disabilities.

Bob Barrett

5. The role and impact of online instructors in today’s learning environment Over the past two decades, educational reform has started to make changes in the classroom, along with the many technological improvements and changes to learning in general. Educators have started to see instructional methods moving from a sheet of paper to the computer screen. The role and function of these instructors has changed with the introduction and placement of personal computers in the classroom. As more technology to appear in the educational environment, schools started to find that computers had many different qualities. They had a larger storage capacity; were cheaper to purchase; easier software was being developed to navigate; students were starting to learn computers at an earlier age; and adaptation of course materials to the computer environment was being made easier for instructors. In light of these many technological advances, schools systems still had to face other problems due to poor planning and constraining budgets. As the number of computers was limited in the classroom, it also had an impact on the number of users with the computers. In addition to budgetary problems, sometimes there was a limited use of software licenses due to the amount of budgetary investment. This limited training given to some instructors did impact how many students would receive quality technological instruction. While the focus on the instructor may be of a primary importance, we have to consider another key problem. Not all of this type of technology has been fully accessible for students with disabilities. Finally, not all instructors were trained to teach students with certain types of disabilities wishing to learn and operate computers and various computer software packages. These problems were only a representation of why some school systems sought additional funding to increase e‐Learning opportunities for all students. We need to consider the key fact here that E‐learning is not for all students. For students with disabilities, these many facets or approaches that e‐Learning offered more opportunities than ever before. For example, visual learners were able to benefit from applications in PowerPoint and Flash Multi‐Media technology. Auditory learners could benefit from online classrooms with auditory lectures, Podcasts for students, as well as live chats. From a blended‐approach perspective, some online programs offer both auditory lectures, as well as PowerPoint slide presentations. Also, live chats (both auditory and visual – i.e., Elluminate, Horizon Wimba, etc) offer more opportunities for a variety of learners. Thus, this leads us to another question for consideration. How can we train instructors to become more accessible with technology and help bridge the learning gap for new online learners, especially students with disabilities with assistive technology requests? First, we have to motivate our instructors to become more “computer literate” and want to use more technology in the classroom. Second, we need to show these instructors how to use various types of technology to achieve learning objectives in the online learning environment. Finally, we need to help instructors to learn different ways, approaches, techniques, and strategies to help motivate students with disabilities to use these new technologies and use it for more interaction. While not all instructors are educated or trained to work with students with disabilities, the use of technology can be quite helpful in this educational mission in order to bring this student population and instructor (and overall class) closer together during the learning process. During this time, it is important to focus on the building of trust to help engage the student with disability and to help encourage them to take advantage of this new technology.

6. Building trust during the learning process It is important to focus on establishing and nurturing trust early on with the learning process, it can help all stakeholders involved in a learning organization to move forward rather than “freezing” or “unfreezing” during critical moments in their development, as noted by Lewin (Kaminski, 2011). However, not all instructors are willing to spend additional time and resources for trust, but later learn the importance of this key strategic element. We can see that the concept of trust has many meanings, but for this paper there will be a focus on trust in the context of swift trust. ChangingMinds.org (2010) focused on swift trust and how it affects certain groups and organizations. They stated that “Sometimes there is no time to build a trusting relationship, such as when group of people are thrown together and must start work immediately” (para. 4). As people are thrown together into a temporary setting, they may quickly organize and develop a temporary system in order to function as a group and meet their immediate needs (i.e., job duties, tasks, etc.). While in some educational institutions classes are large or a variety of learners are placed in learning environments with barriers, the use of online learning has helped to change some of these learning impediments and help encourage more people to return back to education.

Bob Barrett

7. Course preparation and strategic planning Can instructors find other ways to motivate students to seek additional resources and ways of preparing course assignments and projects during the learning process? Why is this important to the students and instructors? While an instructor’s key goal is to teach the course content, what happens if he or she also motivates the students along the way to want to discover and inquire more on their own? Therefore, if an instructor wants to carry the same momentum from the live classroom setting into the world of online learning, there are several areas to consider, as noted below. Also, during the instructional period, the instructor needs to determine the level and extent that all students achieve in terms of mastering the learning objectives for the course. One way that online instructors can help changed the level of learning, quality of instructions, and adapting the course material for all types of learning styles is by assisting with the creating and design of the learning course. It should be noted that not all online schools permit instructors to design and develop courses, but many do encourage the use of additional teaching and learning materials. Zhu, Payette, DeZure (2006) wrote about several areas of consideration when one is creating an online course: 1) course content; 2) delivery of instruction; 3) communication and interaction; 4) student time spent on learning tasks; and 5) assessment of student learning. While the course content may remain the same, but the delivery of instruction may be given in either synchronous or asynchronous learning formats. Instead of live chat discussion in a physical classroom, the discussion/dialogue could be done in a chat session, discussion thread, and/or video session. Each student’s participation in the course would depend on the learning format, as designed. Finally, selected assessment activities of student learning can be done in terms of live chats, discussion threads, assignments, quizzes/exams, and/or projects. In order to consider such items within a course design, one needs to look at key areas to incorporate each of these items.

8. Impact of teaching strategies, tools, and techniques Not all teaching tools work or apply in all given learning environments. However, as some teaching tools may be viewed effective in one learning environment, they may not be as successful in another type of learning situation. Therefore, since each environment is unique, as well as the learners in it, the educator needs to assess their online environment and determine if change is necessary currently or for later. However, not all educators may be as flexible in their teaching method, and they may not be willing to change. This leads us to the next question for exploration. Do educators incorporate different teaching strategies and techniques to meet the ever‐changing needs of these virtual learners in terms of learning from their cultural differences in order to enhance the learning experiences of all? Further, can these instructors also address the changing learning needs of all adult learners, especially people with disabilities? Therefore, this type of learning environment has helped to break down a few of the barriers as previously discussed in this paper. Also, this new type of learning environment has helped to “level” the playing field, in which students can act, react, and be proactive in the learning process. White (2002) noted “Nowhere is thinking more evident than in the textual environment of the online classroom. If writing is thinking, then online students display their thinking throughout the course, illustrating their individual styles and changing attitudes” (p. 6). Consequently, educators can incorporate various strategies to help draw upon the experiences of all class members – rather than just a select few. This helps to demonstrate the beauty of online learning – because online learning is a continuous process (not limited to a set time and place as a traditional course is scheduled). The creativity and innovative ways of teaching must also continue to change. Educators and course designers need to enhance and modify their online programs. While many educators will agree that learning concepts, principles, and theories are necessary in establishing a good foundation of learning in any course, this may not be enough to reach all of the learning needs of today’s online learners. While these educators may have a challenging position of motivating and keeping the attention of young learners, we can also see the same problem with adult learners. It is important to note here that the challenges that online instructors may face in motivating and keeping the attention of virtual learners when there are issues of differences in generational learning, time zone challenges, technology skills, and perceptions of course value. As we look at the various evolutionary steps of technology in today’s learning environment, we can see how it has influenced the diminishing workforce. If we look at the impact of changes in the workplace, we can see some impacts as a result. This may have caused many human resource professionals to search for information and ideas on best practices in human capital management (Fitz‐enz, 1997). Thus, these changes in

Bob Barrett technologies have occurred in the workplace, as well as changes in the various federal, state, and local laws, and there has been a shift from these changes to human problems, especially behaviorally, one can see on the individual group, and organization levels. Chin and Benne (1969) discussed this shift to the human problems as “dealing with the resistance, anxieties, threats to morale, conflicts, disrupted interpersonal communications, and so on, which prospective changes in patterns of practice evoke in the people affected by the change” (p. 94). Therefore, those individuals involved in technological and human asset management work as “change agents” to help facilitate change in terms of best practices. Fitz‐enz (1997) described best practices as: “an enduring commitment to a set of basic beliefs, traits, and operating stratagems. These are the guidelines for an organization: the driving forces that distinguish it from all others” (p. 98). In early 1990, The Saratoga Institute conducted a formal research project on common traits of the best human asset management companies. The question that they used over and over with each participant was, “Who is good at _______?” The identified eight factors that appeared constantly in their study as: “1) value; 2) commitment; 3) culture; 4) communication; 5) partnering; 6) collaboration; 7) innovation and risk; and 8) competitive passion” (Fitz‐enz, 1997, p. 100). This Institute wanted to find out what was being done better in each of these factor areas. The study was to help them to identify what was being done right and if it was part of the individual’s interaction with their organizational culture. Thus, as many people have researched and interviewed others to learn more about best practices, the educational arena has had to work even hard to develop, create, implement, and share best practices with others in order to help improve upon the current approach to online learning and teaching. While we can many reasons why most online instructors may have to “rethink” and “reevaluate” their teaching approaches, strategies, and techniques, the fact remains that students with disabilities need a different approach by their instructors to help motivate and encourage them to use this new technology in order to improve their possibilities for equal and better educational opportunities.

9. Conclusion In consideration of the key points noted in this paper, we have to consider several final questions. How and when do instructors start to help motivate students with disabilities with the use of technology? Do they hope that technology will provide the necessary motivation or should they be more socially present to help ensure the motivation is there and continues throughout the learning process? While overviewing the concepts of disability, learning, and technology, we do need to think further about the role and function of e‐learning for the student with disabilities, as well as the instructor. Can E‐Learning be used as a strategic tool for students with disabilities and encourage them to become more interactive with the course technology and achieve even higher levels of course interaction in terms of participation and use of computer technology? Yes, they can, but educational institutions may need to rethink how they present material and in what types of software applications they may offer their students to help motivate them to become more “involved” with the course, course members, and instructor. Further, e‐Learning has provided many unique and creative opportunities for instructors and learners to learn and grow from each other’s personal and professional experiences. As more students with disabilities enroll in online programs, more educational institutions are noting a growing need to train more instructors to learn additional computer technology, increase teacher educational offerings, as well as helping them to learn new methods and strategies for motivating online student participation and human computing interaction.

References Barrett, B. (1999). Best disability employment practices: A case study. Dissertation. The George Washington University, Washington, DC, (UMI). Chin, R., & Benne, K.D. (1969). General strategies for effecting changes in human systems. In W.G. Bennis (1969). The planning of change. New York, NY: Holt, Rinehart and Winston. Changing Minds (2011). Retrieved on October 1, 2011 from http://changingminds.org/disciplines/change_management/lewin_change/lewin_change.htm. Disability Status 2000, Census 2000 Summary File #3; and Census 2000 Brief (March 2003) (C2KBR‐17) Retrieved on March 20, 2008 from http://www.dol.gov/dolfaq/go‐dol‐ faq.asp?faqid=66&faqsub=Statistics&faqtop=People+with+Disabilities&topicid=11. Fersh, D., Thomas, P.W. (1993). Complying with the Americans with disabilities act: A guidebook for management with people with disabilities. Westport, CT: Quorum Books. Fitz‐enz, J. (1997). The truth about best practices: What they are and how to apply them. Human resource management, 36(1), 97‐103.

Bob Barrett Hudson, R. (2005). Web usability – disabilities and technologies. Retrieved Dec. 1, 2012 from http://usability.com.au/2005/10/disabilities‐and‐technologies‐2005/. Kaminski, J. (2011). Theory applied to informatics: Lewin’s change theory. Canadian Journal of Informatics, 6 (1). Knowles, M.S. (1987). Adult learning. In R.L. Craig (ed.). Training development handbook. New York: McGraw‐Hill, pp. 168‐ rd 179 (3 ed.). Neal, L. & Miller D. (2006). The basics of e‐learning: An excerpt from Handbook of Human Factors in Web Design, para. 4‐5. In R.W. Proctor & K.L. Vu (2004), Handbook of Human Factors in Web Design, Lawrence Erlbaum Associates. Retrieved March 19, 2008 from http://www.elearnmag.org/subpage.cfm?section=tutorials&article=20‐1. Oblinger, D. & Ruby, L. (2004, January). Accessible technology; Opening doors for disabled students. Retrieved March 1, 2008 from http://www.nacubo.org/x2074.xml. Preece, J. (2000). Online communities: Designing usability, supporting sociability. Chichester: Wiley. Thakker, D (1997). Employers and the Americans with Disabilities Act: Factors Influencing Manager adherence with the ADA, with special reference to individuals with psychiatric disabilities. Dissertations Abstracts International, ( ). (University Microfilms No. 9727300). Timesonline (Dec. 4, 2012). Republican opposition downs UN disability treaty. para. 14. Retrieved Dec. 5, 2012 from http://www.timesonline.com/news/politics/republican‐opposition‐downs‐un‐disability‐treaty/article_37e2c544‐ 3200‐5845‐934f‐40234ec66178.html?photo=0. United Nations Enable (2010). Factsheet on persons with disabilities. Retrieved December 15, 2012 from http://www.un.org/disabilities/default.asp?id=18. White, K.W., Weight, B.H. (2000). The online teaching guide: A handbook of attitudes, strategies, and techniques for the virtual classroom. Needham Heights, MA: Allyn & Bacon. Zhu, E., Payette, P., & DeZure, D. (2006). An introduction to teaching online. CRLT Occasional Papers. Center for Research on Learning and Teaching, University of Michigan. No. 18.

Achieving Improved Usability Outcomes With Different Usability Practitioner Roles Vince Bruno and Martin Dick School of Business Information Technology and Logistics, RMIT University, Melbourne, Australia vince.bruno@rmit.edu.au martin.dick@rmit.edu.au Abstract: This paper examines two sets of usability roles: the consultant vs the organisation based practitioner and the usability manager vs. the usability practitioner. This paper will review the current literature discussion of the usability practitioner’s role and present findings from interviews with industry practitioners. This research interviewed twenty one usability practitioners with five or more year’s industry experience. The interview transcripts were then analysed using the grounded theory methodology. The analysis provided various findings which highlighted concepts that impacted on the usability outcome of an IS project. The analysis produced twenty seven concepts which were formed into four themes, which included usability mindset, collaborative approach, usability practice and project constraints. This paper will focus on the noticeable differences between the various usability roles in relation to the resulting twenty seven concepts. The key findings show that usability managers look to strategic usability issues, by improving stakeholder collaboration and need to focusing on the skillsets of the usability practitioners. Consultants had a higher tendency to focus on usability activities compliance within a process, selecting and performing activities based on constraints (which include technological constraints, time and budget constraints), needed to have a degree of flexibility in their usability practice and often were used to validate usability practices in an organisation. Organisational practitioners were more focused on nurturing and educating usability understanding within the organisation and stakeholders involved in an IS project. A usability practitioner needs to be flexible enough to adapt to the situation they find themselves in when engaged in an IS project in order to maximise usability outcomes. Keywords: usability, practitioner, role, industry, usability outcome

1. Introduction The concept of usability (Bruno and Al‐Qaimari 2004) and the range of usability activities (Ivory 2001) available to be performed have been in use for many years. The ICT industry, organisations, and the users have not matured enough in many instances to embrace usability as an important part of the development process. The credibility and benefits of usability can often suffer with the lack of usability maturity (Jokela, Siponen et al. 2006) within project teams and organisations. The usability literature has a strong focus on performing individual usability activities. The effective performance of these usability activities is clearly important in achieving a successful usability outcome. However, it is necessary to ask, is it sufficient to ensure a successful usability outcome for a project just by performing these activities correctly. The gap in usability knowledge between research and industry practice is well documented in the literature (Parush 2006). It is important to understand what the critical issues in industry that impact the success of usability outcomes. It is also important for the usability research literature to reflect practice and add value to practice. The industry practice can inform the literature and help give it focus. This paper provides insight into the practice of usability practitioners, highlighting key concepts that can impact on a usability outcome. The focus of this paper is to examine the role played by usability practitioners and how this can impact the usability outcome of a project. The data used is from usability practitioner interviews that focused on what enables a good usability outcome to be attained in a project. This paper highlights the higher level roles of a usability practitioner, i.e. Managing or Practice, and organisational vs consultant based practitioners. Section 3 will examine some of the current literature relating to usability practitioner research on usability outcomes. Section 4 will describe the methodology being used to gather and analyse the data. Section 5 will describe some of the findings of this research, while Section 6 will look at the analysis done.

2. Literature review The research literature does describe various aspects to the role of a usability practitioner in practice of usability activities in an ICT project. Usability practitioners have been given many titles in industry practice, like usability practitioner, human factors specialist, user experience expert… etc. The title could include the management role. Iivari (2006) discusses various usability practitioner roles within an engagement in a project.

Vince Bruno and Martin Dick This looks at the predominant activities performed by usability practitioners when engaged to perform usability. These include: Informative role, Consultative role, Participative role, and Configurer role. The literature covers a lot of the skill and practice orientated aspects of usability practitioner’s role. The role of improving usability understanding to promote usability adoption (Robertson and Hewlett 2004; Seffah, Desmarais et al. 2005). The need to evangelise usability (Dayton 1993; Levi, Melo et al. 2007) to sell and convince project stakeholders of its importance, these include organisational stakeholders (Bloomer and Croft 1997). Promoting of the value of usability during usability activities (Bernhaupt and Weiss 2007), as something done to introduce usability (Levi, Melo et al. 2007) where usability maturity is low (Nielsen 2005). The need for a usability champion during early stages of introducing usability in a low usability maturity organisation (Nielsen 2005) to help evangelise usability. The role performed by usability practitioners is often not valued and lacks credibility with the IS project team (Rosenbaum, Bloomer et al. 1999; Sherman 2006; Wilson 2007). Often when integrating usability into an organisation credibility needs to be attained (Fellenz 1997; Mayhew 1999). Various surveys (Gulliksen, Boivie et al. 2004; Ji and Yun 2006) have also produced some key findings that improving usability’s credibility and that the usability maturity of organisations needs to be increased. The role of a usability practitioner is continuously changing (Wilson 2005). As the usability practitioner becomes more involved in projects and with organisations, and the usability maturity improves across the industry, it is inevitable that the role will change. As discussed at the start of this section, there are various communities of practice (Iivari 2005) for usability practitioners, which highlight the changing role of usability practitioners. The need for a usability practitioner role has been identified as important, i.e. Boiver et al. (2006) believes ‘that usability issues require a “specialist” role’. Having usability practitioners whose role and responsibility lies with usability can ensure the performance of usability activities throughout a development process. Ferrara (2005) highlight that often a usability practitioners practical role is given to other ICT project team members on top of their other role in the project. This paper looks beyond these practice orientated roles that can improve the usability outcomes of ICT projects. This paper looks to the established usability practitioner roles within ICT projects. It examines the differences between organisational vs consultant roles and managing usability vs performing usability.

3. Research methodology This research paper will attempt to answer the following research question: “How do the organisation roles played by usability practitioners impact on their perceptions of how to improve usability outcomes in IS projects?” This question arose while examining data that was elicited from current industry usability practitioners, where the questioning was focused on the usability outcome attained in projects. The usability practitioners interviewed had a minimum 5 years’ experience in the usability industry. During each interview four open questions were asked:

Describe academic background, and work experience of practitioner

Describe a typical day’s work as a usability practitioner.

Describe a project, that you were involved in, where usability had a good impact on the outcome.

Describe a project, that you were involved in, where usability did not have a significant impact on the outcome.

Participants were recruited using various means. This research started with an email on the CHISIG (2011) mailing list (an Australian mailing list for usability practitioners), which provided many volunteers. Various locally based usability consultancies companies were also contacted. The practitioners interviewed also provided other contacts of usability practitioners. This research will continue to perform interviews until saturation of data is attained, i.e. when no significantly new concepts are appearing in the analysis. Ethics approval was obtained for the audio recording of each interview session. Each interview was then transcribed into text and loaded into a qualitative analysis tool, i.e. Nvivo7 (QSRInternational 2011). The tool facilitated the “coding” of interesting concepts in each transcript, things that may be significant for the general area of this research question and warrant more analysis.

Vince Bruno and Martin Dick The research process employed was Eisenhardt (1989), which specifies eight stages, see Figure 1. The research methodology being used to analyse this rich source of data is grounded theory (Strauss and Corbin 1990). This methodology provides a mechanism to iteratively gather and analyse data and build a set of concepts. This grounded theory analysis stages have been superimposed onto the Eisenhardt research process that allows for an iterative interpretivist analysis of the primary data to derive a theory.

Figure 1: Eisenhardt (1989) theory building process combined with the grounded theory analytical process (Strauss and Corbin 1990) The dotted ellipses (i.e. (a), (b) and (c)) indicate the iterative loops performed during the analysis phases of the research process. This paper takes up the research process, for an additional set of analysis iterations to shape a different hypothesis (Axial Coding) and then generate theory (selective coding) which is enfolded with related existing literature. Initial results were published (Bruno and Dick 2007) and the final results will be published shortly. This paper is looking to describe a separate phenomenon that fell outside the scope of the primary research. This paper looks to answer a different question, as specified earlier, to the initial project research question.

4. Findings This section will provide a summary of the concepts that have emerged from the initial research (Bruno 2011). These findings provide the basis for the additional analysis done in relation to these separate phenomena that this paper is presenting.

4.1 Key themes Four major themes were highlighted in this research, which include “Collaborative Approach” (C), “Usability Mindset” (M), “Usability Practice” (U) and “Project Constraints” (P). These high level themes were derived from grouping emergent concepts (discussed in section 4.2). Analysis of these themes has highlighted various relationships between them. Figure 2 shows the major themes and relationships on which this papers analysis is based. The relationship emerged between one more or more concepts between themes or within themes. All relationships shown were significantly strong in this theory, both between themes and within themes. The key finding of this framework is the importance of the relationships between the collaborative approach and usability mindset themes. The relationships “project shared vision” and “prevailing usability mindset” highlighted a very strong correlation between the concepts in the two themes involved in these relationships. This strong relationship is the key outcomes for the research.

Vince Bruno and Martin Dick

Figure 2: Major themes and relationships that influence the usability outcome for projects

4.2 Emergent concepts There are twenty‐seven emergent concepts that makeup the four themes described in the previous section. These concepts highlight various issues or considerations that were highlighted by usability practitioners when describing their involvement in performing usability in a development process. Table 1: List of twenty seven concept that impact on the usability outcome of an IS project C1 C2 C3 C4 C5 M1 M2 M3 M4 M5 M6 M7 P1 P2 P3 P4 P5 P6 U1 U2 U3 U4 U5 U6 U7 U8 U9

Establish a shared usability vision using collaboration Involvement by all project stakeholders enhances the collaborative approach Crucial involvement by IS project team members Senior organisational stakeholder involvement Project stakeholder relationships must be fostered Create and follow the usability requirements Nurture usability understanding Making usability real to create a shared vision for project stakeholders Project decisions embrace a usability mindset Usability goals promotes a usability mindset Usability maturity requires transformation of the organisational culture Usability activities involvement enhances usability mindset Usability activities compliance within a project lifecycle Constraints dictate usability activity selection & performance Technological constraints Allocating resources to usability activities Organisational constraints external to project When usability is initiated Demonstrate value in engaging usability practitioners Measuring usability goals Maintain flexibility with usability practice Managing stakeholder involvement Evangelising usability to project stakeholders Skillset and experience of usability practitioner Validation of usability practice Usability team practices Usability education of project stakeholders

Vince Bruno and Martin Dick This analysis that follows is based on these findings, information on these concepts and themes please refer to the thesis (Bruno 2011) on which it is based.

5. Analysis This analysis looked at comparing what was said by practitioners in various roles across the 27 usability concepts outlined in findings. This will help understand, based on the role, what is more important or possible for a usability practitioner to focus on during their engagement in IS projects The practitioner roles highlighted by this research include:

Organisational based usability practitioners

Usability Consultants

Usability Managers

Usability Practitioners

This research examines a number of comparisons to highlight key differences among the concepts based on the usability practitioners role, which will be discussed in the following sections.

5.1 Usability manage vs usability practitioner When examining the interview data, each usability practitioner was coded as either a usability manager or a usability practitioner. The usability managers (twelve practitioners) were those who headed a team of usability practitioners, and whose role included managing and allocating usability resources to projects. The managers also, if in a consultancy company, may be looking for the next usability engagement, which requires usability selling. The usability practitioners (nine practitioners) were those who did not have usability manager responsibilities, and were focused on the performance of usability activities in projects, either as a usability consultant or an organisationally based usability practitioner. Table 2: Usability manager vs usability practitioner, percentage of interviewees that discussed concept Concepts Crucial involvement by IS project team members Skillset and experience of usability practitioner

Manager 100% 75%

Practitioner 55.6% 44.4%

There are two concepts that had significant (more than 30%) difference in the number of usability managers and usability practitioners that refers to it, as shown in Table 2. Both usability managers and usability practitioner discussed the importance of involving the IS project team members in the concept “Crucial involvement by IS project team members”. All twelve usability managers discussed the importance having the IS project team involved and understanding the usability value. It was predominantly discussed in the typical day (4 usability managers), good story (6 usability managers and 3 usability practitioner) and bad story (5 usability managers and 1 usability practitioner) during the interviews. In the bad stories, the IS members involvement was low, usability ownership was questioned, vendors were part of the project and resisted involvement, understanding of the usability value was low and technological constraints were not understood. This was identified and discussed predominantly by the usability managers. In the good stories, IS project team members were involved, through observation of usability activities, or sharing the usability journey with usability practitioners, or being skilled up as usability practitioners by mentoring to be groomed as usability champions for project. Both usability managers and practitioners discussed this. The concept “Skillset and experience of usability practitioner” describes the skills and experience required to achieve good usability activity outcomes. The usability managers, with their role as allocating usability resource, saw this as a crucial consideration when resourcing usability for a project. Being able to provide usability practitioners with appropriate skills and experience to provide usability value to a project given the project constraints. This concept was part of an internal relationship in the usability practice theme, which related to the concept about working in usability teams. The usability manager’s focus on these two concepts is clearly important, skills, experience and working in teams are key determinants used to help allocate usability resources to projects.

Vince Bruno and Martin Dick In summary, the usability managers see the importance of usability practitioners having a good skillset and experience base when engaged to perform usability in a project. Where needed, usability mentoring is a great way to improve usability practitioner skills. Usability managers are also more aware of problems with involvement, especially the need to get involvement from the IS project team members, to bring the technological issues (constraints) into consideration. The examination of the other concepts found little different between number of managers and practitioners that discussed them.

5.2 Usability consultant vs. organisational based practitioners The comparison of usability consultants (eight practitioners) and organisational‐based practitioners (seven practitioners), presents an interesting set of differences that may impact the usability outcome. There are six usability practitioners that had been in both roles, which for purposes of highlighting the concepts that have a significant difference (more than 30% of practitioners) have been ignored in this initial analysis, see Table 3. Table 3: Usability consultants and organisational usability practitioners, percentage of interviewees that discussed a concept Concepts Usability activities compliance within process Constraints dictate usability activity selection & performance Technological constraints Time given and Cost to usability activities Nurture usability understanding Education of project stakeholders Maintain flexibility with usability practice Validation and Limitation to Usability practice

Consultant 75% 100% 50% 100% 62.5% 37.5% 75% 50%

Organisational 42.9% 57% 14.3% 57.1% 100% 85.7% 42.9% 0%

Organisational‐based usability practitioners did not discuss working within the organisation process as being problematic, because they are working within the same organisation structure and have a clear understanding of what is possible. The selection and performance of usability activities, and being flexible with performance of usability was not a highly discussed concept. The technological constraints are less of an issue, because working within the organisation they can have conversations and discussion to gain an understanding of what is possible with the appropriate IS project team stakeholders. The time and budget for a project can be discussed at the inception of a project when involved from the start where within an organisation is more likely. The nurturing of usability understanding and performing usability education can be done, when involved in the organisation because the usability practitioner is available from day‐to‐day to have conversations with the various project stakeholders. Working within the organisations provides an opportunity to take the usability mindset beyond the project to the organisations culture, to improve the usability maturity. Validation of usability findings by external usability consultants and limitations encountered by usability consultants were not discussed. The usability consultants find themselves engaged into the project lifecycle at the specific points where their expertise is needed, which assumes a high level of usability maturity in the organisation and the project stakeholders. Project constraints have a huge bearing on what can be selected and performed as usability activities for project. Technological constraints have a larger impact due to the time and access needed to foster relationships with IS project team members. The time given and budget allocated also provide limitations on what can be done because often they are not engaged at the start of a project to have an impact on the project plan. The usability education of stakeholders is limited by their short‐term engagements. The generation of usability documents is the main deliverable for their engagements, they are not judged on the overall usability outcome for a project. Due to the project constraints they find it difficult to be flexible with the performance of usability activities so they can maximise the usability findings generated. Often usability consultants will be asked to validate usability findings generated by internal organisation usability practitioners. They are more likely to encounter limitations that are out of their realm of control, in relation to the access to stakeholders (involvement), organisations usability maturity and organisational constraints (like legal issues or politics). This discussion shows that these concepts can vary significantly in relation to the performance of usability activities by a usability consultant or an organisationally based practitioner. The main differences include the following topics:

Vince Bruno and Martin Dick

Level of access to project stakeholders

Time when usability is initiated in a project

Involvement in project plan at start of project

Opportunity to change the usability mindset for project or organisation (beyond usability activities and beyond project)

Traction with mitigating project constraints

Selection and performance of usability activities

There are other differences between these two types of usability practitioners. For example, 100% of the usability consultants found the time and cost constraint concept along with the constraints dictating the selection and performance of usability as important concepts for consideration when engaged in a project to perform usability, where just over 50% of organisationally based usability practitioner said this. On the other hand, 100% of organisational‐based usability practitioners discussed the opportunities to help nurture usability understanding to project stakeholders, where just 60% of usability consultants discussed this as important. The type of engagement a usability practitioner has with the project, as a consultant or an organisational‐based practitioner, does present various constraints and relationship limitations in the performance of usability activities. The examination of the other concepts found little different between number of organisational‐ based and consultants that discussed them. This section would suggest, based on the discussions, that organisational‐based usability practitioners have a better opportunity to have a beneficial impact the project’s usability outcome. Usability consultants are reliant on the usability maturity of the organisation, without it they’re impact on the project’s usability outcome can be significantly stifled by the various concepts discussed in this section. The usability practitioners (mixed) that had been both organisational‐based and consultant practitioners discussed most of the concepts discussed by both organisational‐based and usability consultants. There was no value in looking at differences between mixed and organisation‐based or mixed and usability consultants. It does not provide any significant differences.

5.3 Success and failure concepts This section will examine the success and failure orientated concepts in relation to the usability practitioner roles. Table 4 shows the six top concepts which were discussed during a good story provided by usability practitioners interviewed. Table 5 provides the top six concepts that had an impact on the usability income during the bad story discussed. This section will compare these against the roles of the practitioner. Table 4: Success factors drawn from key concepts emergent from good story discussion Success Factors Involvement by all project stakeholders enhances the collaborative approach Project stakeholder relationships must be fostered Making usability real to create a shared vision for project stakeholders Usability goals promote a usability mindset Crucial involvement by IS project team members Nurture usability understanding

Table 5: Failure factors drawn from key concepts emergent from bad story discussion Failure Factors Involvement by all project stakeholders enhances the collaborative approach Organisational constraints external to project Usability goals promote a usability mindset Technological constraints Create and follow the usability requirements Project stakeholder relationships must be fostered

When examining the top concepts for organisational‐based usability practitioners in comparison to the usability consultants, there are some interesting differences. Table 6 shows the four concepts that were

Vince Bruno and Martin Dick common to both types of usability practitioners. These concepts are the major concepts that also appeared in the success and failure concepts (tables above, i.e. Table 4 and Table 5), except “Organisational constraints external to project” which only appeared in detrimental key concept list. Table 6: Success and failure common key concepts for organisational‐based vs usability consultants Concept Usability goals promotes a usability mindset Involvement by all project stakeholders enhances the collaborative approach Organisational constraints external to project Project stakeholder relationships must be fostered

Organi‐ sational Y Y

Consult‐ ant Y Y

Success

Failure

Y Y

The interesting concepts are those that are different between organisational‐based usability practitioners and usability consultants (see Table 7). These differences seem to suggest that usability consultants look to make usability issues real to project stakeholders. In order to gain credibility with project stakeholders they look to eliciting and measure usability goals and provide more quantitative measures with their deliverables. Their skillset has been applied to multi‐domains and experience across a broader range of domains and stakeholders that enable a more flexible approach to performing usability in a project lifecycle that can maximise usability findings potential. Table 7: Differences in top concepts for organisational‐base vs usability consultants Usability Consultants Making usability real to create a shared vision for project stakeholders Measuring usability goals Skillset and experience of usability practitioner

Organisational‐based usability practitioner Usability team practices Nurture usability understanding Usability maturity requires transformation of the organisational culture

Organisational‐based usability practitioners look to nurture usability understanding of project stakeholders. They look to create a usability mindset by improving the usability maturity of project stakeholder and the organisation. They also look to work in usability teams, of at least two people, to improve usability findings, and to allow for enough resources to perform usability activities and maximise their outcomes. The usability team approach allows for improved mentoring and education of project stakeholders to aide in the transformation of organisational usability culture and maturity. The reasons for the different concepts would be based on the following: the length of an engagement, when engaged in project lifecycle, time given and monies allocated to usability. Usability consultants would be engaged for a specific set of activities, more often than not, at a particular point in the project lifecycle, which highlights the need for making usability issues real to project stakeholder and eliciting quantitative usability measures for deliverables. Organisational‐based practitioners can potentially be involved from start of the project (till the end). This allows them from within the organisation to have the conversations that can nurture usability understanding and develop a usability mindset for the project. The six usability practitioners interviewed that had been both an organisationally based practitioners and usability consultant also discussed the same four concepts in common with success and failure concept lists (shown in Table 6). In addition, they all described the importance of trying to transform the organisations usability culture and maturity that was discussed by organisationally based practitioners. They also discussed the important of involving IS project team members in usability activities like it was mentioned in the success concept list (Table 4). Table 8: Key concepts discussed by interviewees that were both organisational‐based and consultants (mixed) Mixed usability consultant key concepts

Mixed

Usability goals promotes a usability mindset Project stakeholder relationships must be fostered Organisational constraints external to project Usability maturity requires transformation of the organisational culture Crucial involvement by IT project team members Involvement by all project stakeholders enhances the collaborative approach

Con‐ sultant Y Y Y

Success

Failure

Y Y Y Y

Organ‐ isational Y Y Y Y

Y Y

Y Y Y

Y Y

Vince Bruno and Martin Dick These mixed usability practitioners, corroborated core concepts that are of importance to the beneficial and detrimental list of key concepts lists. These also provided the key elements of the organisational‐based versus consultant key concept list. This in essence reinforces the importance of involvement with usability by all project stakeholders, especially IS project‐team members. It promotes the importance of developing and fostering project stakeholder relationships for a project to enabled the promotion of usability goals and an overall usability mindset for a project, where conflicting goals have been concorded in consultation with project stakeholders. The mixed usability practitioner’s list of concepts was associated more closely with the organisational‐based practitioners and the beneficial outcome list of concepts.

6. Conclusion This analysis presents the impact on usability outcomes through the lens of a usability practitioner’s role within a project, i.e. organisational based or consultant engagement, usability manager or usability practitioner. This highlights the potential difference of practice in these roles, but also highlights what concepts become more prominent and/or important. This ultimately can be used, based on the type of engagement with a project a usability practitioner has, as a list of concepts that can guide practice and lead to an improved usability outcome. The various tables presented (in Section 6) provide this guide to help understand the impact on the usability outcome based on the perspective/role played by usability practitioners in an ICT project.

References Bernhaupt, R. and A. Weiss (2007). How to Convince Stakeholders to take up Usability Evaluation Results? Downstream Utility 2007 : The Good, the Bad, and the Utterly Useless Usability Evaluation Feedback., Toulouse, France, Institute of Research in Informatics of Toulouse (IRIT) / IRIT Press ‐ Toulouse, France. Bloomer, S. and R. Croft (1997). "Pitching usability to your organization." interactions 4(6): 18‐26. Boivie, I., J. Gulliksen, et al. (2006). "The lonesome cowboy: A study of the usability designer role in systems development." Interacting with Computers 18(4): 601‐634. Bruno, V. (2011). Improving Usability Outcomes in IS Projects: the Views of Usability Practitioners. School of Business Information Technology and Logistics. Melbourne. RMIT University. Bruno, V. and G. Al‐Qaimari (2004). Usability Attributes: An Initial Step Toward Effective User‐Centred Development. OZCHI, Wollongong, Australia. Bruno, V. and M. Dick (2007). Making usability work in industry: an Australian practitioner perspective. Proceedings of the 2007 conference of the computer‐human interaction special interest group (CHISIG) of Australia on Computer‐human interaction: design: activities, artifacts and environments, Adelaide, ACM. CHISIG. (2011). "Computer‐Human Interaction Special Interest Group (CHISIG)." Retrieved 22 April, 2011, from http://www.chisig.org/. Dayton, T. (1993). "Skills needed by user‐centered design practitioners in real software development environments: report on the CHI'92 workshop." ACM SIGCHI Bulletin 25(3): 16‐31. Eisenhardt, K. M. (1989). "Building Theories from Case Study Research." The Academy of Management Review 14(4): 532‐ 550. Fellenz, C. B. (1997). "Introducing usability into smaller organizations." interactions 4(5): 29‐33. Ferrara, J. C. (2005). "Building positive team relationships for better usability." interactions 12(3): 20‐21. Gulliksen, J., I. Boivie, et al. (2004). Making a difference: a survey of the usability profession in Sweden. Proceedings of the third Nordic conference on Human‐computer interaction, Tampere, Finland, ACM Press. Iivari, N. (2005). Usability Specialists ‐ ‘A Mommy Mob’, ‘Realistic Humanists’ or ‘Staid Researchers’? An Analysis of Usability Work in the Software Product Development. Human‐Computer Interaction ‐ INTERACT 2005: 418‐430. Iivari, N. (2006). Understanding the work of an HCI practitioner. Proceedings of the 4th Nordic conference on Human‐ computer interaction: changing roles, Oslo, Norway, ACM International Conference Proceeding Series. Ivory, M. Y. (2001). An Empirical Foundation for Automated Web Interface Evaluation. Computer Science. California, University of California at Berkeley: 300. Ji, Y. G. and M. H. Yun (2006). "Enhancing the Minority Discipline in the IT Industry: A Survey of Usability and User‐Centered Design Practice." International Journal of Human‐Computer Interaction 20(2): 117‐134. Jokela, T., M. Siponen, et al. (2006). "A survey of usability capability maturity models: implications for practice and research." Behaviour & Information Technology 25(3): 263 ‐ 282. Levi, F., P. Melo, et al. (2007). Evangelizing Usability to 700 People: Strategies for Building a User‐Centered Organizational Culture. Human‐Computer Interaction – INTERACT 2007. C. Baranauskas, P. Palanque, J. Abascal and S. D. J. Barbosa, Springer Berlin. 4663: 673‐674. Mayhew, D. J. (1999). "Business: Strategic development of the usability engineering function." interactions 6(5): 27‐34. Nielsen, J. (2005). "Evangelizing Usability: Change Your Strategy at the Halfway Point." Retrieved 22 April, 2011, from http://www.useit.com/alertbox/20050328.html. Parush, A. (2006). Toward a Common Ground: Practice and Research in HCI. Interactions. 13: 61‐62.

Vince Bruno and Martin Dick QSRInternational. (2011). "NVIVO7." Retrieved 22 April, 2011, from http://www.qsrinternational.com/. Robertson, T. and C. Hewlett (2004). HCI Practices and the Work of Information Architects. Computer Human Interaction: 369‐378. Rosenbaum, S., S. Bloomer, et al. (1999). What makes strategic usability fail?: lessons learned from the field. CHI '99 extended abstracts on Human factors in computing systems. Pittsburgh, Pennsylvania, ACM. Seffah, A., M. Desmarais, et al. (2005). HCI, Usability and Software Engineering Integration: Present and Future. Human‐ Centered Software Engineering — Integrating Usability in the Software Development Lifecycle: 37‐57. Sherman, P. (2006). Usability Success Stories: How Organizations Improve by Making Easier‐to‐use Software And Web Sites, Gower Publishing Company. Strauss, A. and J. Corbin (1990). Basics of qualitative research. grounded theory procedures and techniques. Newbury Park, California, Sage. Wilson, C. E. (2005). "Usability and User Experience Design: The Next Decade." Intercom 52(1): 6‐9. Wilson, C. E. (2007). "Please listen to me!: or, how can usability practitioners be more persuasive?" interactions 14(2): 44‐ 45 & 55.

Information Availability as Driver of Information Security Investments: A Systematic Review Approach Duy Dang and Mathews Nkhoma RMIT International University, Ho Chi Minh City, Vietnam dptdang@gmail.com mathews.nkhoma@rmit.edu.vn Abstract: Despite that information security issues have started to gain managers’ attention since computers were first put in use, information security management has not yet reached its maturity and is still requiring input from both the academia and industry. Indeed, today’s businesses have still not been widely convinced to invest in information security initiatives, resulting in the shrinking budget allocated for organisational information security. One common finding shows that organisational awareness towards information security can serve as a great driver that would help firms realise the business values of such investments. In addition, such emphasis on the awareness suggests the essential role of training, education and dissemination of quality information. As a result, one could argue that the available information has an indirect influence on the adoption rate of information security, through the impact of awareness. This research analyses the possibility of whether information availability could directly drive the intention to invest in information security initiatives by removing the uncertainty surrounding such investments. In other words, information availability per se could drive investing intention by reducing the obstacle – its uncertainty – rather than stimulate business needs through the enforcing of another factor that is awareness. Through intensive reviews on the literature, this paper synthesises and reports on the definition of information availability and how it could drive the intention to invest in information security. Specifically, the researchers examine the driving force of internal information (risk management, staff suggestions), external information (consultants, external audit) and general information (white papers, security reports). By exploring the direct relationship between information availability and intention to invest in information security, more practical recommendations and directions to promote organisational information security can be suggested. Before that, the researchers aim to update the readers with an understanding of the role of information availability in information security management. Keywords: information security, investment, information sharing, systematic review, driver, information availability

1. Introduction Many studies have been investigating the factors that influence organisational information security management. Specifically, practitioners and academics are interested in determining best practices to foster the development and implementation of information security initiatives. One popular practice is to improve awareness towards information security by means of training, education and disseminating of information. As a result, organisations become less reluctant to make decisions on information security investments as awareness is enforced. Likewise, such information’s role is often considered as an indirect impact on information security adoption through the increase of awareness. Managers tend to be reluctant when making decision on information security investments as they are associated with uncertainty. Indeed, such uncertainty clouds the managers’ vision about information security and many perceive it as a cost or a difficulty, rather than an initiative with strategic values. (Lesk 2011; Rakes et al. 2012; Stewart 2012) Uncertainty of information security investments can be reduced with sufficient and quality information, thus allowing managers to justify their strategic decisions and information security improving adoption rate. This approach differs from how information is used to deliver awareness as it adds clarity and encourages managers to balance their decisions on investments, rather than purely promoting the necessity of information security investments. Contemporary researchers are focusing on the emerging topics of information sharing and asymmetric information in the recent years. (Moore and Anderson 2011) They attempt to reduce the situation in which firms’ knowledge of information security are not at the same level (i.e. asymmetric) due to security stakeholders’ refusal to report security equally. However, the details about shared information is often in research about information sharing practices. In other word, many studies treated the importance of such information as “as‐is”. This paper investigates these gaps especially the direct driving function of Information Availability on managerial intention to invest in information security.

Duy Dang and Mathews Nkhoma The paper first introduces the term Information Availability and its links to other emerging topics. Section 2 explains the chosen research method and describes the research process. Section 3 and 4 synthesise knowledge from other studies to answer our research questions and elaborate the research. Finally, Section 5 concludes this study by discussing limitations and providing future research directions.

1.1 What is “information availability”? Information Availability is actually not a discovery but rather obscured by more well‐known emerging topics such as information sharing and vulnerabilities prioritisation and other activities alike. Being created by different information sharing practices – either internal or external – Information Availability can assist decisions‐making ability significantly. In this study, we leverage the definition of information resources from Rowe and Gallaher (2006) which is now referred as the term Information Availability. Accordingly, information resources can be classified into three categories: (1) Internal, (2) External Public and (3) External Private. (Rowe and Gallaher 2006) The authors provide these categories based on how their respective information is acquired by the business. For instance, internal resource refers to the information that the business can only retrieve by scanning internal vulnerabilities. On the other hand, external private resource includes the business’s customers and suppliers that provide information such as suggestions and requirements and so on. However, this study does not report the driving force of customer’s demands, partner/supplier’s requirements and regulations. This is due to the fact that customer’s demands are often prioritised by companies (especially those offering products and services that are exposed to information security risks) and partner/supplier’s requirements should be treated as compliance (an agreement acknowledged by both parties to proceed on doing business).

1.2 Why “information availability” is important? It is common that managers often look at the result of cost‐benefit analysis of the investment before deciding to allocate budget. Determining the factors that are included in the analysis to produce accurate result thus becomes crucial. However, many businesses struggle with performing such task. Among numerous types of investments, information security initiatives remain to be one of the most uncertain ones. This uncertainty – the lack of concrete information about costs‐benefits – is argued to mainly belong to information security’s nature: obscured by complexity and changing rapidly. There are many reasons explaining the challenges organisations face when deciding which security measure should be invested in. The rapid increase of new technologies and the respective cyber threats have resulted in an overwhelming number of security options that organisations have to decide for their business. (Baker and Wallace 2007) In addition, security managers often complain that it is confusing to conclude whether an attack does not occur thanks to the security measure or just mere coincidence when measuring security investment’s effectiveness. (Kleinfeld 2006; Pettigrew and Ryan 2012) Likewise, Pfleeger et al. (2007) describe security practitioners “still felt at sea” when it comes to evaluating security investments. It is clear that the potential benefits of security investments are unclear and managers feel reluctant to invest in something that yields immeasurable profits. The changing and complex nature of threats also creates the uncertainty of varied costs of security investments. Businesses seek to remove the uncertainty of how much should be spent to retrieve the potential benefits from the investments, either tangible or intangible. For example, the researchers examine the case of investing in a security measure that protects the organisation from external attacks, given that the business pays for every attack that can be stopped. To efficiently decide the appropriate cost of that investment in exchange for its benefits (i.e. protection against certain attacks), security managers need to identify the likelihood of how many and which types of attack would possibly target their assets. Answering these questions requires the results from their risk management activities or internal vulnerabilities audit. Once decision makers can determine the costs (how much they have to spend to prevent the expected attacks) and the benefits (how much can be saved from preventing the attacks) they can proceed on implementing the investment. Therefore, uncertainty is associated with the intention to invest in information security.

Duy Dang and Mathews Nkhoma From our literature review, we found Information Availability can help reduce such uncertainty. To fully discuss the supporting roles of Information Availability in Section 3 and 4, we first discuss the chosen research method in the next section.

2. Research method 2.1 Concept Systematic Literature Review (SLR) is an emerging qualitative technique that inherits the spirit of Positivist as it favours the way knowledge is accumulated. Indeed, the goal of SLR is to control the exhaustive amount of literature, especially in research topics that include multiple fields and subfields, in a fashion from which comprehensive syntheses and traceability can be provided. In this study, the researchers adopt the 5‐stage guideline provided by (Denyer and Tranfield, 2009) as below: 1. Question formulation 2. Locating studies 3. Study selection & evaluation 4. Analysis & synthesis 5. Reporting & using the results As stage 5 covers the descriptive and thematic analyses which are both treated as separated sections in this study, the researchers remove this stage to avoid confusion.

2.2 Question formulation In SLR it is essential to form a panel of reviewers who will be responsible for formulating the research questions. This research question continues to define the scopes of the research and ensure the content is well‐concentrated and consistently fit with the intended research purposes. The panel consists of two researchers whose fields including information security management, IT governance and cybercrimes. Their prior and current researches focus on the different facets of information security thus is relevant to this paper’s theme. Through discussions the formulated research questions are:

RQ1: Can information availability (internal and external) drive information security investments?

RQ2: What have been done to investigate their driving function since 2007?

2.3 Locating studies This stage justifies the researchers’ decisions on (1) from which databases and (2) by which search strings should relevant literature be retrieved. The chosen combination was based on the researchers’ experiences in conducting literature reviews on similar topics as well as suggested by various prior studies. For further information, Table 2 encapsulates the details of the researchers’ search instruments.

2.4 Study selection and evaluation Given the research questions above, the researchers developed a search strategy that utilises the search string and databases in the most efficient fashion in terms of time and quality. The result was a 2‐stage strategy in which the data collected separately from the first stage was cross‐reviewed and filtered in second stage. During the first stage, two search protocols according to the formulated questions were assigned to each researcher with different approaches. On one hand, a search string was employed to search for all drivers of information security investments. Each paper’s title and abstract were scanned by one researcher to retrieve the relevant ones at the end of stage 1. On the other hand, the journal of Security & Privacy from IEEE Xplore database was hand searched and each of its articles was reviewed following the same method. Finally, all materials were thoroughly reviewed and evaluated by both researchers to determine which ones are the most relevant to this study. The researchers employed the statistical software package SPSS to record and run analyses on the data.

Duy Dang and Mathews Nkhoma

3. Thematic analysis This section seeks to answer Research Question 1 by presenting an in‐depth thematic analysis on the selected literature (Table 2). To evaluate the collected data’s relevance, the researchers set up a scoring framework which is explained in Table 1. Table 1: Scoring framework

Item

1 3 5 6 7 8

Does not contain bias/advertising/subjective content Follows scientific methodologies Content is directly relevant to research question’s context i.e. mentions keywords Content is indirectly relevant to research question’s context i.e. mentions related concepts Is published from 2007 onwards Is published in a peer‐reviewed journal Max/Least possible score

Score Y/N 1/0 1/0.5 1/0 0.5/0 0.5/0 1/0.5 5/1

Table 2: Details of search strategy

PROTOCOL 1

PROTOCOL 2

STRING

DATABASE

"information security investment" AND (driv? OR antecedent OR incentive)

ACM: Association for Computing Machinery Digital Library Emerald Management Xtra (Emerald) ScienceDirect (Elsevier) SpringerLink BERITA: Malaysia/Singapore/Brunei/ASEAN Database (Informit) Computer Database (Gale) ProQuest Computing (ProQuest)

Hand‐search from IEEE Security & Privacy Journal

SEARCH CRITERIA

From 2007 to 2012 English

5 5 50 ‐ ‐ 11

IEEE Xplore

TOTAL SEARCH RESULTS

19 Total

118

The descriptive analysis (Section 4) shows that the majority of our papers reviewed on the subject matter of information availability fell into Information Security Management and Investment topics. In this section the researchers continue elaborating the collected papers and providing further discussions on their implications regarding the roles of information availability. In particular, two types of information sources – internal and external (as introduced in Section 1) – will be discussed.

3.1 Internal information resources Internal information resources can be most useful to support organisation’s decision‐making ability. Internal information can be retrieved at faster rate and more flexible than those drawn from external sources. In addition, organisation with well‐established security management procedures can retrieve high quality information from its own staff that understands their business better than external parties. Business is recommended to conduct a number of different procedures to gather supporting internal information and reduce as much as possible the uncertainty of security investments. (Baer 2003; Baker et al. 2007; Rakes et al. 2012) These privately own information play crucial roles in driving managerial intention to invest in information security by suggesting the needs to implement security measures while supporting rational analyses. Today’s well known quantitative tools are NPV, SLE, ALE, ROI, and ROSI, just to name a few. (McFadzean et al. 2007; Strigini 2012) These tools calculate variables such as value and cost (commonly derived from the values of the assets and vulnerabilities) of the investment and their result’s accuracy depends on the available information of those variables. While rational predictions can be produced from these tools, managers are warned to not solely rely on economic model such as NPV to make investment decisions. Accordingly, Gordon and Loeb (2006) argued that NPV analysis can only get approval from one managerial level while the final decision should combine non‐quantitative (qualitative) factors. Pfleeger (2012) emphasised the importance of

Duy Dang and Mathews Nkhoma combining quantitative and qualitative approach in analysing investments. On one hand, results of internal technical audits, risk management activities or intrusion detection systems such as vulnerabilities reports, risks breakdown structure etc. can be retrieved quantitatively. On the other hand, managers can also rely on expert judgements from internal security specialists as qualitative methods. (Buck et al. 2008) Prior researches also suggest companies to be proactive in developing their cyber defence so that they can gain several monetary benefits such as lower costs and higher ROI’s result. (Diamant 2011; Huang et al. 2008) To be proactive, managers are suggested to define business mission (Pettigrew and Ryan 2012) and align information security with business values once the data is collected (Pfleeger et al. 2007). More importantly, security discussions should be more opened to staff from all sections to build mission that reflects accurately organisation’s needs. Fortunately, engagement between risk owners from all aspects developed as the concept of information security governance has been receiving attention (Moulton and Coles 2003; Solms 2006). This trend is reflected by the developments of new vulnerabilities prioritisation frameworks that allow incorporation of inputs from multiple stakeholders from different security dimensions (Chen 2007; Fruhwirth and Mannisto 2009). Likewise, (Johnson et al. 2009) asserts that encouraging staff’s appropriate and regular security inputs is beneficial to drive changes in organisations. Despite much knowledge about the best practices to acquire and use information about security were drawn from the literature, the researchers did not find any statistically tested findings that such information can encourage business to spend on security investments. Since it appears logical that decision‐makers can make clear of why and how they should spend on security measures with adequate and sufficient information, we suggest future studies to test Research Question 1 as a hypothesis.

3.2 External information resources As discussed in section 1 we exclude customer and partner/supplier’s requirements from this report as their driving force is usually acknowledged. Instead, the driving force of external audit services and security consultants are examined. While internal and external sources provide the same function of scanning and managing vulnerabilities, businesses may treat them differently due to differences in skills and objectivity (Matthews 2012). Fenz and Ekelhart (2011) suggested businesses to have their defence reviewed by expert team consist of external consultants that have fresh thinking and focused on the emerging threats. This may be true to SMEs or businesses with little experienced in information security to have external parties advise on their infrastructure. Other sources of external information are provided by security vendors, security firms’ reports and security conferences. While advertisements about security product can be subjective to promote sales and overwhelmed for making decisions (Moore 2010), organisations may also feel discouraged to invest in information security if they are not aware of the products’ existences in their local market. On the other hand, researchers have been addressing the importance of voluntarily sharing security information within the industry to improve threat detection and risk management, thus strengthens the overall defence (Liu et al. 2011; Moriarty 2011). However, it is unclear about whether the availability of white papers that report on local security updates would influence intention to invest in security measures. In addition, the same question is raised by the researchers that whether organisations participating in an information sharing network would be driven to focus more on security innovations in comparison to those that are not.

4. Descriptive analysis To answer Research Question 2, we have analysed the content of 28 articles as results of our search protocols. The articles found were published from 2003 to 2012, with the majority of them falls in the year 2007, 2011 (5 articles per year) and 2012 (6 articles). This statistic can be referred to Figure 1. The graph displays more attention has been paid on the topic of measuring the values of information in information security through years since 2010. Next we examine the number of articles per publication and per topic. Table 3 shows that the majority of analysed articles (23 articles) belong to peer reviewed journals thus ensures the overall quality of content. Other types of publication used (2 articles) include news and book chapter.

Duy Dang and Mathews Nkhoma

Figure 1: Number of articles per year Table 3: Type of publication per year 2003 2006 2007 2008 2009 2010 2011 2012

Year

Total

Other ‐ 1 ‐ ‐ ‐ 1 ‐ ‐ 2

Type of Article Conference ‐ ‐ 1 1 1 ‐ ‐ ‐ 3

Total Journal 2 3 4 1 1 1 5 6 23

2 4 5 2 2 2 5 6 28

Figure 2 displays the main topic of each article. Except specific topics such as Cyber Insurance, Risk Management and Security Metrics, we summarise the rest as Information Security Management and Investment. Specifically, Information Security Management refers to discussions about management issues (e.g., security governance, awareness, etc.) and Investment refers to investment strategies and the developments of economic models that determine optimal spending. All topics are related to the subject matter of this study, or at least have mentioned the roles of information availability on security investment. We take a closer look at the publication trend of topics through years in Table 4. Accordingly the concept of cyber insurance gained attention early in 2003 and was not widely discussed since then. One possible reason explaining this phenomenon may be due to the great challenges of asymmetric information and measuring values of breaches of different insurees. These challenges leave cyber insurance remain as a concept and discourage further discussions about it. Information security management, on the other hand, receives much attention in comparison to the other topics. This continuing trend can be explained by the recent need to investigate the influences of socio‐technological factors on information security. We also found a significant body of researches that build and simulate economics models to determine the optimal amount of investment on security. However, there seems to be a lack of studies that focus on investment strategies and the role of information availability as driver of security investment. Finally, risk management and security metrics often receive equal attention due to their close relationship. Risk management issues are mostly about determining and making plans to fix the potential security vulnerabilities while studies about security metrics provide measurements and categories of the risks.

Duy Dang and Mathews Nkhoma

Figure 2: Number of articles per topic Table 4: Number of topics per year Cyber Insurance 2003 2006 2007 2008 Year 2009 2010 2011 2012 Total

1 ‐ ‐ ‐ ‐ ‐ ‐ ‐ 1

Info Security Management 1 2 3 ‐ ‐ 1 3 4 14

Number of Topic Investment Risk Management Security Metrics ‐ 2 ‐ 2 ‐ 1 1 1 7

‐ ‐ 1 ‐ 1 ‐ 1 ‐ 3

‐ ‐ 1 ‐ 1 ‐ ‐ 1 3

Total

2 4 5 2 2 2 5 6 28

5. Synthesis and the way forward The previous sections 3 and 4 have provided the findings gained from the research progress. In particular, the descriptive analysis in section 4 provides an overview of the recent research body of the information availability’s roles since 2007. While the researchers are delighted to realise the increasing trend of studies about information availability, the driving function of such information sources on security investments still remains unconfirmed from the elaborated literature in section 3 – thematic analysis. As the researchers found a few evidences positing the direct driving function of information availability on intention to invest in information security, this should be suggested as a set of hypotheses to be tested. Again, the researchers encourage future studies to investigate and confirm these hypotheses of whether internal and external sources of information can influence investing intentions. To start an initiative, the researchers are conducting a funded research project to develop a conceptual model of the drivers of information security adoption (which includes information availability and their roles) in SMEs in South East Asia. Similarly, future researches could be conducted in different regions so that comparison can be made to draw new findings about adoption of information security measures. In the mentioned ongoing study, a questionnaire was sent to more than 500 IT decision‐makers operating in Vietnam to understand their insights about the drivers of information security investment. The participants were asked to rank the four most important drivers out of nine options in which the focuses of this research – Internal and External recommendations – were included. Accordingly, “Internal recommendations” received st the most nominations to be the 1 prioritised driver (30.16%), followed by “Other information security firms’ reports and whitepapers” (14.88%). “External recommendations” was among the least picked option (4.960%)

Duy Dang and Mathews Nkhoma to be the top motivation of information security investments (Figure 3). The result implied that managers tend to consider insights from their own staff or from published whitepapers first when a need to invest in information security arises. As they refuse to consider external recommendations, one possible reason that explains the trend is the other options provide better objectivity for decision making. This finding is consistent with the dominating amount of the discussed literature that advocates the importance of internal information sources.

Figure 3: Percentage of information security’s drivers – first priority While “External recommendations” were not favoured in the first place by decision‐makers when assessing information security investments, they appeared to be useful as the second option. Indeed, among the candidates to be the 2nd prioritised driver, “External recommendations” ranks first with 16.87%. This indicated the important driving role of external insights that can influence the decision making process after the internal information has been considered and discussed. On the other hand, it also suggested that SMEs with low initial information security knowledge and skills can potentially leverage their defence to external parties that have better focused mindsets on the emerging risks, thus supported the findings from the Thematic Analysis section. Nevertheless, the chance for external insights to replace internal ones seems low as Figure 4 displayed “Internal recommendations” competitive percentage (16.07%) in comparison with “External recommendations” (16.87%). This study emphasises the essential role of information availability and encourages businesses to be open to acquire quality information from various sources to justify their investment decisions. Security managers are encouraged to be proactive in collecting and synthesising information not just from the rigid risk management results but also opinions from staff in different sectors and advice from external parties in appropriate manners. Nevertheless, business should monitor closely the gathering information process to prevent over spending, as shown in the research of Böhme and Félegyházi (2010) about cost of penetration testing and security investment. Last but not least, information sharing practices have been receiving focus from both the academe and industry recently. However, such potential good practices are being challenged by many issues thus discouraging its growth in both public and private sectors. Therefore, it is crucial for future researchers to investigate the drivers and ways to promote information sharing; hence information availability can be enriched by effective sharing practices.

Duy Dang and Mathews Nkhoma

Figure 4: Percentage of information security’s drivers – second priority

References Baer, W.S. (2003) “Rewarding IT security in the marketplace”, Contemporary security policy, Vol. 24, No. 1, pp 190–208. Baker, W.H., Rees, L.P. and Tippett, P.S. (2007) “Necessary measures: metric‐driven information security risk assessment and decision making”, ACM Communication, Vol. 50, No. 10, pp 101–106. Baker, W.H. and Wallace, L. (2007) “Is Information Security Under Control?: Investigating Quality in Information Security Management”, IEEE Security Privacy, Vol. 5, No. 1, pp 36 –44. Böhme, R. and Félegyházi, M. (2010) “Optimal Information Security Investment with Penetration Testing”, in Alpcan,T., Buttyán,L. and Baras, J. (Eds.), Decision and Game Theory for Security, Lecture Notes in Computer Science, Springer Berlin Heidelberg, Vol. 6442, pp 21–37. Buck, K., Das, P. and Hanf, D. (2008) “Applying ROI Analysis to Support SOA Information Security Investment Decisions”, 2008 IEEE Conference on Technologies for Homeland Security, Presented at the 2008 IEEE Conference on Technologies for Homeland Security, pp 359 –366. Chen, Y. (2007) “Stakeholder Value Driven Threat Modeling for Off the Shelf Based Systems”, Companion to the proceedings of the 29th International Conference on Software Engineering, ICSE COMPANION ’07, Washington, DC, USA, IEEE Computer Society, pp 91–92. Denyer, D. and Tranfield, D. (2009) “Producing a systematic review”, The Sage handbook of organizational research methods, Thousand Oaks, CA, Sage Publications Ltd, pp 671–689. Diamant, J. (2011) “Resilient Security Architecture: A Complementary Approach to Reducing Vulnerabilities”, IEEE Security Privacy, Vol. 9 No. 4, pp 80 –84. Fenz, S. and Ekelhart, A. (2011) “Verification, Validation, and Evaluation in Information Security Risk Management”, IEEE Security Privacy, Vol. 9 No. 2, pp 58 –65. Fruhwirth, C. and Mannisto, T. (2009) “Improving CVSS‐based vulnerability prioritization and response with context information”, Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, ESEM ’09, Washington, DC, USA, IEEE Computer Society, pp 535–544. Gordon, L.A. and Loeb, M.P. (2006) “Budgeting process for information security expenditures”, ACM Communication, Vol. 49, No. 1, pp 121–125. Huang, C.D., Hu, Q. and Behara, R.S. (2008) “An economic analysis of the optimal information security investment in the case of a risk‐averse firm”, International Journal of Production Economics, Vol. 114, No. 2, pp 793 – 804. Johnson, M.E., Goetz, E. and Pfleeger, S.L. (2009) “Security through Information Risk Management”, IEEE Security Privacy, Vol. 7, No. 3, pp 45 –52. Kleinfeld, A. (2006) “Measuring Security”, Information Systems Security, Vol. 15 No. 5, pp 7–12. Lesk, M. (2011) “Cybersecurity and Economics”, IEEE Security Privacy, Vol. 9 No. 6, pp 76 –79. Liu, D., Ji, Y. and Mookerjee, V. (2011) “Knowledge sharing and investment decisions in information security”, Decision Support Systems, Vol. 52, No. 1, pp 95–107.

Duy Dang and Mathews Nkhoma Matthews, B. (2012) “Optimizing Product Improvement Spending with Third‐Party Security Consultants”, IEEE Security Privacy, Vol. 10, No. 1, pp 91 –93. McFadzean, E., Ezingeard, J. N. and Birchall, D. (2007) “Perception of risk and the strategic impact of existing IT on information security strategy at board level”, Online Information Review, Vol. 31, No. 5, pp. 622–660. Moore, T. (2010) “The economics of cybersecurity: Principles and policy options”, International Journal of Critical Infrastructure Protection, Vol. 3, No. 3–4, pp 103 – 117. Moore, T. and Anderson, R. (2011) “Economics and Internet Security: a Survey of Recent Analytical, Empirical and Behavioral Research”, Oxford Handbook of the Digital Economy, 2011. Moriarty, K.M. (2011) “Incident Coordination” IEEE Security Privacy, Vol. 9, No. 6, pp 71 –75. Moulton, R. and Coles, R.S. (2003) “Applying information security governance”, Computers & Security, Vol. 22, No. 7, pp 580–584. Pettigrew, J. and Ryan, J. (2012) “Making Successful Security Decisions: A Qualitative Evaluation”, IEEE Security Privacy, Vol. 10, No. 1, pp 60 –68. Pfleeger, S.L. (2012) “Security Measurement Steps, Missteps, and Next Steps”, IEEE Security Privacy, Vol. 10, No. 4, pp 5 –9. Pfleeger, S.L, Libicki, M. and Webber, M. (2007) “I’ll Buy That! Cybersecurity in the Internet Marketplace”, IEEE Security Privacy, Vol. 5, No. 3, pp 25 –31. Rakes, T.R., Deane, J.K. and Rees, L.P. (2012) “IT security planning under uncertainty for high‐impact events”, Omega, Vol. 40, No. 1, pp. 79 – 88. Rowe, B.R. and Gallaher, M.P. (2006) Private Sector Cyber Security Investment Strategies: An Empirical Analysis. Solms, B. von. (2006) “Information Security – The Fourth Wave”, Computers Security, Vol. 25, No. 3, pp. 165 – 168. Stewart, A. (2012) “Can spending on information security be justified?: Evaluating the security spending decision from the perspective of a rational actor”, Information Management & Computer Security, Vol. 20, No. 4, pp. 312–326. Strigini, L. (2012) “Resilience: What Is It, and How Much Do We Want?”, IEEE Security Privacy, Vol. 10, No. 3, pp 72 –75.

A Revised User Interface Design for the Discovery Search Tool “Summon”: A Case Study and Recommendations Jamal El‐Den, Lien Bui and Wendy Nguyen Charles Darwin University, School of Engineering and IT, Darwin, Australia jamal.el‐den@cdu.edu.au Abstract: The research investigated the usability of the “Summon” discovery tool’ user interface among External and Internal students’ at Charles Darwin University (CDU). Summon is a “Discovery” search tool which supports users in their search for scholarly literature (library catalogs, journal articles, and books). The library at CDU was faced with the challenge of offering users with an easy experience with the tool as a result of increase users’ dissatisfaction. Issues and concerns were raised regarding the adequacy and the usability of the tool. The research is conducted in order to find out these issues and in order to provide users with a highly depend tool for finding academic sources. Two experiments based on interviews and questionnaires were conducted on external and internal students using diary studies, heuristic evaluation and usability testing in order to explore the users experience with the tool and in order to identify its usability problems. Data from external and internal users were collected as a set of measureable factors for user interactions. The data collected from the interviews and the questionnaires was analyzed and the poor usability aspects of the tool were identified as well as the reasons of the users’ dissatisfactions was identified. The research did not address error prevention and a lack of step‐by‐step instructions. The findings from the experiments together with a revised interface for the tool where highlighted and the administration of the CDU library were provided with the findings and the recommended changes to the tool. These recommendations were adopted by the CDU library. These recommendations included a revised user interface design for the Summon tool which highlights the necessary changes to be considered. These changes contribute to an easier, clear and user friendly experience which will enhance the user’s interaction and user experience. Keywords: usability evaluation, heuristic evaluation, usability testing, summon tool, user interface design and usability

1. Introduction The research investigates the Charles Darwin University CDU library user experience of external and internal undergraduate students with the “Summon” discovery tool during their search for resources at the CDU library. Summon, developed by “Serials Solutions”, is a system that provide users with access to unified indexes through one search. Instead of searching the Internet as a whole, Summon is supposed to provide users with a satisfying experience by narrowing the search results down to academic sources only.The content found in the library collections ranged from books and videos to e‐resources such as journal articles. This system goes beyond the library catalogs and databases that created an all‐new service for libraries (Serial Solutions 2012). This search system was set up to provide an easier use for external/internal students to find assignment materials. Recent CDU library client survey showed confusion and user‐unfriendliness by external/internal users of Summon. Some areas are more important to library clients than others. This was to meet the needs of users by ease of usability, usefulness of the web site and Summon and availability of off‐campus access to resources and services. The major finding was the 47% of the 394 respondents wanted to learn more about Summon. (Insyncsurveys 2012) The research explores, investigates, analyses and evaluates the usability aspects of Summon at CDU. It focuses on the current problems that produce poor user interaction. It portrays a list of recommendations to improve the system. These recommendations are based on the results generated from the methodologies used and data collected from the users. The data gathered from the external/internal students and IT evaluators allowed the library clients to recognise what the students’ needs were. To help achieve a more evidenced based solution, this project identified a number of external and internal students to be involved. They were required to use diary studies, heuristic evaluation and usability testing to record their experience with Summon. The results collected were analysed and a list of recommendation was established for the library clients. This allowed the library clients to recognised the issues within Summon and decide if there were changes needed to improve the Summon interface.

Jamal El‐Den, Lien Bui and Wendy Nguyen

2. Methodologies As stated earlier, three different methodologies were used in this research, namely, dairy studies, heuristic evaluation and usability testing. They were used to analyse the usability of the tool. The background research contributed to the investigation by identifying potential usability criteria for the Summon tool. Preece, Rogers and Sharp (2002) stated that user frustration is caused by poorly planned designs. This concept addresses questions such as (Borchers 2001):

Whether Summon has adapted the users’ native terminology and concepts within its interaction design.

Whether Summon is rapid and error‐free with its performance of the operator; or

Whether Summon is accommodated to human diversity. This can be in terms of cognitive and perceptual abilities, culture and personality differences.

To analyse the HCI of Summon, usability evaluation methodologies must be used. There are many usability evaluation methods available. However, the appropriate methods for the project are heuristic evaluation and usability testing, because the purpose of these methods is the most relevant to the project, to identify usability problems and errors. (Johnston, Eloff&Labuschange 2003)

2.1 Comparison of methods Heuristic evaluation was used in the study by Peng, Ramaiah and Foo (2004) to evaluate the user interface of the library’s service tool, called gateway to electronic media services (GEMS Usability testing is a systematic way of identifying what is easy or difficult to users by observation and collecting the information for improvement (Corry, Frick & Hansen 1997). The research highlighted different methods for usability testing, which is deemed to be relevant to the project, by suggesting the methods that would be more effective in collecting data. The research did not conduct a focus group test; this is because it is hard to recruit all users to do the test at the same time, since users might have conflicting schedules. According to Alaszewski (2006), a diary is a document created by an individual who makes regular entries about events in their life or at the time that those events occurs. Diary studies have the advantage of being a more accurate research method when it’s necessary to focus on changes over time, where individual’s moods or multiple events that occur within the day are recorded. To clarify how the methods used in this study complement one another, table 1 is anoverview about the purpose of each, how the data was collected and how it was used. Table 1: Comparison of methods Method Purpose

Diary Studies Identified the step‐by‐step process of how external students used Summon to do their research. Distance external students.

Usability Testing Evaluated what usability issues that occurred when the users interacted with Summon. Four local external students.

Data Collection process

Diary template and questionnaires were provided and they were three‐ five diary entries.

Analysis

Analysed all the templates and the recorded video that the users sent back.

Observation and screen recording were conducted during the testing. The participants completed the post interview after they had completed their four tasks. Analysed the notes, video recordings and interviews that were conducted during usability testing.

Participants

Heuristic Evaluation Finding the usability issues in a user interface design based on heuristic principles. Expert users (IT students who have experience in undertaking Heuristic Evaluations of interfaces) Survey evaluation based on Nielsen’s (1993) heuristics, adapted from Peng (2004).

Analysed all the survey evaluation that the user sent back.

Jamal El‐Den, Lien Bui and Wendy Nguyen

2.2 Comparison, problems and analysis The research and based on the methodologies described earlier identified the problems shown on table 2. These problems formed the basics upon which the data were collected and the analysis was done. Table 2: Internal/external students’ and common problems Internal students’ Problems Terminology used (E.g. Everything (home screen) and ‘Any’). Lack of headings for interactive links on home screen. No heading or label for ‘relevance, oldest, newest’ refinement option. Logo used on Summon main interface (CDU logo) is misleading, making users think that it would lead them back to the main CDU website interface. Lack of white spacing between grouped elements (E.g. Radio buttons on home screen). Icons blend too much into the background colour. Email feature. No visual indication of its location on the interface. Confirmation pop‐up for email (E.g. forget ‘@’). Confirmation pop‐up for starting new search Does not provide step‐to‐step instruction or help documentation or tutorials (documents or YouTube

External Students’ Problems Search box is no large enough to fit more than 5 keywords Quick links under Summon on the library home screen makes the users assume that they are a part of Summon, when they are not. Online tutorial link is cluttered within many quick links, making it harder for users to see it. There is no place to report broken links when the articles aren’t opening. The results information are too clutter making it hard for users to read the results. Page number location are located at the bottom of the page, therefore it is inconvenience to scroll to the bottom of the page to go to the next page.

Common Problems There is no auto fill prompt in the search box on the library home screen. Refinement descriptions (side bar) are too lengthy in text/ too many options. Text length makes it looks messy and unorganised. Time consuming for the users to go through all the options. Publication graph is not user‐friendly looking and does not appeal usable. Date input boxes is too complex (search by dd/mm/yy). Unnecessary for a simple search. Doesn’t look simple and is inconsistence as the users have to select the date and month and type the year, therefore creates artificial confusion. Save icon does not appeal as it is a save feature. Location of ‘keep search refinement’. Covered by auto complete dropdown box, thus users have a tendency to miss it.

2.3 Project approach ‐ heuristic evaluation A plan for the heuristic evaluation was developed as follows:

Phrase1: Evaluators are briefed on what is required of them and what the evaluation is about.

Phrase2: Each evaluator inspects the user interface at least twice alone.

Phrase3: Evaluators are required to gather together and communicate their findings to get rid of duplication of problems and prioritised the problems discovered..

Figure 1: Heuristic evaluation stages (Yushiana& Rani 2007)

Jamal El‐Den, Lien Bui and Wendy Nguyen

The evaluators are expected to have rich background knowledge towards the methodology of heuristics, usability and the Summon interface.

Nielsen’s heuristic principles will be used as guidance to develop heuristic questions, and the questions will also be based on previous questions that were used in the other libraries’ studies.

Each evaluator will be given an appropriate timeframe of one hour and a half to two hours to complete the evaluation.

Heuristic evaluations have three phrases and can be implemented quickly and conveniently. Please see Figure 1 below for the description of each stage.

A Heuristic Evaluation Criteria has been drawn up based on Nielsen’s heuristic and the background research above. The criteria outline the goals and questions that would guide the expert in evaluating the Summon interface.

2.4 Project approach ‐ usability testings To test and determine the usability of Summon, the ‘DECIDE’ framework (Preece, Rogers & Sharpe 2002) was used to develop a plan. This plan described the sequence used to conduct the usability tests in detail. A brief plan for the user tests is as follows:

User testings

Signing of consent forms

The facilitator explained the concept of video recording, how it will be used etc.

Background questions were asked before users can perform the tasks, for example:

How often do you use the Summon tool?

Do you know much about the tool? Etc.

The participants were requested to go through the Summon tool to be familiarised with the interaction flow.

Tasks assignment to participants.

Facilitator notes recording.

Post completion questions

After the usability evaluations, the results were analysed by gathering all of the data collected from interviews, questions

The goal of the usability tests are highlighted in the hypothesis below. Hypothesis

90% of the users should perform the test correctly for the test to be considered as successful.

90% of the users will be able to find the ‘save’ button and send the list of citation to themselves or to me.

90% of the users will be able to complete every single task according to the set task time.

2.5 Project approach ‐ diary analysis The researcher provided the diary study users with a diary entry template. The diary entry template ensured that the data collected was consistent among the users. The users recorded three ‐ five diary entries within the duration of the assessment due date. The users recorded their entries based on one assessment. The diary entry method identified how the students used the Summon system for assessment research and the identified problems that occurred within their own researching approach. After the completion of the diary entries, the users completed a questionnaire. The diary entry template included the following questions:

What are you trying to do (tasks)

Jamal El‐Den, Lien Bui and Wendy Nguyen

On a scale of 1 (low) to 10 (high): how important was this task to completing your assessment

List the steps you took in completing your task in Summon

On a scale of 1 (low) to 10 (high): how successful were you in completing this task

Explain why your task was successful or not successful

The diary study questionnaire included three questions:

What are the positive features in Summon?

What are the negative features in Summon?

What changes would you suggest for Summon?

3. Results and discussions 3.1 Combined results (internal/external students) Table 3: Combined user patterns User Patterns Problems Identified

Internal Users U1

U1 0

U1 1

U1 2

U1 3

U1 4

U1 5

Problems identified with the homepage User did not know that Summon was a unified search tool.

Problems identified with the side‐bar User did not see the refinement option of books. User skipped the refinement option of books because user doesn’t normally use it. User had difficulty with the peer‐review option because the user couldn’t see it. User had difficulty with the peer‐review option because they overlooked it. User had difficulty refining the date. User had trouble with using the graph. User had trouble understanding the date/month

Jamal El‐Den, Lien Bui and Wendy Nguyen User Patterns Problems Identified input.

Internal Users U1

U1 0

U1 1

U1 2

User didn’t see the ‘keep search refinement’ option. User didn’t know that it existed.

U1 3

U1 4

U1 5

Problems identified with the interface User didn’t see the ‘save’ icon. Thus had to place mouse over it. User overlooked the ‘save’ icon because it didn’t stand out from the background. User overlooked the icon ‘save’ because it was misleading User didn’t understand that the email function is within the ‘save item’ feature. The user stated that the input date sections looks complex. The user expressed that the graph was not very usable because it takes a few attempts to click onto it. Use stated that they didn’t like the word ‘Any’ on the side‐bar. User stated that it’s hard to see the results pre page option which is positioned at the bottom of the interface. User used the

Jamal El‐Den, Lien Bui and Wendy Nguyen User Patterns Problems U1 U2 U3 U4 U5 Identified back button on the web browser to get back to the homepage. Should replace the CDU logo with Summon logo. Problems identified with the homepage User did not know that Summon was a unified search tool.

Internal Users U6

U1 0

U1 1

U1 3

U1 4

U1 5

U1 2

6/25

4/25

2/25

8/25

7/25

4/25

5/25

4/25

22/25

Jamal El‐Den, Lien Bui and Wendy Nguyen User Patterns Problems U1 U2 U3 U4 U5 Identified existed. Problems identified with the interface User didn’t see the ‘save’ icon. Thus had to place mouse over it. User overlooked the ‘save’ icon because it didn’t stand out from the background. User overlooked the icon ‘save’ because it was misleading User didn’t understand that the email function is within the ‘save item’ feature. The user stated that the input date sections looks complex. The user expressed that the graph was not very usable because it takes a few attempts to click onto it. Use stated that they didn’t like the word ‘Any’ on the side‐bar. User stated that it’s hard to see the results pre page option which is positioned at the bottom of the interface. User used the back button on the web browser to get back to the homepage. Should replace the CDU logo with Summon logo.

Internal Users U6

U1 1

U1 2

U1 3

U1 4

U1 5

12/25

17/25

4/25

6/25

2/25

11/25

4/25

6/25

U1 0

11/15

Jamal El‐Den, Lien Bui and Wendy Nguyen

4. Recommendations 4.1 Internal students’ recommendations

Figure 2: Summon (terminology) Change the word ‘Any’ to ‘Everything’ like on the homepage to keep it consistent or vice versa.

Figure 3: Summon (home interface) Re‐size the quick links on the library homepage and have a phrase like ‘Quick links’ to describe the links underneath the radio buttons.

Figure 4: Logo (Summon & CDU) Replace the CDU logo within the Summon interface with a Summon logo. Make the font colour darker so that the font can stand out more from the background colour.

Figure 5: Summon (save icon) Highlight the icon darker for visibility. Provide a tutorial link (videos, text and graphics) located on the homepage to give the users an introductory on Summon and how to use it and how it operates. Have ‘saved and email items?’ title instead of ‘saved items’ to inform the users that they can email the source citation to themselves from the beginning, instead of having to discover it later on after using it for a long period.

Jamal El‐Den, Lien Bui and Wendy Nguyen

4.2 External students’ recommendations

Search field box – most of the usability testing participants found that the search field box was too small when they entered more than five keywords.

Quick links under Summon –The recommended suggestion is to group the quick links using useful labels that explain what the quick links are for and relocate it alongside with the other grouped boxes.

Online tutorial more visible – the online tutorial link needs to be more visible so that the users can quickly refer to it. The recommended suggestion is to relocate the online tutorial link to the Summon area, as this new location will catch the users’ attention where it does not require the users to search for the link.

Report broken links – The recommended suggestion is to have a separate link for the broken links labelled as “Report Broken Links” at the top right. A window would expand and require the user to enter the URL of the broken link without explaining, “what is the problem?”

Spacing of the results information –The recommended suggestion is to provide line spacing between each line that makes the interface less cluttered and enhance readability.

Page number location – The recommended suggestion is to add a page number at the top of the Summon interface instead at the top.

Proposed Summon Interface for CDU Library

Figure 6: Summon's proposed interface The ideas were initiated from various university library websites that used Summon as their search tool. The interface is simple as it provides visibility of system status from the heuristic principles, where it informs the location of the user using “breadcrumbs”.

4.3 External/internal student’s (common) recommendations

Auto fill prompt – most participants felt that Summon needed to offer features such as auto fill prompt like other search engine such as Google Scholar. The recommended suggestionswere to implement an auto fill prompt just like Google Scholar to assist the users in their search.

“Keep Search Refinements” and “New Search” – The recommended suggestion were to remove the default of the “New Search” which gives the users the option to select what they want or re‐phrase the ‘keep search refinement’ to ‘keep selected options’ and re‐position the ‘keep search refinement’ link

Jamal El‐Den, Lien Bui and Wendy Nguyen above the ‘refine your search’ section on the side‐bar so the users would be able to visual see it when they limit their options.

Less search refinement options – The recommended suggestion is to reword the search refinements making it more like everyday language rather than technical jargon and simplifying the subjects in the “Subject Terms”.

Publication tables – The recommended suggestion is to demonstrate how to use the publication chart through online video tutorial provided by the librarians or replace the current graph with one that is more visible of its function and eliminate the date input of day and month. The interface would look more neat and simply to

Saved items more visible – the participants hardly noticed that saved item because it was too small. The recommended suggestion is to increase the size of the “saved items”.

Other recommendations by the librarian were identified and implemented into the new design:

Search field box

Auto fill prompt

Quick links under Summon

Online tutorial more visible

“Keep Search Refinements” and “New Search”

Less search refinement options

5. Conclusion The research highlighted the current findings based on the literature review. Three methods were implemented in order to demonstrate the Summon effectiveness and efficiency to the participated students and IT evaluators. Visibility, lack of instructions and help documentation provided and no consideration towards error prevention are the main usability issues with users. The participants were able to search for related assignment materials using several keywords to get their result. Although there were some minor usability issues such as the “New Search” button that was set on default which resulted to the loss of the selected search refinements, the saved items and the online tutorials were not visible enough to the users and there were too many search refinement for the participants to choose from. This resulted confusion for the users and in many instances, as the research showed, the user resorted to other search engines. In addition, Summon showed a lack of error preventions that would inform users of the results of their action. Implementing the suggested recommendations such as providing better labels or less refinement options in the search refinements, enhance the visibility of the saved item, online tutorials and removal of the defaulted “New Search” button had resulted in a better interface and less confusion to the users. The students, especially the externals, had identified their needs forbetter online assistance on how to use Summon and its other features than what was provided in the library workshop in multimedia format for quick reference. It was recommended that the tutorial videos should include features on how users can identify keywords, how users can email citation information to their email address, and they can use the advance search tool. These tutorial videos should be posted on YouTube and the quick link should be noticeable on the Summon interface where the external students could see it. Although only 25 students participated in this study, the data found from these students were significant in identifying the major functionality issues with Summon. The three methodologies used in this study were important because it helped gather useful information from the external students and the IT evaluators. Throughout the research, many sources of errors and limitations were identified which were related to the small sample size of the participants, how the diary studies participants’ interpreted their task and how usability testing was conducted. The participants in the diary studies and usability testing identified useful usability issues which helped improving the Summon usability and accessibility for all students. Some data inconsistency was visible as the participants in the diary study may have interpreted their diary task differently.

Jamal El‐Den, Lien Bui and Wendy Nguyen As a result of this project, the clients at CDU libraryunderstood the problems associated with external/internal students’ experiences with Summon. They provided more online assistances for the students in order to eliminate further problems. These improvements should increase the usability and accessibility of Summon and minimize the confusion of internal and external students. Professionals and academics were consulted to plan the usability testing, heuristic evaluation and diary studies methods used in this investigation.

References Alaszewski, A 2006, Using Diaries for Social Studies, Sage, Nidia Borchers, J 2001, A Pattern Approach to Interaction Design, John Wiley & Sons, Inc., New York, USA. Insyncsurveys, 2012, ‘Charles Darwin University Library Client Survey May 2012’, Northern Territory, viewed 25 July 2012. st Lazar, J, Feng, J &Hochheiser, H 2010, Research Methods in Human‐Computer Interaction, 1 edn, John Wiley & Sons, Glasgow. Nielsen, J 2003, Usability 101: Introduction to Usability, useit.com, Fremont, viewed 18 August 2012. Nielsen, J 2005, Heuristics Evaluation, useit.com, USA, viewed 20 July 2012, <http://www.useit.com/papers/heuristic/>. Preece, J, Rogers, Y & Sharp, H 2002, Interaction Design: beyond human‐computer interaction, John Wiley & Sons, Inc., Danvers, MA. Peng, L, Ramaiah, C & Foo, S 2004 ‘Heuristic Based User Interface Evaluation at Nanyang Technological University in Singapore’ Program: Electronic Library and Information Systems, vol.38, iss.1, pp.42‐59. Rieman, J 1993, The Diary Study: A Workplace‐Oriented Research Tool to Guide Laboratory Efforts, Interchi `93, Colorado. Serial Solutions, 2012, The Summon Service, Serial Solutions, viewed 24 September 2012,<http://www.serialssolutions.com/en/services/summon Sharp, H., Rogers, Y. &Preece, J. 2006, Interaction design: beyond human‐computer interaction, Wiley, Hoboken, NJ. Tan,W, Lui, D &Bishu, R 2009, ‘Web Evaluation: Heuristic Evaluation vs. User Testing’, International Journal of Industrial Ergonomics, vol.36, pp.621‐627 Yushiana, M & Rani, A.W. 2007, ‘Heuristic evaluation of interface usability for web‐based OPAC’, Library and Information Studies, vol. 25, no. 4, pp. 538‐549, viewed 19th August 2012, via Emerald.

The Study of the Relationship Between Individual Factors and Usage of ICT in Tehran’s Public Libraries Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati Faculty of Library and Information Science, University of Tehran, Tehran, Iran ghfadaie@ut.ac.ir mnakhoda@ut.ac.ir soghrashafaghati@gmail.com Abstract: In today’s networked society, many organizations rely on various Information and Communication Technology (ICT) applications in their day‐to‐day business activities to effectively and efficiently deliver services to public. The aim of this study is to investigate the relationship between the human resources factors and ICTs usage to improve service delivery in Tehran’s public libraries. Using the survey data collected from 110 librarians, this study investigates the human factors such as: age, gender, the level of educational attainment, ICT skills and training, attitudes toward ICT usage, working experience, ICT access as well as the level of skill in English language among others. To test the level of ICT usage, a linear regression analysis was applied on panel data. The reliability of ANOVA regression was measured by Cronbach's alpha coefficient. Other relevant factors such as the Pearson correlation coefficient and t‐test statistics have been reported. The results of the regression analysis show an inverse relationship between age, work experience, and computer anxiety with the ICT usage. The results also show that there is a positive and statistically significant correlation between ICT usage and computer self‐efficacy, the level of skill in English language, ICT access, attitude toward information technology, ICT training, as well as past experiences in working with ICT application. The results indicate that there is significant disparity among librarians in applying ICT in terms of education level. However, the findings show no significant disparity between males and females in the usage of ICT applications. The findings represent that while computer anxiety has strong impact on ICT usage, the age parameter, however, showed the lowest score in predicting ICT usage. Keywords: ICT, individual factors, public libraries, librarians, Iran

1. Introduction Organizations have to deal with information and communication technology (ICT) and employ it to improve the organizational effectiveness in order to meet the current and future needs of the community (Fani and Mosleh, 2007). When deployed, ICTs attempt to eliminate repetitive cycle of redundant processes for the sake of affectivity and efficiency. Thus, appropriate employment of ICT in libraries would enhance their effectiveness (Rahnamafard, 2005). Today, ICT usage is a major topic related to public libraries. Making use of computers, library software, computer networks, along with the growth rate of internet users, and the advancement of information systems and resource sharing made the libraries in general and the public libraries in particular to adopt ICTs in almost all functions of the libraries; from providing information to, data organization, storage, retrieval, and distribution (Haneefa, 2007). However ICT deployment is a complicated process in which many technical and non‐technical factors are involved. These factors may be related to the library or ICT itself (Davarpanah, 2001). In the literature, these factors are discussed as the foundations of ICT deployment in libraries. One of them is human resources (staff) of the library. The main goal of this study is to evaluate the relationship between human resources and ICT usage within the context of Iranian public libraries. This study enhances the literature in the field of related factors to ICT usage. The results would be helpful for managers of public libraries in order to organize their resources in the way to reach the success in using ICTs. Managers can provide the growth of factors that have positive significant correlation with ICT usage.

2. Statement of the problem Using ICT not only enhances the services of public libraries, but also improves the librarians’ abilities and the organization of the library (Zavareghi, 2007). Enhancements of the researchers’ effectiveness, enrichment of the individual’s occupational life, facilitation of decisions made by managers, and strengthening the general competitiveness of the organizations have been confirmed as the effects of ICT. In spite of the advantages that ICTs bring to libraries, ICT deployment causes different problems in personal, organizational, cultural, and social aspects of the libraries (Dehdarizadeh, 2011). The ICT value and effectiveness are revealed only when it is used to achieve organization goals. Success in making use of technology depends on technology itself as well as the level of users’ personal skills. In other

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati words, the behavior related to employment of technology is controlled by factors outside of the working environment including but not limited to the characteristics of the job, the scope of job, managerial responsibilities, and the physical arrangements as well as the personal characteristics such as age, education, attitudes, and perceptions (Harrison and Rainer, 1992). Personal factors such as perceptions and attitudes of staff toward ICT and their demographic characteristics are factors that influence the level of technology acceptance and usage by staff (Sheikh‐Shojaei, 2006). The aim of this study is to evaluate the relationship between personal factors and ICT usage in public libraries. Identification of personal characteristics would affect systematic usage of ICT and also brings about new challenges for managers of the libraries in different aspects such as recruitment and selection of human resources, and managerial support. All these measures are expected to lead to a higher level of occupational and organizational productivity as well as customers’ satisfaction (Nakhoda, 2005). To investigate the relationship between personal factors and the level of ICT usage, this study raises three main questions:

Are there statistically significant differences in usage of ICT with regard to the demographical variables and level of education?

Are there relationships between personal factors and ICT usage?

What is the prediction level of each personal factor towards ICT usage?

3. Definition of the study variables In the study, independent variables are personal factors, while ICT usage is dependent variable. Personal factors were extracted from the literature published within 1980‐2012. These factors were suggested as effective personal factors towards ICT usage from studies conducted in Iran as well as on a global scale. Although these personal factors are not related to a specific theoretical model, further studies can be based on the present study to reach a theoretical model of ICT usage in the Iranian public libraries. Factors are:

Age;

Gender;

Level of education (high school, bachelor and master’s degrees);

Level of ICT skills (i.e., the individual’s past experience and knowledge of computers in job related activities). This includes the experience in using word processing software packages, familiarity with Internet and its various applications such as email as well as library software packages.

ICT training: i.e., ICT trainings that an individual passed through in different ways including seven skills of ICDL and library software). For instance, one may have passed academic courses, private training courses, self training, or in‐service training courses.

Attitude towards ICT: i.e., the informed willingness and preparedness of an individual to respond appropriately or inappropriately to ICT aspects evaluated in the study (Igbaria and Chakrabarti, 1990).

Duration of working experience in libraries: It is generally defined as the total working experience of a librarian in libraries (not a specific library).

ICT access: It is the possibility of easy access of a librarian to computer, its accessories, and internet to fulfill occupational responsibilities.

Level of English skills;

Computer self‐efficacy: It is defined as the judgment of a person about his/her skills and abilities in working with computer (Paraskeva, Bouta, and Papagianni, 2008).

Computer anxiety: That is the presence of negative emotions when using the computer (Doronina, 1995).

In the current study, ICT was defined as being familiar with library software packages (at least one package) and being familiar with seven skills of ICDL. ICT usage was the dependent variable of the study and could not be directly evaluated. Therefore, we considered four aspects of ICT usage according to previous studies. Data were gathered based on the self‐report librarians provided about their own performance. In the current study, the concept of ICT usage was defined with the following four items:

Number of software packages used by the librarians;

Number of tasks related to the library that are performed by computer;

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati

The amount of time that librarians use computer each day, and

Frequency of computer use by the librarians (e.g., less than once in a month to several times a day).

4. The empirical study According to data provided by the administration of public libraries of the Tehran province, 110 librarians work in the public libraries of Tehran. We distributed our survey questionnaires among all librarians. Out of 110 distributed questionnaires, 100 questionnaires or 90.9% responses were returned back. Of these responses, eight were excluded since were filled out incompletely. Finally, 92 responses (83. 63%) were used for the purpose of this study. Data were collected using a pilot study; the questionnaire was given to 25 librarian of Varamin town, which had demographical characteristics similar to the study population. All distributed questionnaires were returned back. The participants were not aware that this is a pilot study, and thus, the conditions were similar to those of the real experiment. Collected data were analyzed using SPSS software, v. 19. The Cronbach’s alpha coefficient was determined for any item evaluated in the study. Cronbach’s alpha value was more than 0.7 for studied items indicating the reliability of the questionnaire items. Among the study variables, four items (age, gender, education level, and working experience) were evaluated as open‐end questions, while ICT access and level of English skill were evaluated on Likert scale. “Training in the ICT” and “experience with computers” was evaluated by the measures devised from Igbaria, Zinatelli, and Cavaye (1998). Each of these items was measured by two questions according to ICT definition used in the current study. To evaluate the “attitude toward ICT”, we adopted the measures proposed by Igbaria and Chakrabarti (1990). Computer anxiety was evaluated using computer anxiety scale used by Heinssen, Glass, and Knight (1987). Moreover, computer self‐efficacy was measured using the computer self‐efficacy measure (Compeau and Higgins, 1995).

5. Findings According to results, among 92 participants, 22 (23%) were male. The mean age of participants was 35 years with the age range of 25‐54, which shows that most participants were young. Considering the education level of participants, 10 (10.9%), 12 (13%), 67 (72.8%), and 3 (3.3%) had high school diploma, A. A., B. Sc., and M. Sc., respectively. Thus, most librarians had B. Sc. The working experience of participants in library was in the range of 2‐26 years, with the mean working experience of 9 years. ICT usage score was 4.97 and 5.10 for female and male participants with regard to the t statistics, respectively. The results demonstrated that alpha level of 0.05 was more than the critical value in the table (1.96); thus, null hypothesis is supported. As a result, with 95% confidence it could be stated that men and women were not significantly different in ICT usage (Table 1). Table 1: Difference in librarians’ ICT usage with regard to gender Female Male t Sig. Mean SD Mean SD ICT application 4.97 1.43 5.10 1.62 0.718 0.362 Gender Variable

According to results, the mean scores for the ICT usage were 7.75, 5.18, 5.14, and 3.02 for the librarians with M. Sc., B. Sc., A. A., and high school diploma respectively. Furthermore, the result of F statistics showed that with regard to the education level, the ICT usage was significantly different among the librarians (see Table 2). Table 2: Difference in librarians’ ICT usage with regard to level of education Statistics Education level High school diploma A. A. B. Sc. M. Sc.

Number Mean 10 12 67 3

3.02 5.18 5.14 7.75

Sig.

1.701 0.675 13.93 0.000 1.246 0.000

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati We used the Tukey’s post hoc test to investigate the difference among librarians with regard to the usage of the ICT as shown in Table 3. It was not surprising to see that librarians with high school diploma were less willing to use the applications of the ICT, compared with the librarians with A. A., B. Sc., and M. Sc. Moreover, librarians having M. Sc. are more willing to use ICT in comparison with those having A. A. and B. Sc. Table 3: Results of Tukey’s post hoc test analysis for librarians’ ICT usage with regard to level of education Variable Mean difference Sig. High school diploma and A. A. ‐2.162 0.001 High school diploma and B. Sc. ‐2.116 0.000 High school diploma and M. Sc. ‐4.724 0.000 A. A. and M. Sc. ‐2.562 0.009 B. Sc. and M. Sc. ‐2.608 0.003

According to data provided in Table 4, correlation coefficient between ICT usage and the librarians’ age and working history was reverse and statistically significant (p≤0.01). In other words, with 99% confidence it could be stated that younger librarians with shorter working history use the ICT more and those at higher ages and longer working history are less willing to use the ICT. As shown in Table 4, the correlation coefficient between applications of ICT and ICT access, level of English skill, experience with ICT, trainings in ICT, and librarians’ attitude toward ICT was positive and statistically significant (p≤0.01). In other words, with 99% confidence, it can be stated that librarians with higher levels of these variables would spend more time using ICT in their daily activity and are more willing to use ICT. Table 4: Correlation coefficient between independent variables and dependent variable (ICT usage) No. Independent variable Correlation coefficient (r) Sig. 1 Age ‐0.275 0.008 2 Working experience ‐0.370 0.000 3 ICT access 0.380 0.000 4 Level of English skill 0.442 0.000 5 Experience with ICT 0.377 0.000 6 ICT training 0.428 0.000 7 Attitude towards ICT 0.328 0.002 8 Computer self‐efficacy 0.366 0.000 9 Computer anxiety ‐0.649 0.000

Furthermore, the findings indicated that the correlation coefficient between ICT usage and computer self‐ efficacy was positive and statistically significant (p≤0.01). Therefore, with 99% confidence, it can be stated that when the librarians’ computer self‐efficacy level is higher, ICT usage is higher either. In other words, when computer self‐efficacy of the librarians increases, they are more willing to use ICT. It was found that the correlation coefficient between ICT usage and computer anxiety is reverse and statistically significant (p≤0.01). Thus, with 99% confidence, it can be explained that when the librarians’ computer anxiety is lower, they use the ICT more. In other words, librarians with higher levels of computer anxiety are less willing to use ICT. The findings show that while computer anxiety has strong impact on ICT usage, the age parameter, however, showed the lowest score in predicting ICT usage (see Table 5). Table 5: Regression coefficient of independent variable with the dependent variable of the ICT usage No. Independent variable Regression coefficient (R2) 1 Age 0.0756 2 Working experience 0.1369 3 ICT access 0.1444 4 Level of English skill 0.1953 5 Experience with ICT 0.1421 6 ICT training 0.1831 7 Attitude towards ICT 0.1075 8 Computer self‐efficacy 0.1339 9 Computer anxiety 0.4212

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati

6. Conclusion In this study, a negative correlation was observed between the librarians’ age and level of ICT usage; i.e., younger librarians are more willing to use ICT. In this regard, the level of ICT usage is higher among young Iranian librarians and older librarians are less willing to use ICT. This finding confirms the general understanding in developing countries like Iran that younger people are more prepared and eager to use ICT. It was found that male and female librarians were not significantly different with regard to ICT usage. This is consistent with the findings reported by Zeinali (2009) in the study performed on ICT usage in public libraries of the southern Kerman province of Iran. The finding is in contrast with an earlier study conducted by Nakhoda (2005) and Harrison and Rainer (1992) highlighting a gender difference in ICT usage among librarians. However, ICT usage was significantly different among librarians with different educational background. This study shows that librarians with higher education levels were more willing to use ICT. The finding confirms earlier studies conducted by Nakhoda (2005) and Harrison and Rainer (1992). As such it can be concluded that more educated people have enough self confident to use ICT tools. We observed that there is a negative correlation between the librarians’ working experience and the level of ICT usage; that is librarians with lower working experience use ICT more frequently and those with longer working history are less willing to use ICT. This finding is related to the first question of the research. Librarians with shorter working experience with a high probability are those who have been recently employed in libraries. These librarians are usually younger, have shorter working history, and have minimum education level of B. Sc. During their academic courses, they have acquired some experience in ICT usage. Therefore, in spite of shorter working history, ICT usage is higher among them. In the current study, there was a statistically significant positive relationship between the librarians’ ICT usage and their ICT access. In other words, librarians with a higher ICT access are more willing to make use of it. It was observed that there is a statistically significant positive correlation between the librarians’ level of English skills and their ICT usage level. Moreover, we found a positive correlation between experience with ICT and ICT usage level (0.377) and this correlation is statistically significant. As such librarians with higher experience in the usage of different ICT applications are likely to use technology to fulfill their responsibilities. This is in agreement with the findings of Igbaria, Zinatelli, and Cavaye (1998) and Igbaria and Chakrabarti (1990). In this study, we observed a statistically significant positive relationship between the librarians ’ICT usage and the level of their trainings in ICTs (0.428). This has been emphasized in the studies carried out by Nakhoda (2005), Igbaria and Chakrabarti (1990), and Igbaria (1990). Moreover, a statistically significant positive correlation was observed between the librarians’ positive attitude towards ICT and ICT usage level by them (0.328). This is consistent with the findings of Sheikh‐Shojaei (2006). To improve the Iranian library system, it is crucial to recruit young educated individuals who are familiar with ICT application and who are willing to use their skills in different sections of the library system. Computer and internet training workshops need to be held to improve the librarians’ computer and internet skills. Also, a statistically significant positive relationship was observed between computer self‐efficacy and level of ICT usage (0.366). This is in agreement with the results reported by Wilfong (2006) and Sobhani‐nezhad et al (2011). The availability of computer help and facilities, guidelines related to computer usage, and management support would enhance the staff’s self‐efficacy. A statistically significant negative relationship was observed between computer anxiety and ICT usage level (‐0.649); such that when computer anxiety level is lower in librarians, they use ICT more frequently. This is supported by the findings of Igbaria and Chakrabarti (1990) and Johnson and Markas (2000). Due to the strong impact of computer anxiety in predicting ICT usage (see Table 5), attentions need to be paid to psychological factors in the recruitment of new staff, receiving help from the psychiatrists, and performing psychological interviews. Psychological help can be facilitated by managers to those employees who suffer from computer anxiety in order to boost employees’ self‐confidence and personal motivation in dealing with ICT applications.

References Compeau, D. and Higgins, C. (1995) “Computer self‐efficacy: Development of a measure and initial test”, MIS Quarterly, Vol. 10, No. 2, pp 189‐211. Davarpanah, M. R. (2001) “Level of information technology application in Iranian university libraries", Library Review, Vol. 50, No. 9, pp 444‐450.

Gholam Reza Fadaie, Maryam Nakhoda and Soghra Shafaghati Dehdarizadeh , A. (2011) “A Survey of obstacles of Information Technology application in Shiraz public libraries”, Information Sciences and Technology, Vol. 26, No.2, pp 223‐233. Dorinina, O. V. (1995) “Fear of computer: Its nature, prevention and use”, Russian Social Review, Vol. 36, pp 79‐ 96. Fani, A. A. and Mosleh, A. (2007) “Management and structure factors influencing IT usage in governmental organizations of Bushehr”, Human Sciences Modares, Vol. 11, No. 3, pp 157‐182. Haneefa, M. (2007) “Application of information and communication technologies in special libraries in Kerala (India)”, Library Review, Vol. 56, No. 7, pp. 603‐620. Harrison, A. and Rainer, R. (1992) “The influence of individual differences on skill in end‐user computing”, Journal of Management Information Systems, Vol. 9, No. 1, pp 93‐112. Heinssen, R. K., Glass, C. R., and Knight, L. A. (1987) “Assessing computer anxiety: Development and validation of the computer anxiety rating scale”, Computers and Human Behavior, Vol. 3, pp 49‐59. Igbaria, M. (1990) “End‐user Computing Effectiveness: A structural equation model”, Omega, Vol. 18, No. 6, pp 637‐652. Igbaria, M. and Chakrabarati, A. (1990) “Computer anxiety and attitudes toward microcomputer use", Behaviour & Information Technology, Vol. 9, No. 3, pp 229‐241. Igbaria, M., Zinatelli, N. and Cavaye, A. L. (1998) “Analysis of Information Technology Success in Small firms in New Zealand”, International Journal of Information Management, Vol. 18, No. 2, pp 103‐119. Johnson, R.D. and Markas. G.M. (2000) “Research Report: The Role of Behavioral Modeling in Computer Skills Acquisition ‐ Toward Refinement of the Model”, Information Systems Research, Vol. 11, No. 4, pp 402‐417. Nakhoda, M. (2005) “The Investigation of Individual Factors that Affect Information Technology Usage by Librarians Affiliated to Tehran University” M.A. in Library and Information Science Diss. , University of Tehran. Paraskeva, F., Bouta, H., & Papagianni, A. (2008) “Individual characteristics and Technology: A Comparison of Two Theoretical Models”, Management Science, Vol. 35, No. 8, pp 982‐1003. Rahnamafard, A. S. (2005) “Usage of Information Technology in public libraries”, ketabmah‐koliat, Vol. 9, No. 1‐2, pp 42‐ 55. Sheikh‐Shojaei, F. (2006) “ A study of determinant factors in Information Technology acceptance by the libraries of Technical Faculties libraries at Tehran’s governmental universities: Applicability of Technology Acceptance Model” M.A. in Library and Information Science Diss. , University of Tehran. Sobhani‐nezhad, M. et al (2011) “Explanation the role of organizational support, experience, anxiety, and computer self efficacy in predicting computer application” Journal of Educational Psychology Studies, Vol. 11, pp 45‐68. Wilfong, J. D. (2006) “Computer anxiety and anger: The impact of computer use, computer experience, and self‐efficacy beliefs”, Computers in Human Behavior, Vol. 22, No. 6, pp 1001‐1011. Zavareghi, R. (2007) “Public Libraries of the Future”, Faslname‐ketab, Vol. 17, No. 67, pp 149‐160. Zeinali, A. (2009) “Assessing Educational Needs Relating to Information Technology Among Librarians of Public Libraries of Kerman” M.A. in Library and Information Science Diss. , University of Tehran.

Identification, Evaluation and Ranking of the key Factors to Contribute to the Implementation of Electronic Municipality Marzieh Faridi Masouleh1 and Hadi Mohammadzadeh2 1 Department of Information Technology Management, Science and Research Branch Islamic Azad University, Tehran, Iran 2 Institute of Applied Information Processing, University of Ulm, D‐89069 Ulm, Germany m.faridi@srbiau.ac.ir hadi.mohammadzadeh@uni‐ulm.de Abstract: Electronic municipality, as a modern approach in international civil management, has a potential application in any town and it provides the citizens with its unique services in the areas of municipality responsibilities. In addition, these services could be 24‐hour and without any place limitation. The aim of this research is identification, evaluation, and ranking of the key factors to contribute to the implementation of electronic municipality (a case study in Rasht municipality, Iran). According to the eminent models in acceptance technology, four influential independent factors including tendency to use, ease of use, expected operation, and e‐readiness, called variables, have been extracted. All experts and managers of Rasht municipality constitute the statistical population of this study. The means of research in this study is a random questionnaire based on a standard questionnaire from technology acceptance model. This questionnaire was presented to the participants after confirming its reliability and validity. The obtained results show that there is a significant linear relationship between every independent variable of the study and the implementation of electronic municipality (p<0.01). The greatest proportions of implementing electronic municipality in Rasht are devoted to the tendency to use and the ease of use, respectively. Furthermore, the variable of information technology management affects the relationships between independent variables and variables of implementing electronic municipality. Meanwhile, there is a significant difference between implementation of electronic municipality in good or bad situations of Information Technology management (p<0.05) which indicates that the variable of Information Technology management influences the effect of independent variables in this study. Finally, with respect to the effect of it on the implementation of electronic municipality, ranking of the key factors has been suggested to be as follows: the greatest effect belongs to the tendency to use, and the ease of use get the second rank. Keywords: technology acceptance, tendency to use, ease of use, expected operation, e‐readiness

1. Introduction Electronic municipality is an organization that uses information technology to provide the citizens with its services in the areas of municipality responsibilities quickly, easily, and securely. The services of electronic municipality are 24‐hour and there are no place limitations. Many factors such as e‐readiness, ease of use, social influence, tendency to use, and culture play the key role in successful implementation of the electronic municipality. Many of these critical factors are investigated here in this study. The results show that the tendency to use and the ease of use are more influential in the implementation of electronic municipality in comparison with the other factors. The rest of this article is organized as follows: the second section reviews varieties of technology acceptance, the third section represents research method and research questions, the fourth section introduces methodology, while the obtained results are shown in the fifth section, and finally the conclusions are made in the last section.

2. Varieties of technology acceptance There are varieties of technology acceptance models such as technology acceptance model (TAM), theory of reasoned action (TRA), and theory of planned behavior (TPB) which have shown the most applications in the studies associated with information technology acceptance. Moreover, unified theory of acceptance and use of technology (UTAUT) has also involved many users in different studies. Results of the researches conducted so far confirm that these models show different operations in the field of studying new technologies and their acceptance. Identification of the factors which affect the acceptance of a technology and the creation of circumstances to adopt the intended informational technologies are among the most important researches in the field of information technology. In other words, the reason for acceptance and use of a technology or rejection of it is one of the most important issues in the informative systems.

Marzieh Faridi Masouleh and Hadi Mohammadzadeh For the sake of completeness, basic and underlying concepts for all of these technology acceptance models are depicted in Figure1 by the means of users (Mun Y, Joyce D, Jae S and Janice C, 2006, 350‐363).

Individual reactions to use of technology

Purpose of use of technology

Real use of technology

Figure 1: Underlying factors in user technology acceptance models Technology acceptance model (TAM) was first introduced by Davis et al. in 1989 (Figure 2). This model which has been mainly implemented in North America is based on two major factors called usefulness and ease of use. In most of the previous researches, a direct correlation is proved to exist between these two factors and technology acceptance. Therefore, scientists in different organizations and industries are searching for the influential aspects of these two factors. In addition to these two factors, there are also some other factors called external variables which are not discussed here. The results revealed that the usefulness and the ease of use factors were both applied in most of the organizations and industries, while the external variables were specialized for a particular organization or industry due to the specific characteristics of their staffs (Chau and Hu,2001,699‐719;Delitheou and Maraki,2010,39‐45).

Perceived usefulness

External variables

View

Purpose of use

Use

Perceived ease of use Figure 2: Acceptance technology model Perceived usefulness and perceived ease of use can be seen in Figure 2. A brief description of these two factors is given below:

Perceived usefulness: a degree that a person believes that the use of a specific system can increase his/ her performance (Jahangir and Begum, 2008). In the new technology, the perceived usefulness causes tendency and ability of an individual, as well as the purpose of use for that technology.

Perceived ease of use: a measure that a user expects to use a system without considerable effort (Ajzen, 1985, 11‐39; Fang, 2002, 145‐156). Based on the technology acceptance model, perception of an individual from the usefulness of a system is influenced by the fact that one believes that application of a system is easy.

By January 2000, 424 articles in scientific journals had cited technology acceptance model. Through these years, this model has been found robust enough to predict the technology acceptance by the users.

3. Conceptual model and research questions A review on the theoretical basis and the existing literature in the field of new technology acceptance implies that there are many influential factors in this field. The independent variables in Figure 3 are the most important ones. The discussions on implemented technology management and its operation in the society are also addressed in some studies. A summary of all the factors and the conceptual model is illustrated in Figure 3.

100

Marzieh Faridi Masouleh and Hadi Mohammadzadeh

Independent variables: 1. E-readiness 2. Ease of use 3. Tendency to use 4. Expected operation

Dependent variable: Acceptance Technology (successful implementation of electronic municipality)

Intervention variable: Information Technology Management

Figure 3: Conceptual model Based on the model shown in Figure 3, research questions can be listed as below: Major research question:

Which factors are the most important ones among the studied factors in the implementation of electronic municipality?

Minor research questions:

How is the relationship between each variable of the independent variables and its implementation?

To what extent does the variable of information technology management influence implementation of the electronic municipality?

4. Methodology Current research obeys a descriptive surveying method. The surveying tool here is the questionnaire which is designed by 32 main questions within Likert Spectrum as well as some demographic questions. The statistical population of the study was composed of 135 managers and experts in Rasht municipality. According to the table of Krjsi and Morgan, some 100 questionnaires were distributed among the participants.

5. Results In this paper, regression correlation coefficient was employed for analysis of the relationships between these two variables. The correlation between each of the four independent variables and the dependent variable of implementing electronic municipality was significant at 1% error content. Thus it can be argued that there is a significant correlation between these two variables at 99% confidence level. The amount of correlation between the four abovementioned key factors and implementation of the electronic municipality are 0.466, 0.370, 0.458, and 0.278, respectively. All these values are statistically significant (p<0.01) (see Table 1). Table 1: Correlation matrix between key factors and implementation of electronic municipality Variables Implementation of electronic municipality Ease of use Expected operation Tendency to use E‐readiness

Implementation of electronic municipality 1

Ease of use

Expected operation

Tendency to use

E ‐ r e a d i n e s s

0.466**

0.370** 0.458** 0.278**

0.679** 0.711** 0.563**

1 0.802** 0.252*

1 0.459**

In the next step, multiple regression is used and each of the independent variables is examined in the presence of dependent variable by introducing them. Table 2 represents that F (4, 95) = 8.02 is statistically significant (p<0.0001). So, the conceptual model presented in Figure 3 would also be significant. However, Durbin Watson statistics (D = 2.04) showed that it is possible to apply regression analysis. It is obvious that an amount of D between 1.5 and 2.5 would represent the possibility of using the regression test.

101

Marzieh Faridi Masouleh and Hadi Mohammadzadeh Table 2: Variance analysis and Durbin Watson tests for the significant model Range of changes Regression Remained

df Mean squares

1029.476 4 3047.274 95

257.369 32.077

P‐Value Durbin Watson test

8.02

0.0001

2.04

According to Table 3, ease of use (B = 0.420, p< 0.034) and tendency to use (B = 0.738, p< 0.029) indicate significant roles in the implementation of electronic municipality. For determination of the most important factor among the factors under study here, Beta standard coefficient is applied. The greatest proportions are associated with the tendency to use and the ease of use, respectively, so the main factor would be tendency to use. Table 3: Standard and Non‐standard coefficients for implementation of electronic municipality via predictor variables (independent variables) Coefficients of predictor variables Non‐standardized Coefficients Standard Coefficients B Standard Errors Significance level T BETA Constant 16.564 2.129 0.0001 7.779 Ease of Use 0.420 0.196 0.034 2.146 0.316 Expected Operation ‐0.212 0.363 0.559 ‐0.586 ‐0.096 Tendency to Use 0.738 0.87 0.029 2.910 0.322 E‐readiness ‐0.023 0.112 0.839 0.204 ‐0.023 According to the findings, the expected implementation of electronic municipality equation is: The implementation of electronic municipality = 16.564 + 0.316 (ease of use) + 0.322 (tendency to use) Finally, the statistical case was divided into two sample groups with good or bad viewpoints toward information technology management. For analyzing the influence of information technology management on acceptance technology, T‐Student test is employed. The significance level of T test is 0.04 and is smaller than 0.05. As a result, it was perceived that the information technology management has modified the influences of the independent variables. According to the mean value, it can be declared that the technology acceptance will be greater if the operation of information technology management is better from municipality staffs point of view. In other words, according to the results obtained from comparison between the means of independent cases in Table 4, there is a significant difference between the mean values of acceptance technology from electronic municipality in two states of information technology management (good or bad) showing a 5% error content. Therefore, it can be concluded that information technology management has modified the influences of the independent variables. Table 4: The amount of acceptance technology from electronic municipality in two states of the information technology management IT management position Good Bad

Number Mean Mean Standard Error T‐Student Sig. Level 64 36

27.5 24.8

6.290 6.357

‐2.059*

0.04

Degree of Freedom 98

6. Conclusions and future works There are many factors such as culture, social influence, ease of use, tendency to use, and e‐readiness that influence the implementation of a successful electronic municipality. The first step for having a successful electronic municipality is to properly identify these factors. Having identified these variables, many questions may raise, e.g. to what extent these variables are influential on the implementation of a successful electronic municipality and which variable has the greatest proportion of influence. The purpose of current work was to find these influential factors and evaluate their role in the implementation of electronic municipality. The case

102

Marzieh Faridi Masouleh and Hadi Mohammadzadeh study was done in Rasht municipality for this purpose. The obtained results uncovered that all of the independent variables including ease of use, tendency to use, expected operation, and e‐readiness demonstrate a significant linear relationship with the dependent variable that is implementation of electronic municipality. The greatest proportions in the implementation of electronic municipality are attributed to tendency to use and ease of use, respectively. By introduction of the information technology management, the proportion of each of these variables is changed in the implementation of electronic municipality, though the greatest proportion still goes to tendency to use and ease of use. There is a suggestion for future work as a follow‐up study: electronic municipality can be simply simulated and be made available for a statistical population during a limited period of time. The participants could then be asked to answer a questionnaire electronically via the site. Therefore, it will be possible to compare the results in two states; before using the technology and after experimental use of it.

References Ajzen, I. (1985) “From intention to action: A theory of planned behavior”, In J. kuhl & J. Beckman, Action‐control: From cognition to behavior, Berlin Heidelberg: Springer‐, pp 11‐39. Chau, P. Y. K. and Hu, P. J. –H. (2001) “Information Technology Acceptance by Individual Professionals: A Model Comparison Approach”, Decision Sciences, Vol. 32(4), pp 699‐719. Davis, F. (1993) “User Acceptance of Information Technology: System Characteristics, User Perceptions, and Behavioral Impacts”, Int. Man‐Machine studies, Vol. 38, pp 475‐487. Delitheou, V. and Maraki, M. (2010) “Research into citizens’ attitude towards electronic municipal services (e‐local government)”, Journal of Public Administration and Policy Research, Vol. 2(3), pp 39‐45. Fang, Z. (2002) “Government in digital era, Practice and development”, International Journal of Management of Computer, The Internet Management, Vol. 10, No. 2, pp 145‐156. Jahangir, N. and Begum, N. (2008) “The role of perceived usefulness, perceived ease of use, security and privacy, and customer attitude to engender customer adaptation in the context of electronic banking”, African Journal of Business Management, Vol. 2(2), pp 032‐040. Mun Y. Yi, Joyce D. Jackson, Jae S. Park and Janice C. Probst. (2006) “Understanding information technology acceptance by individual professionals: Toward an integrative view”, Journal of Information & Management, Vol. 43, pp 350–363.

103

E-Procurement in the Era of Cloud Computing Antonio Grilo1, Ricardo Jardim-Goncalves2 and Sudeep Ghimire 3 1 UNIDEMI, FCT-UNL, Assistant Professor, Lisbon, Portugal 2 CTS, FCT-UNL, Associate Professor, Lisbon, Portugal 3 CTS, FCT-UNL, PhD Candidate, Lisbon, Portugal acbg@fct.unl.pt

Abstract: This paper aims to present the Cloud-Marketplaces approach that delivers a new paradigm to e-marketplaces. The proposed Cloud-Marketplaces architecture is a combination of the Cloud Computing paradigm and Service-Oriented Architecture (SOA), and decouples the traditional e-marketplaces into E-marketplaces and Community Services Clouds. The new e-marketplace architecture is being experimented in a real business scenario, and the case study of Vortalway, an industrial-based research project conducted by a major international e-Marketplace, is presented. The paper concludes that despite the Cloud-Marketplaces paradigm is technically a sound concept, besides the engineering and technological issues, it poses interesting challenges regarding business models. Keywords: e-marketplaces, cloud computing, SOA, cloud-marketplaces

1. Introduction It is acknowledged that existing e-commerce platforms do not provide an adequate foundation for Business to Business applications. Hence, their architecture, business processes, interoperability and the integration to back-end systems is usually very poor if done at all. Lack of interoperability between e-marketplaces, is also making cross-platform information flows and commercial transactions impossible. This paper proposes a conceptual approach for overcoming some of the shortcomings of earlier researchersâ&#x20AC;&#x2122; and practitionersâ&#x20AC;&#x2122; approaches. In the advent of new computing paradigms like Service-Oriented Architecture and Cloud Computing, the challenge of this research work is, Can Cloud Computing paradigm sustain a new way to develop e-marketplaces platforms and their businesses, particularly as some of their services are sometimes offered as Software as a Service (SaaS)? This paper presents current research on the concept of the CloudMarketplaces, aiming to deliver a new paradigm to e-marketplace vendors. The Cloud-Marketplaces architecture is proposed and a description of the application on scenarios experimented in real business scenario, in the case study of Vortalway, is presented. The research method is based on Action Research, that considers that the investigator influences the action (Reason & Bradbury, 2001); the research is simultaneous conducted with the action (Coughlan and Coughlan 2002); and the reflective way of Action Research is critical because aggregates the thought on comments and relevant literature for planning the next cycle of action and research. The paper is organized as follows. Section 2 reviews the current state of play of e-marketplaces and their main challenges in Cloud Computing and SOA paradigms. Section 3 lays out the Cloud-Marketplace concept for the development of new transactional and collaborative e-marketplaces, based on SOA, and Cloud Computing. Section four presents the Vortalway case study, describing part of the validation and implementation of the conceptual model. Finally, Section 5 concludes, with some words on the relevance and applicability of the Cloud-marketplace approach and the challenges to its development

2. State of Play for E-marketplaces An e-marketplace is a virtual space in an electronic network, an inter-organizational information system that allows the participating buyers and sellers to exchange information about prices, product offerings, and an Internet-based electronic commerce platform that matches multiple buyers and suppliers in transactions along with traditional project-based collaborative functions (Bakos, 1998). Despite the widespread use of e-marketplaces with transactional and collaborative functions, there is today a plethora of electronic formats, product descriptions, and classification schemes, seeking to provide guidelines for the exchange of data between transacting agents, and regarding e-procurement especially, the challenge of having electronic catalogs (e-Catalogs) among buyers and suppliers (Jardim-Goncalves and Grilo, 2006). Apart from the need for standardizing processes and messages for conducting business electronically, the adoption of additional standards is necessary for unifying the manner in which products and services are described in projects in a digital format. Besides the e-Catalogs issues, e-procurement presents several additional technical challenges that create interoperability concerns regarding electronic marketplaces at the

104

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire European and global scales, and that are being addressed by several R&D and industry initiatives. For example, European public tendering procedures require that companies submit certificates and attestations to prove that they comply with selection and exclusion criteria. Electronic business certificates that are interoperable are thus one of the major challenges (CEN/ISSS, 2005). Electronic signatures interoperability is also an important issue, despite the availability of technical standards, such as X.509v3 for electronic certificates. However, even though electronic signatures are relatively widespread today, in practice certification authorities do not recognize each other in every case, thus creating identification hurdles. Other less challenging issues are e-ordering and e-invoicing, as the ongoing standardization work in CEN/ISSS WS/BII (CEN/ISSS, 2005) is becoming mature, and these business documents are now standardized and XML-based (at least for system to system communication). E-marketplaces architectures have been embracing new technological developments like the emergence of Service-Oriented Architectures (SOA). The World Wide Web Consortium (W3C) refers to the SOA as “a set of components which can be invoked, and whose interface descriptions can be published and discovered” (W3C, 2012). Also, and according to Microsoft, the goal for SOA is a worldwide mesh of collaborating services that are published and available for invocation on a service bus (Microsoft, 2010). SOA does not consider the services architecture from merely the technology perspective, but also proposes a normalized Service Oriented Environment (SOE) offering services’ description, registration, publication, and search functionalities. Placing emphasis on interoperability, SOA combines the capacity to invoke remote objects and functions, i.e., the services, with standardized mechanisms for dynamic and universal service discovery and execution. Today, the use of W3C’s web services is expanding rapidly as the need for application-to-application communication and interoperability grows. Web services are implemented in XML (eXtended Markup Language). The network services are described using the WSDL (Web Services Description Language), and the SOAP (Simple Object Access Protocol) is the communication protocol adopted. The registration of the services is in the UDDI registry (Universal Description, Discovery, and Integration). The Web Services Interoperability Organization, WS-I, is an organization supporting Web services interoperability across platforms, operating systems, and programming languages, and that has been developing efforts for the convergence and support of generic protocols for the interoperable exchange of messages between Web services (WS-I, 2010). Less clear has been how Cloud Computing can impact on e-marketplaces. Cloud Computing involves a set of key technologies to address resource sharing based on business requirements. Cloud Computing can be seen as an evolution over the traditional hosting and application service providers, though more aligned with the service-oriented environments, and less with client-server architectures (Weiss, 2007; Vaquero et al. 2009). Cloud Computing follows two key enabling technologies that play very important roles in this revolutionary phase: Virtualization Technology and SOA. The virtualization technology handles how images of the operating systems, middleware, and applications are pro-created and allocated to the right physical machines or a slice of a server stack. The SOA is used for addressing componentization, reusability extensibility, and flexibility. Within the Cloud-Computing paradigm, there are some variations on what service is included. The most common reference is the Cloud Software as a Service (SaaS), which is the capability to use the provider’s applications running on a cloud infrastructure (Vaquero et al., 2009; Mell, and Grance, 2012). The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., webbased email, Google Docs, etc.). There is no management of the underlying cloud infrastructure (network, servers, operating systems, storage, or even individual application capabilities), with the possible exception of limited user-specific application configuration settings.

3. Evolution to Cloud-marketplaces The proposed evolution to e-marketplaces is here designated as Cloud-Marketplaces, which is a combination of the Cloud Computing paradigm and Service-Oriented Architecture (SOA), and decouples the traditional emarketplaces into E-marketplaces and Community Services Clouds (Figure 1). E-marketplaces define the working environment in which buyers, sellers, and economic agents can transact and collaborate. Therefore, the e-marketplace will be where users configure and interact their procurement, transactional, and collaborative business environments and platform functions. From the e-marketplace service provider perspective, each client, remotely and through the Internet, can have as many e-marketplaces as they need, and each client or each economic agent is likely to have its own e-marketplace.

105

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire Since there is a need for disparate working environments for the procurement, transactional, and collaborative processes, according to companies’ requirements, markets, or projects, the approach is to “virtualize” the emarketplace (Sottomayor, 2009).

Figure 1: Vision of a Cloud-marketplace Ecosystem The Community Services Cloud manages the back-end of the services made available on the E-Marketplaces. It sets the rules, profiling, authentication, security, generic and specific functionality, in a federated environment managed in a service-oriented basis. It also shall be responsible to guarantee interoperability between the various E-Marketplaces, being a connecting hub, along with sustaining interoperability with external emarketplaces (Balocco et al. 2010). The envisaged architecture is also conceived to support integration with other cloud-based applications, which can occur in two different approaches (Wayner, 2008). Firstly, on a Service Consumer approach, part (or all) of the applications of the E-marketplace are provided in an embedded way to other cloud-based applications and systems (e.g. other e-commerce applications). In this approach, the platform can also incorporate in an embedded way other cloud-based applications from other software vendors or suppliers. Secondly, on a Service Provider approach, with clients having the ability to design customer specific services, which are not on the scope of the services provided by the E-marketplace. An important concept in the envisaged Cloud-Marketplace platform is how communication flows occur between the Community Services Cloud and the E-marketplaces. The conceptual architecture of Cloud-Marketplaces is shown in Figure 2. By looking at the conceptual architecture it is possible to understand that the system is organized into two different sets of conceptual layers. The outer layers (grey boxes) can be regarded as the structure necessary to provide the cloudmarketplaces services in a cloud based model, i.e., they contain all the necessary artifacts that make a cloud system. The inner layers (numbered 1 to 7 in red) represent how the services are built, deployed, and accessed following an SOA based approach. The presentation tier’s main goal is to deal with the users’ interactions, providing them access to the CloudMarketplace services. These services are exposed either as services, which users can access and incorporate into their own private systems or as complete applications with dedicated end points following the SaaS model. At this level, the platforms delivers both as Web Applications and Web Services, traditional and advanced i) eSourcing services (Savings Management, Sourcing, e-Auctions, Supplier Info Management, Contract Management, etc.); ii) e-Commerce services (Market Intelligence, Leads Management, Sales Management, Offers and Promotions, Sales Contract Management, etc.); iii) e-Procurement services (Orders, Invoice, Payments, etc.); or can also deliver iv) collaborative services.

106

B2B

Business Process Layer Composition (Chorography, Orchestration)

Service Layer Atomic / composite

Service Components Layer

Operational Layer Custom Application

WSRP

Business Logic

Business Logic Software Platform Computing Platform

Business Logic Tier Data Tier

Portlet

QoS layer (security, management and monitoring of services)

Consumer Layer

Integration Layer (Enterprise Service Bus)

Presentation Tier

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire

OO Application

Figure 2: Cloud-markeplaces Conceptual Architecture The Business logic tier responsibility is to interpret on one hand the userâ&#x20AC;&#x2122;s request and according to this analysis, devise what methods are necessary and in what order they should be triggered so that the end result will comply with both the user expectation and the business constraints of the platform. The business process layer (4) is the key element for this. It triggers the methods and does it according to the order, coming up with a valid and always updated result. The service layer (3) in turn offers to the business process layer access to the methods so that they can be chosen and called upon for execution. The Data tier contains all the elements that will provide the continuity of the platform. It includes the methods whose interfaces are exposed on the business logic tier (2: service component layer) and also includes the resources that assure the proper foundations for the computational environment, like IT middleware services, operating system, runtime environment, hardware devices and so on (1: operational layer). The sixth and seventh layers of the architecture are transversal to the entire system. The sixth layer contains all the elements in charge of interoperability issues, while the seventh deals with the quality of service aspects, such as monitoring response time, availability, security, and so on. The dynamics of the architecture, which follows a SOA-based approach, shows how the services are deployed, exposed, and orchestrated into processes and applications, so that they can be finally consumed.

4. The Vortalway Case Study One of the first Cloud-Marketplace available in the market has been developed and implemented by Vortalway, an industrial pilot project that has been conducted by Vortal, a Portuguese leading e-marketplace company and third largest e-marketplace in Europe. Vortal has currently over 25,000 companies connected to its platform, covering the markets of Public eTendering, Construction, Health, Energy & Utilities, Industry and Office & Supplies, mainly in Portugal, but entering now in Spain and UK geographical markets. In 2010, 3.7 billion Euros worth of transactions were carried out on its platforms.

107

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire The Vortalway system can act as both a service provider and a consumer. As provider, it exposes a number of atomic or aggregated services that can be used by other service consumers and applications. As a consumer it can incorporate services exposed by external providers into its own applications or use them to make other composite services. To manage the services deployment and interactions, a logical component of the Vortalway’s baseline architecture, called Vortalway Service Platform (VSP), is used as a central point of mediation between the platform’s consumers and providers, providing an interoperable and manageable service consumption environment. VSP is important to achieve interoperability between providers and consumers since it will be responsible for data adaptation and protocol transformation as necessary. At the same time, it will provide a centrally manageable environment in order to ensure that the QoS is maintained as per the agreement between the various parties. The overall operation scenario of Vortalway involves a number of clouds. Vortalway itself consists of a private cloud and a public cloud, and the overall platform has the ability to interact with independent third-party clouds, which in aggregate can be designated as a Cloud-Marketplace Ecosystem. Each cloud makes use of standard service consumption gateways and service provider interfaces for consuming and providing service, respectively. The standard interfaces are the points through which cloud service consumers access and monitor their contracted services. The interface covers SLA negotiation, service access, service monitoring, and billing. This interface is also the interface through which a cloud service developer interacts with a cloud service provider to create a service template that is added to the service catalog. In the case of Vortalway the service interface is a Web Service. Vortalway basically handles the case of SaaS, which is easily achieved by using the standard interfaces explained above. Note that IaaS and PaaS can also be handled similarly by adding more connection points. For instance, if IaaS is to be implemented, then add the connection points at the data layer and expose the accessible interface as web service. The Vortwalway Cloud-Marketplace Ecosystems concept is being tested in real-life context pilots, validating it in different operational scenarios e-commerce processes, and this paper describes one of these piloting scenarios: “Request For Proposal” and “Submit Proposal” within an environment that allows multiple heterogeneous e-platforms to link with each other. In order for this to occur, the environment supports transactions between platforms; ensure opportunity dissemination through all; manage data flow among them, and other interoperability activities. This environment is able to interpret, map, and transform, from all of the involved platforms, values, functionalities, rules, and other aspects that are relevant to reach interoperability within the environment, including covering mobile devices platforms. This means that some of the opportunities showed in the platform (user chooses which type to receive) are available for viewing on a mobile platform. The reason for supporting this choice is to empower the user to act more promptly upon new opportunities placed in the market. Figure 3 depicts the conceptual model for a transactional user story: (1) Launch of a RFP through vortalGOV and distribution through VortalCommunity; (2) RFPs are directed to suppliers’ platforms within Vortalway platform, in this case VortalOFFICESUPPLY and VortalECONSTROI); (3) RFPs regarding supply of products and equipment are distributed to two different external platforms (X and Y platforms), where the two potential suppliers are enrolled; (4) Only one of the suppliers replies to the RFQ, replying through its platform, and is then re-directed by the VortalCommunity. Suppliers that are in the Vortalway platform, replying directly over their platforms and then seamlessly channelled by Vortal Community to the contractor’s VortalGOV platform. Although not covered in detail in this user case, the user story can likewise address the RFP being issued from an external partner platform. Conceptually, there is no important difference from the interaction described above. It still deals with the dissemination of an RFP and presentation of the due reply. The differences are that the RFP is now written on the partner’s platform, its dissemination is only to Vortalway’s platform, and the reply now comes from Vortalway’s platform. Hence, the buyer from the partner platform will submit an RFP on his platform and this one will seamlessly trigger an interoperation service on Vortalway’s platform to send the RFP. Upon receiving all the RFP data, and since there are many marketplaces on Vortalway’s domain (Cloud-Marketplace), the first set of activities is to decide which marketplace best fits the RFP (which an algorithm determines if there are no specific instructions). After this, the RFP is sent to the proper marketplace and shown to all eligible suppliers. If one of the suppliers decides to reply to the RFP (produce a proposal), then through its interoperation service, Vortalway’s platform will handle the delivery of the reply back to the partner’s platform.

108

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire

Figure 3: Conceptual model for User Story RFP-Proposal

5. Conclusions The proposed Cloud-Marketplaces approach aims to deliver a new paradigm to e-marketplace vendors. The envisaged Cloud-Marketplaces architecture is being experimented in real business scenario by Vortalway, an industrial-based research project conducted by a major international e-Marketplace. The project, that is in its roll-out phase, has delivered e-sourcing, e-commerce and e-procurement full functionality to Vortal’s electronic platform, offering which is envisaged as being strategic considering their business options to move to other countries in Europe, and to covering disparate vertical markets like public administration, construction, health, etc. Nevertheless, the implementation of the Cloud-Marketplaces paradigm, besides the engineering and technological issues, poses interesting challenges regarding business models. The new possibilities brought by the new architecture do also require such new business models that move away from traditional fee or transaction-based revenue mechanisms, which is also currently being researched.

Acknowledgments Authors would like to thank to the company Vortal for partially funding the research work. Particularly we would like to thank you Ermelinda Veloso, Miguel Soares and Abílio Mendes. The research project is also partially funded by IAPMEI and the European Funds QREN, which provide the research grants of Sudeep Ghimire.

References Bakos, Y. (1998) Emerging Role of Electronic Marketplaces on the Internet, Communication of the ACM, vol. 41, No.8.

Balocco, R., Perego, A. and Perotti, S. (2010), “B2b eMarketplaces – A classification framework to analyse business models and critical success CEN/ISSS Workshop on Electronic Procurement (2005), CWA 15236, ftp.cenorm.be/Public/CWAs/eEurope/eProc/CWA15236-00-2005-Feb.pdf, last access November 2012 Coughlan, P., & Coughlan, D. (2002). Action research for Operations Management. Internacional Journal of Operations and Production Management V.22, nº2 , 220-240. Mell, P. and Grance, T. Draft NIST Working Definition of Cloud Computing, http://csrc.nist.gov/groups/SNS/cloudcomputing/index.html, last access February 2012 Microsoft (2010), “The Service Oriented Architecture”, http://msdn.microsoft.com/architecture/soa/default.aspx, last access in April 2010

109

Antonio Grilo, Ricardo Jardim-Goncalves and Sudeep Ghimire R Jardim-Goncalves, A Grilo, Challenging the interoperability between computers in industry with MDA and SOA, 2006,Computers in Industry, Elsevier Reason, P., & Bradbury, H. (2001). Introduction : Inquiry and Participation in Search of a World Worthy of Human Aspiration. Handbook of Action Research, Participative Inquiry and Practice, Sage publications, Thousand Oaks , 1-14. Sottomayor, et.al., (2009), An Open Source Solution for Virtual Infrastructure Management in Private and Hybrid Clouds. IEEE Internet Computing, Special Issue on Cloud Computing, October Vaquero, L.M., Rodero-Merino, L., Cáceres, J. , Lindner, M. A Break in the Clouds: Towards a Cloud Definition ACM Computer Communication Reviews. 2009;39(1):50-55 W3C, World Wide Web Consortium, http://www.w3c.org , last access in November 2012 Wayner, P (2008), ‘Cloud versus cloud: A guided tour of Amazon, Google, AppNexus, and GoGrid’ Weiss A. (2007) Computing in the clouds. netWorker 1111(4):16–25 WS-I (2010), Web Services Interoperability Organization, http://www.ws-i.org, last access in April 2010

110

A Comparative Study on Waseda e‐Government Indicators Between Vietnam and Japan Nguyen Manh Hien Graduate School of Asia‐Pacific Studies, Waseda University, Tokyo, Japan nguyen.hien@akane.waseda.jp Abstract: E‐Government is increasingly becoming an important concept in the application of Information and Communication Technologies (ICT) in administration and management of government. E‐Government may be applied by the legislature, judiciary or administration, in order to improve internal efficiency, the delivery of public services, or processes of democratic governance. The use of ICT can deliver the services and information to citizens, effective interactions with business and industry, citizen empowerment through access to information, or more efficient and effective public sector management in most countries not only developed countries but also in developing countries. The Waseda world e‐Government ranking has introduced since 2005 with the aim of evaluating, identifying, measuring and facilitating policies for the best practice of e‐Government development, application of ICT in administration and leadership of 55 countries (2012). Based on this survey, this study introduced some critical remarks related to e‐Government development in Vietnam and suggestions to develop e‐Government in Vietnam. Through the ninth consecutive year of monitoring and surveying the development of e‐Government with seven main indicators, thirty sub‐indicators and 154 questions, the author found that the assessment of e‐Government development in a country based on Waseda e‐Government ranking main indicators is completely foundation and high reliability Keyword: e‐government, ranking, Waseda, indicators, Japan, Vietnam

1. Introduction 1.1 Background One of the most remarkable and dramatic changes of ICTs usage in recent decades has been the explosion of interest in Internet not only in private sectors but also public sectors. The usage of ICTs means in public sector is so called digital or electronic government (e‐Government) which is a global phenomenon and public servants around the world are adopting novel ways to leverage ICT to better serve their constituents (Marchionini et al., 2003) Nowadays, many governments in developing countries have recognized the importance role of ICT application into life and have taken progressive steps toward ICT use to streamline business processes and communication as well as to promote and delivery services better to citizens.

1.2 Research objectives The aim of this research is to show the overall picture of e‐Government in Vietnam. The main objective of this research is to assess e‐Government development in Vietnam by using main seven indicators from Waseda e‐Government ranking and compare it performance with Japan. Through this research, the strengths, weaknesses are identified as finding and helping readers understand e‐Government development in Vietnam. Finally, this research provides recommendations to Vietnamese government to overcome the weaknesses and develop more effective in future.

2. Literature review 2.1 E‐Government definition There are a lot of researches and definitions regarding e‐Government derived from various view points, each with its own perspective (Chang Chih‐Hao, 2011) E‐Government has several aspects, including social, technical, economic, political and public administrative. However, most dominating concepts of e‐Government arise from the technical perspective and a combination of the socio‐economic and public administrative perspectives (Mahmud et al., 2010). Nevertheless, whatever the perspective, the transmission of information and deliver e‐services to citizens have an importance role in building and development e‐Government.

111

Nguyen Manh Hien Therefore, e‐Government can be defined as the use of primarily Internet‐based information technology to enhance the accountability and performance of government activities. These activities include government’s activities execution, especially services delivery, access to government information and processes, and citizens and organizations participation in government (DeBenedictis et al., 2002)

2.2 Waseda e‐Government ranking Waseda e‐Government Ranking has been introduced annually since 2005 and conducted by Waseda Institute of e‐Government. It is a measure of evaluating the application of ICT in administration and leadership of each countries government. The aims of this ranking are to advance the state of ICT practice, to identify and facilitate policies for the best practice of e‐Government development and to improve the quality of life and sustainable community development as well as e‐democracy in Digital Society in the long term goals. For the first time, it has only 4 main indicators. Over the past few years, this is the eighth consecutive year of monitoring and surveying the development of e‐Government worldwide, the primary categories of indicators used in the benchmarking model continue to include 7 main indicators, they are: Network Preparedness (NP), Required Interface‐functioning applications (RI), Management Optimization (MO), National Portal (P), CIO in government (CIO), e‐Government Promotion (EGP), and e‐Participation (EP) (Press Released, 2012). Waseda e‐Government Ranking is a group of sub‐indicators to evaluate the overall e‐Government development in one country, from the policy development to e‐services implementation, from the management optimization to e‐Government promotion. To better the evaluation e‐Government development in one country, from 2010 the ranking was added e‐participation indicator. The main methodology is to evaluate all indicators and sub‐indicators through 154 questions (Waseda, 2012). Each indicator has maximum raw score, minimum raw score and weighted score (by percentage). Based on these scores, the ranking will determine the final score for one indicator and total score. Waseda e‐Government Ranking has been constant changes made to improve the quality and quantity of the ranking. The ranking is based on a principle of executing a comprehensive and systematical survey. The survey analyzes the development of government websites, ICT initiatives as well as the relationship between government and their stakeholders.

2.3 E‐Government in Vietnam Vietnam is the easternmost country on the Indochina Peninsula in Southeast Asia. With an estimated 88 million inhabitants as of 2012 (GSO, 2012), it is the world's 13th‐most‐populous country. In a pass decade, Vietnam ICT has been growth very fast and became the world’s second fastest growing ICT market in 2010, estimated 20%‐25% annually (ICT to Vietnam, 2011). Up to date, the number of Internet users reached a whopping 31 million and accounted 35.4 percent of population, of which 4.3 million are landline broadband subscribers (VNNIC, 2012) From 1994, Vietnam had the Information Technology (IT) program, the aims of this program was training officers and setup the network and initially computerization. The concept of e‐Government has been developed around a decade ago with a project called 112, which focused on computerization of state administration in the framework of the Public Administration Reform (PAR), starting in 1999. The project 112 has been implemented in all ministries and government department and all 64 provincial governments developed their portal to provide information and services to citizens and business. However, the project was discontinued due to inefficient and did not achieve its stated purpose. In late of 2007, the government was preparing for its next e‐Government project by issuing Decree 64 (Vietnam ICT report, 2008). The aims this project was to make IT application plan in state bodies with enhance government capability of management and more effective transparent government from local to central government with more concentration on serving citizens as well as initially provide simple e‐services. Until late 2010, Vietnam had the first e‐Government master plan, the period from 2011 to 2015. The objectives of this master plan are to provide all basic public e‐services with online registration, transaction (online payments) and setup national databases. Up to date, Vietnam has many favorable conditions for implementing e‐Government, such as a combination between government and business sectors (Viettel – the biggest telecommunication company in Vietnam deployed free internet connection to all training education),

112

Nguyen Manh Hien broadband market in Vietnam has started to grow strongly annual growth running at 25% (BuddeComm, 2011) and the Internet subscribers around 35.4 percent (VNNIC, 2012). All ministries and government department and all 64 provincial governments developed their portals which were link to the national government portal (http://www.chinhphu.vn) with the aim to provide information and some e‐services to citizens and business, but all portals contain static information and news rather than provide online services (e‐services), therefore the benefits of Vietnamese e‐services were still very limited and inefficient. As discussed above, e‐Government in Vietnam has a favorable environment to develop, but until now e‐Government is in the process of Administration Reform with very limited e‐service to public. Therefore this research would like to know how to develop e‐Government in Vietnam, especially to delivery e‐service to citizens and which are the e‐Government models suitable for Vietnam by mapping e‐Government development in Japan.

2.4 E‐Government in Japan Japan is the one of the most advanced ICT nations for ICT sector. The ICT infrastructure is well provided and widely covered: 99% of households have access to and 85% of the populations used at least once the Internet in 2011 (Japan e‐Government report, 2012). The government has steadily made efforts to expand the utilization of online services as a main part of the e‐Government initiative since 1999. In 2011, online applications at the national level covered 55% of all the administrative procedures. To enhance e‐Government, the government is making a new action plan to improve online applications for the further promotion of their use. E‐Government in Japan divided into 3 stages (Global e‐Governance Series, 2010). The first stage is nd Administration, Informatization, and focused on optimizing system. The 2 stage called e‐Government stage rd with aimed at building online system for the public. The 3 stage called the One Stop e‐Administrative Service. In 2001, the IT Strategic Headquarters created e‐Japan Strategy, after two years, they introduced e‐Japan Strategy II and May 2010, The IT Strategic Headquarters announced the New IT Reform Strategy. One of the goals of the New IT Reform Strategies is to complete IT‐led reform by FY 2015 before any other country in the world. The other goal is to transform Japan into a cooperative‐model IT society with the autonomy to make sustainable development with the capacity that anyone can subjectively participate in society’s activities. The e‐health, e‐education and e‐Government are the three major priorities in the plan (Japan, 2012) Online application systems such as e‐Tax, e‐Payment, and Social Security services provide transactional operations (covering all the service processes including requests, payments, decisions, and delivery). There are about 20.000 online services by e‐Government, now Japanese government is carrying out various policies from the standpoints of user‐friendliness and cost savings with the aim of achieving convenient, effective, and active local e‐Governments.

3. Research methodology One important method that used in this research is documentary research. It was used to analyze current status of Vietnam e‐Government and to compare it with e‐Government development in Japan. Documentary research involves the use of texts and documents as source materials: government publications, newspapers, certificates, census publications, novels, film and video, paintings, personal photographs, diaries and innumerable other written, visual and pictorial sources in paper, electronic, or other `hard copy' form (John Scott, 2006)

3.1 Data collection In this research, all the data were collected from the Waseda e‐Government Ranking reports 2005 to 2012. The e‐Government reports, ICT white book in Vietnam and Japan as well as some articles were picked up to analyze and evaluate. All e‐Government statistics in Vietnam were picked up from government website, while the data in Japan from ICT white book and government’s reports.

113

Nguyen Manh Hien

3.2 Data analysis and finding 3.2.1 E‐Government development in Vietnam and Japan Institute of e‐Government Waseda University published e‐Government ranking in 2005 for the first time through 8 consecutive years of monitoring and surveying, we received comparative analysis of e‐Government development in Japan and Vietnam during the period 2005‐2012, the results are presented as table below: Table 3.1: E‐Government development ranking for Vietnam and Japan Year 2005 2006 2007 2008 2009 2010 2011 2012

Japan Score Ranking Rank Change 93.80 7 ‐ 60.10 4 3 61.44 4 0 62.20 5 ‐1 82.30 5 0 76.80 6 ‐1 86.85 6 0 81.50 8 ‐2

Vietnam Score Ranking Rank Change 60.30 23 ‐ 32.85 32 ‐8 33.33 31 1 39.80 30 1 40.77 31 ‐1 44.00 34 ‐3 55.70 38 ‐4 52.10 38 0

Vietnam e‐Government ranks were 23rd, 32nd, 31st, 30th, 31st, 34th, 38th and 38th in 2005, 2006, 2007, 2008, 2009, 2010, 2011 and 2012. E‐Government development in Vietnam was erratic and tends to decrease in the final ranking. If we compare the score, it increase year by year, excluded 2012 which shows that Vietnam government had specific actions led to the impact of changes in the e‐Government development. But view all the ranking, Vietnam’s position was descending. It demonstrates that, the e‐Government development in other countries is also fast and efficient. In 2012, the total score in Vietnam was dropped down due to there were no policies updated till date but in the final ranking, Vietnam still kept the same position compared 2011. Compared with Vietnam, the downward tendency has been seen but it wasn’t clear and small in Japan. To compare with the world economic situation during this time, the world economic crisis may have affected to the e‐Government development in each country. Japan has always been in the top countries in term of e‐Government development in Waseda e‐Government th ranking and it got high score in all indicators. Japan tied at 7 place, after one year Japan jumped 3 steps and th tied at 4 place, the same place with following year. In 2008 and 2009, Japan was dropped down to 5th place. In 2010, Japan was dropped down continuously and stood at 6th place. In 2012, the newest e‐Government ranking issued by Waseda, Japan had a big change in the final ranking and dropped down to 8th place, same situation with Vietnam, the total score in Japan also dropped down (81.50) compared with 2011 but still very high compared with Vietnam (52.1). 3.2.2 E‐Government indicators analysis in Vietnam and Japan To assess the details of e‐Government development in Vietnam, this research will break down e‐Government ranking by focus on each indicator. These indicators reflect and evaluate the most detailed for e‐Government development in one country. As discussed above, Waseda e‐Government Ranking includes seven indicators each indicator has raw score, max score, min score and weighted score. Table 2 will show the weighted score for each indicator: Table 2: Weighted score for indicators Indicators Weighted score Network Preparedness 10% Required Interface 20% Management Optimization 20% National Portal 15% CIO 15% e‐Government Promotion 10% e‐Participation 10%

114

Nguyen Manh Hien Based on raw score, max score and weighted score, the total score for one indicator will be determined and not exceed weighted score. It means that, for example the maximum score for Network Preparedness is 10 points. Waseda e‐Government ranking did not publish details final score for each indicator from 2005 to 2007 therefore we have the score for all indicators from 2008 to 2012. Table 3 is a breakup of E‐Government Development in its sub‐indicators – Network Preparedness, Require Interface, Management Optimization, National Portal, Government CIO, e‐Government promotion and e‐Participation Table 3: E‐Government ranking by indicators Year 2008 2009 2010 2011 2012

Country NP RI Vietnam 3 7.2 Japan 7.5 13.68 Vietnam 3 7 Japan 8 19.11 Vietnam 4 9.12 Japan 8 18.25 Vietnam 4 9.78 Japan 8.5 15.11 Vietnam 4.25 7.72 Japan 9 14.39

MO 8.36 14.55 8.91 18.56 16.36 18.18 7.27 18.18 10.91 20

P 10.15 10.17 10.25 13.22 11.44 12.97 9.15 13.22 10.17 13.22

CIO 5.68 9.26 6.20 13.68 8.82 12.35 8.38 12.35 10.15 13.68

EGP 5.41 7.04 5.41 9.73 5.41 8.92 5.41 8.38 5.41 8.92

EP N/A N/A N/A N/A 5.91 8.18 6.36 9.09 6.82 9.09

Network Preparedness is very important indicator for e‐Government development in every countries, based on the ITU (International Telecommunication Union) Network preparedness includes Internet users, Broadband subscribers and Mobile cellular subscribers per 100 inhabitants, in which the Internet users is the most important sub‐indicator marked the success of e‐Government or not. Through the score from 2008 to 2012 in Vietnam, we can see the increasing trend on Network Preparedness, which shows that the Vietnamese government realized the importance of telecommunication infrastructure is an essential communication and management tool in e‐Government and its use which has strong impact on the performance of e‐Government. However, this increase trend has only been due to the increase of mobile subscribers – a prominent feature in developing countries today (Table 4). Table 4: Breakdown network preparedness indicator (ITU data) Year 2005 2006 2007 2008 2009 2010 2011 2012

Internet Broadband Mobile Users Subscribers Subscribers Vietnam Japan Vietnam Japan Vietnam Japan 12.74 66.92 0.25 18.44 11.54 76.34 17.25 68.69 0.61 20.91 22.47 78.94 20.76 74.30 1.52 22.37 52.96 84.84 23.92 75.40 2.38 23.80 87.11 87.24 26.55 78 3.70 25.96 113.03 91.90 27.56 78.21 4.13 26.94 127 97.43 30.65 80 4.18 26.91 175.30 95.39 35.07 79.53 4.32 27.60 143.39 104.95

The number of Internet users per 100 inhabitants in Vietnam is increasing every year, but the increase is not significant around 3% per year. From 2005, the number is 12.74 per 100 inhabitants but this figure increased to 35.07 in 2012 around three times positive trend after seven years. But Vietnam still lags far behind Japan where the number was around 80 per 100 inhabitants in 2012. In the coming years, thanks to the combination between government and business sectors (Viettel Corporation – the biggest telecommunication company in Vietnam) deployed free internet connection to all training educational, hoping that the telecommunications infrastructure of Vietnam will be significantly improved in the future. Broadband is also an important factor in the development of e‐Government services, the use of broadband connections to help governments improve the efficiency in the deployment of new services such as m‐Government. In 2005 the number of broadband subscribers in Vietnam was 0.25 per 100 inhabitants, but in 2012 the number was 4.32 that mean, the use of broadband connection has increased more than 17 times within 7 years. This index shows that the government has recognized and strong invests for broadband

115

Nguyen Manh Hien infrastructure. Compared with Japan, although the number has increased near double in 7 years, but the gap between Vietnam and Japan is too big. As discussed above, a prominent feature in developing countries is booming of mobile users, Vietnam is no exception to this rule. In 2005, Vietnam has only 11.54 per 100 inhabitants mobile subscribers but in 2011 the number was 175.3 with the positive trend value of 15 in 2011. This increase will be a precondition for promoting e‐Government, especially mobile government. Management Optimization: Management Optimization indicator shows the usage of ICT for improving internal processes and measuring the government’s computerization efforts and the level of ICT integration. It also focus on e‐Government strategy at national level, this national e‐Government strategy which has time‐line for execution target and the targets have been achieved (Evaluate e‐Gov Ranking, 2012). The score of management optimization in Vietnam has been erratic over the period 2008 – 2012. In 2008, Vietnam received 8.63 score but in 2010, the number increased into 16.36 score, after one year in 2011, this indicator got only 7.27 score (Table 3.3). Through this figure which shows that the Vietnamese government has not long‐term strategies to develop e‐Government. After surveying, we did not find any information concerning e‐Government strategy in Vietnam at this time. th Therefore, for this indicator, Vietnam got very low score and tied at 26 place, after one year, in 2012 Vietnam tied near the last in indicator ranking. One of the reasons why this indicator received a very low score in 2012, because there were some policies and e‐Government projects expired while there have not new project been added. Compared with Japan, in 2008 the score Japan received was 14.55 and increase in the following years, in 2012 they got full score and tied at top of ranking on this indicator. The figure also shows that, Japan always updated and revised annual e‐Government development strategy. Required Interface related to e‐services such as e‐Tender system, e‐Tax system, e‐Payment system, e‐Voting system, Social Security Services, Civil Registration Services and e‐Health system. With this indicator, it has covered almost all the online services. Same with the Management Optimization, Vietnam has been erratic over the period 2008 – 2012 in this indicator (Table 3.3), because there are not policies and strategies e‐Government development, so there is virtually none e‐services in Vietnam now. However, some e‐services such as e‐Payment and e‐Tax also are implemented but the level of service is not high and citizens still have not believed in these services due to security issues. In 2008, the score was 7.2 after 5 years, the number was 7.72 the changing is not so much. This also shows that, most of the policies on e‐Government development in Vietnam are out of dated and needs to be replaced with other new plans and strategies to be suitable with the e‐Government trends in the world. Opposed to Vietnam, with there were about 20.000 online services issued by e‐Government in 2012 (Obi, 2012), Japan is one top countries not only in Waseda e‐Government ranking but also other rankings. Although in recent years there is a decrease in score on this indicator National Portal: is one‐stop‐shop service, looking at the overall of ranking, this is the stability indicator in terms of the score in Vietnam. In 2008, the score was 10.15 and in 2012 the score was 10.17 (Table 3.3). Unlike National portal in Japan, National portal is truly a gateway to perform daily transactions and they used portal to provide e‐service to citizens and businesses, but in Vietnam the National portal is simply a page that provides information and news rather than web service provider. Thus, to provide e‐Government better to all grassroots Vietnam need to integrate e‐services into national portal. Government CIO: as discussed above, Government CIO is a very important indicator. It is spitted into four elements: firstly the presence of CIOs in government; secondly, the extent of their mandate; thirdly, the existence of organizations which fostered CIO development, and finally, the special development courses and the degree/quality which taught CIO related curricula. Japan was continuously scoring high in the all Waseda e‐Government ranking (Table 3.3). In 2009, 2010, 2011 and 2012 surveys, Japan got a near full score while Vietnam was lower than Japan. In 2008, Vietnam got 5.68 score for this indicator, in 2009 was 6.2, till 2012 the score was 10.15. Compared with 2008 the score increased double, this is a good sign for developing e‐Government.

116

Nguyen Manh Hien In Vietnam, there is not CIO position but in all provincial government and government department, they had Information Technology (IT) director, who is in charge as a CIO position. Vietnam also has CEO&CIO club, annual this club awards for the most 10 typical CIO. The evaluating council of this award includes prestigious IT professionals, representatives from authorities, leading IT companies, social organizations and the IT community (Communist Party online, 2012) E‐Government Promotion: In this indicator, Waseda e‐Government ranking considered many factors related to the promotion of e‐Government development, such as law or legislation, plans/strategies, policies, government or private entities involved, public and private partnerships, fund for e‐Government development, training e‐Government for citizens, e‐Government activities (conferences, advertisements) and think tanks for e‐Government in national and sun‐national levels. Regarding the factors are listed above, Vietnam lacked many importance elements, there were not e‐Government master plans, no public and private partnership, no fund for e‐Government development, especially there was not training course for citizens on e‐Government and no think tanks. This is shown clearly through the table 3.3. Every year, Vietnam received the same score (5.41) which show that within 5 years, there were no actions on e‐Government promotion. Vietnam also did not have a specialized agency which has responsibilities for e‐Government implementing and financial resources were not centralization but decentralization. This is a hug drawback of Vietnam. Unlike Vietnam, Japan has some government offices work together to promote e‐Government such as: The Government Promotion and Management Office (GPMO) of IT Policy Office of Cabinet Secretariat, the Administrative Management Bureau of the Ministry of Internal Affairs and Communications, and the Program Management Offices (PMO) (e‐Gov, 2010) E‐Participation: the newest indicator in Waseda e‐Government ranking, it was added in 2010 and includes e‐information, e‐consultation and e‐decision making process. The increasing use of web 2.0 technologies in e‐Government applications, otherwise known as Government 2.0 is on the increase. For this indicator, Japan has good position in the ranking, they always on the top of this indicator from 2010 to 2012. Although the new indicator and the application of new technologies but through the data obtained from the survey, Vietnam also has relatively good scores with this indicator. From 2010 to 2012, the score which Vietnam received tend to increase. It turns out the application of new technology in Vietnam are emphasized and applied to the implementation of e‐Government development. 3.2.3 E‐Government gap between Vietnam and Japan Through the analysis of e‐Government development and indicators detail, through all table 3.1; 3.3 and 3.4 which mentioned above, the gap between e‐Government situation in Vietnam and Japan is clearly and very big. The gap appears in all indicators and total ranking, especially for Network Preparedness that table 3.4 showed. In table 3.1 the figure showed that, in 2012 the total score for Vietnam was 52.10 and for Japan was 81.50, the difference was 29.4 score (56.4%). It is relative large distance. But in indicator (National Portal or e‐Participation) the gap is not so big, To gradually reduce this big gap, Vietnam have to do a lot of work and need to learn the experience from some developed countries, such as Singapore, USA or Japan, in which Japan is a typical case for e‐Government development model that Vietnam can refer.

4. Recommendations Based on the indicators in Waseda e‐Government Ranking, from the analysis and evaluation above, this research has shown the overall e‐Government development in Vietnam. The strength and weak points, this research also compared it with Japan. In order to find out the roadmaps and methods that Japan applied to develop e‐Government. However, the conditions of Vietnam and Japan is different, therefore Vietnam needs to focus on its strength and advantage, based on existing conditions to develop e‐Government successfully. Through the survey, this research draws some problems and treated it as suggestions to Vietnam for referring and implement e‐Government with highly effective. Vietnam needs to improve Telecommunication infrastructure by expanding international connections and setup high speed Local Area Network (LAN) and Wide Area Network (WAN). Promote the broadband development to households, combination between government and business to develop Internet use by offers free internet access to public and business without access to computers.

117

Nguyen Manh Hien Advanced training for citizens using computers, Internet, and have special training classes on e‐Government. For managers, need to be trained in IT and especially in CIO and also improving user experience. Vietnam needs to plan a strategy for development – Master plan, due to technology and trends change constantly, thus the master plan should be updated every three year. Master plan needs to have roadmap and clear purpose for each period. To develop e‐Government effective, from Japan experience and other countries, the creation a specialized agency that has responsibilities to make master plan, monitor and manage e‐Government projects is a very important issue. To better monitor and manage its e‐Government, the Vietnamese government should choose a centralized approach, for example in providing and approving the funding of all e‐Government projects. Besides, like Japan, promotion local government and centre government at the same time are the best solution for e‐Government development in Vietnam.

5. Conclusions Vietnam has recognized the importance e‐Government role in country development but until now, it can be said that, e‐Government services in Vietnam is not so good. From research point of view, one of the important causes that are due to Vietnam did not have a strategic vision, long‐term orientation in planning, in development and the policies instability as well as inconsistency. The main objective of this research was to provide an overview of the current status of e‐Government development in Vietnam by using Waseda e‐Government ranking reports as the main source of documents. Through a detailed analysis and based on lessons learned from experience in Japan, this report drives conclusion and hopefully to give some important points for more effective e‐Government development in Vietnam.

References 2012 United Nations E‐Government Survey: Towards a More Citizen‐Centric Approach, Report of the Expert Group Meeting, 2011 Bent Flyvbjerg, 2011, "Case Study," in Norman K. Denzin and Yvonna S. Lincoln, eds., The Sage Handbook of Qualitative Research, 4th Edition (Thousand Oaks, CA: Sage), pp. 301‐316. BuddComm Research Report 2012, Vietnam – Broadband Market and Forecasts, available online at http://www.budde.com.au/Research/Vietnam‐Broadband‐Market‐and‐Forecasts.html Chang Chih‐Hao, 2011, a study on User’s Traits toward Public e‐Service Quality and Satisfaction, PhD dissertation, pp. 11 Communist Party of Vietnam online newspaper, 2012, available online at: http://dangcongsan.vn/cpv/Modules/News_English/News_Detail_E.aspx?CN_ID=533707&CO_ID=30293 DeBenedictis, A., Howell, W. & Figueroa, R. 2002, E‐Government defined: an overview of the next big information technology challenge, International Association for Computer Information Systems. Evaluate on Waseda e‐Government International Ranking 2012 by Nguyen Manh Hien, available online at: http://cio‐japan.waseda.ac.jp/docs/2012spring_4_nguyen.pdf General Statistics Office of Vietnam (GSO), http://www.gso.gov.vn/default_en.aspx?tabid=491 Global E‐Governance Series, 2010, The Innovative CIO and e‐Participation in e‐Government Initiatives, Edited by Toshio Obi, Vol.3, pp. 49‐52. ICT to Vietnam, for Australian exporters, 2011, available online at http://www.austrade.gov.au/ICT‐to‐Vietnam/default.aspx International Telecommunication Union (ITU), http://www.itu.int/ITU‐D/ICTEYE/Indicators/Indicators.aspx# Japan country report, 2011 and 2012, Waseda e‐Government ranking, by Toshio Obi and Hien Nguyen Manh Japan’s e‐Government Initiatives, 2010, available online at: http://www.e‐gov.go.jp/doc/e‐Government.html John Scott, 2006, Documentary Research, Volume 1, SAGE Publications Mahmud Akhter Shareef, Vinod Kumar, Uma Kumar, Yogesh K. Dwivedi, 2010, e‐Government Adoption Model (GAM): Differing service maturity levels, Government Information Quarterly 28 (2011), pp. 17‐35 Marchionini, G., Samet, H. and Brand, L., 2003, “Digital Government”, Communications of the ACM, Vol.46, No.1, pp25‐27 Nguyen Thi Thanh Hai, 2010, Research on Critical Factors of an e‐Government Project – A case study of the State Administrative Management Computerization Project in Vietnam Vietnam country report, 2011 and 2012, Waseda e‐Government ranking, by Hien Nguyen Manh and Toshio Obi th Vietnam ICT Country Report, 2008 by Dr. Nguyen Anh Tuan, in the 4 Government CIO Conference 2008, Sep 29 – Oct 4, 2008, CICC, Japan Vietnam Internet Network Information Center (VNNIC) report, 2012, available online at http://www.vnnic.vn/en/stat/report‐internet‐statistics Vietnam Internet Network Information Center (VNNIC), http://www.vnnic.vn/en?lang=en Waseda e‐Government Ranking Press Released 2012, by Toshio Obi and Hien Nguyen, available online at: http://www.e‐gov.waseda.ac.jp/images/Press Released on e‐Gov ranking 2012.pdf

118

The CIO in the Public Sector: The Role and Requisite Knowledge and Competencies Val Hooper and Beverley Bunker School of Information Management, Victoria University of Wellington, Wellington, New Zealand val.hooper@vuw.ac.nz beverley@knowledgeweavers.co.nz

Abstract: A considerable body of research exists on the role, and desired capabilities and competencies of the CIO. However, most of these studies have been executed in large, private sector organizations. It seems that the challenges faced by public sector CIOs are often very different to those in the private sector, and this might place different requirements on them in terms of knowledge and competence, as well as the roles they fulfil. To date, there has been little exploration into such requirements in public entities. To address this gap, exploratory research was conducted into the role and competency expectations of CIOs in the public sector, and into the impact of the public sector context. A dyadic approach, involving both CIOs and their business colleagues, was adopted in order to gain more meaningful insights. Semistructured interviews were conducted with both the CIO and the head of their main internal “business” partner of 17 local government organizations. The findings indicate that the CIOs and their business partners differ significantly in their views of required competencies. The business partners require a business knowledge and focus similar to theirs, and most manifest scant regard for the technical expertise necessary or the technical requirements of the organization. IT is there to support them. The government environment often places more onerous constraints on CIOs than in the private sector, especially so in terms of reporting level; the ability to influence strategy; decision making flexibility; and resourcing. The findings from this research extend the application of the RBV and also provide greater understanding of the competencies and roles of the CIO. It also provides insights for recruiters of public service IT professionals and CIOs, human resources managers, as well as for providers of training programmes. Keywords: CIO, competency, knowledge, skills, roles, public sector

1. Introduction The role of the CIO has been the focus of research for a number of years. Basselier at al. (2004) were among those who explored the requisite capabilities and competencies of the CIO. They identified both the types of knowledge and experience that was desired. However, their research, like many others on the role and competencies of the CIO, was conducted in large private sector organizations. The public sector in most countries is a sizeable employer, providing a large number of diverse business activities and a wide range of services to their community. Many factors such a legislation, politics, and resourcing, place very different demands on public sector CIOs to those in the private sector. There might thus be differences in terms of the roles they fulfil and the requisite knowledge and competence. To date, there has been little exploration into such requirements. To address this gap, exploratory research was conducted into the role and competency expectations of CIOs in the New Zealand government context, and into the impact of that context. A dyadic approach, involving both CIOs and their business colleagues, was adopted in order to gain more meaningful insights. The following sections report on the underpinning literature, the data collection, the findings, discussion and conclusions of the research.

2. Literature review The competencies required for the CIO to realise their position as a strategic partner with the business has been of particular interest to researchers (Basselier & Benbasat, 2004; Broadbent & Kitzis, 2005; Lane & Koronos, 2007; Wu et al., 2008). In general this research has been based on the CIO position in large private sector (and often US-based) firms. Limited research has been undertaken in small organizsations where the CIO often has to continue to manage the operational aspects of the technology as well as providing strategic advice to their business partners (Duhan, 2007). There has also been limited research that is specifically focused on the role of CIOs in government sector. The role of the CIO has undergone considerable change – from data processing manager in the 1970’s (Lane & Koronos (2007) to senior executive responsible for aligning ICT with business goals (Broadbent & Kitzis, 2005). To a large extent, the role of the CIO is a reflection of the role that IS is expected to fulfil. As IT/IS functions can

119

Val Hooper and Beverley Bunker be divided into two key areas: those concerned with maintaining a stable and robust service (i.e. “keeping the lights on”) and those concerned with delivering business changes (i.e. “adding value”) (Gottschalk, 2000), the CIO role thus moves between technologist, strategist and leader (Remenyi et al., 2005). The resource-based view (RBV) of an organization provides a basis for identifying and describing competencies required to support organizational roles ( (Bharadwaj, 2000); Garavan & McGuire, 2001). The RBV maintains that organizations achieve and sustain competitive advantage through the acquisition and deployment of resources (Peppard & Ward, 2004). Capabilities refer to a firm’s ability to assemble, integrate and deploy resources, usually in combination (Bharadwaj, 2000) thereby transforming inputs into outputs of greater worth. Capabilities subsume competencies (Bharadwaj, 2000). Using the RBV, Feeny & Willcocks (1998) identified a set of nine core IS capabilities to support the organization’s ability to successfully exploit IT (Feeny & Willcocks, 1998): leadership, relationship building, contract facilitation, informed buying, making technology work, architecture planning, business systems thinking, vendor development, and contract monitoring. Leadership and informed buying are regarded as “lynch-pin” capabilities, (Feeny et al., 2006). Each of these capabilities requires business, technical and interpersonal skills to a greater or lesser extent. Broadbent & Kitzis’s (2005) identified five critical roles with matching competencies through which the CIO leadership is achieved: providing an IS vision; IT governance; integration of the IT and business strategies; IS/IT staff management; and vendor management. Gottschalk (2000) identified nine leadership roles of CIOs: three managerial and six consultancy roles but he did not define the relevant competencies. Peppard & Ward (2004) identified three macro competencies (business knowledge, skills and experience, technical knowledge, skills and experience, and behaviour and attitude) which comprise an overall IS capability for the organization. Basselier and Benbasat (2004) identified the CIO’s organization-specific competence and interpersonal management as contributing to business competence, and Preston et al (2008) defined four IT profiles which they matched to IT contribution to the organization. In summary, models such as Feeny & Willcocks (1998) IS Capabilities and Peppard & Ward (2004) show the link between the organizational IS capability and the individual skills and knowledge of the CIO, while others, such as Basselier & Benbasat (2004) and Wu et al., (2008) focus on defining the components that comprise business and technology competency. This has allowed a set of components for each broad competency area to be identified (see Table 1). Table 1: Business and technical competency expected of CIOs: indications from the literature Business domain knowledge Organizational Overview

Basselier & Benbasat, 2004

External environment Organizational units

(Broadbent & Kitzis, 2005, p. 38) Basselier & Benbasat, 2004

Organizational responsibility

Basselier & Benbasat, 2004

IT/Business integration

Basselier & Benbasat, 2004 (Broadbent & Kitzis, 2005)

High-level business knowledge

(Broadbent & Kitzis, 2005)

Knowledge networking

Basselier & Benbasat, 2004

Business competence Management and leadership competency Strategic (Emiliani, 2003) perspective (Broadbent & Kitzis, 2005) Leadership Kotter (1990) (Broadbent & Kitzis, 2005) Human (Lane & resource Koronos, 2007) management Budgetary & (Lane & financial Koronos, 2007) management Business (Broadbent & planning Kitzis, 2005) Decision (Emiliani, 2003) making Change Management

(Broadbent & Kitzis, 2005) (Wu et al., 2008)

120

Behavioural skills and attributes Interpersonal communication

Basselier & Benbasat, 2004

Relationship building

(Broadbent & Kitzis, 2005)

People skills

(Kotter, 1990)

Problem solving and creative thinking

(Broadbent & Kitzis, 2005)

Team work

(Broadbent & Kitzis, 2005) (Willcoxson & Chatham, 2006) (Broadbent & Kitzis, 2005)

Learning Openness/adaptability

Results focus

(Broadbent & Kitzis, 2005)

Val Hooper and Beverley Bunker IT industry knowledge Broad industry (Broadbent & concepts Kitzis, 2005) Technologies (Broadbent & Kitzis, 2005) (Basselier et al., 2001) IT solutions Basselier & Benbasat, 2004

Architectures

Access to IT knowledge

(Broadbent & Kitzis, 2005) (Lane & Koronos, 2007) Basselier & Benbasat, 2004

Technical competence IT management expertise IS strategic (Broadbent & planning Kitzis, 2005) IS processes (Lane & and procedures Koronos, 2007) IS Governance and regulatory compliance Vendor and Supplier Management IT contract development and negotiation IS Service delivery management

(Lane & Koronos, 2007) (Feeny & Willcocks, 1998) (Feeny & Willcocks, 1998)

IT technical expertise IS Risk and Security (Lane & management Koronos, 2007) IS project (Lane & management Koronos, 2007) IS asset management

Basselier & Benbasat, 2004 (Peppard & Ward, 2004)

Application development and support

Basselier & Benbasat, 2004

(Peppard & Ward, 2004)

Only a few researchers, such as Lawry (2007) and Curran (2006) have explored the role of the CIO in the public sector, where indications are that although the CIO is expected to play a significant strategic role, the range of technology management is far broader then in the private sector and the shift will be more towards government processes, people and business efficiencies. However, much is still unknown. In addition, drawing on the alignment literature (Chan & Reich, 2007), there might be differing perceptions between the CIOs and their business partners of the requirements of the CIO. In order to address that gap, this research’s objectives, within the public sector context, were to: 

Identify and compare CIOs' and their business partners' perceptions of the requisite key roles performed by CIOs and individual knowledge, skills and competences required to support them



Ascertain the extent to which CIOs are able to influence the strategic direction of the organization



Determine how the public sector context impacts on these roles

3. Data collection Because the research was exploratory, a qualitative research method was chosen (Klein & Myers, 1999). Semistructured interviews were conducted with both the CIO and the head of their main internal “business” partner of 17 New Zealand public sector organizations. Local governments, of which there are 73, were selected because they represent the largest body of public sector organizations in New Zealand. The participating organizations were chosen based on convenience, especially geographic proximity to the capital where the researchers were based. The responses was analysed, coded and categorized according to themes and sub-themes (Cresswell, 2003). The findings are reported in the following sections.

4. Findings As part of its responsibilities under the Local Government Act (2002), each local government produces a longterm council community plan (LTCCP) that sets out activities planned for the next 10 years. Many councils do not have a separate organization-level strategy and the LTCCP forms the basis for their annual plans and budgets. The respondents represented a range of different sized local governments, from small (<250) to large (>1250). The number of IS staff varied more or less proportionately from <10 to nearly 90. The local governments carry a wide area of responsibilities – up to 50 maximum. The range of functions undertaken by the IS/IT units include business change functions (project management; business analysis; business consultancy) and

121

Val Hooper and Beverley Bunker operational functions (infrastructure support; telephony; applications support; GIS; web; records management). These functions are distributed in a number of different ways, and in some organizations the IT/IS function was split under two managers. The environment is highly regulated, with activities being defined by central government through legislation and regulations. Budgets are limited and consequently, so are resources. The local councils’ customers are also their shareholders and expenditure, indeed all council activity, is open to public scrutiny. The local governments are thus very risk averse, despite the presence of political agendas. However, the environment is not competitive in the same way as the private sector. The areas in which the interviewees felt it was important for local government CIOs to have knowledge, skills and experience, fell into three categories: IT competencies, business competence; and organizational knowledge. They also identified a number of areas where CIOs can provide added value to the organization, and how they can best deliver on their services and advice.

4.1 IT competence In terms of IT competence, all interviewees thought that IT experience was essential but while the CIOs thought that hands-on experience was most useful, the business partners thought it was definitely not a requisite. Reasons offered in support of the need for technical experience included: 

The ability to engage with technical staff and vendors



Ability to see through vendor hype



Understanding the technical issues and stresses of the support staff



Understanding how the technology “hangs together”.

The requisite technical knowledge and skills mentioned are depicted in the table below. Table 2: Requisite technical knowledge and skills IT industry Knowledge area Broad range of industry knowledge Currency/future direction Strategic focus Understanding of architecture

IT Management Skills

Technical Skills

Change Management Consultancy (advice/guidance)

Web/Internet Network architectures & connectivity Project Management (experience/knowledge) Records Management (principles) Business analysis Applications & support

Service delivery (focus) Operations management Strategic Planning (IT)

Both groups of interviewees stressed the importance of a broad range of IT industry knowledge. This reflects the variety of activities the IS function is expected to perform. It also reflects the need to be able to embrace more recent developments in the industry. While the CIOs emphasized the ability of CIOs to have skills in both strategic planning and in operations management so as to understand the whole range of the function’s activities, the business partners’ main focus was on consultancy and the provision of advice and guidance. Technical skills, and the variety of them, were understandably much more frequently mentioned by the CIOs than their business partners. In particular, the CIOs emphasized project management experience and knowledge, but also noted business analysis skills.

4.2 Business competence In terms of business competence, while most CIOs felt that local government experience was useful, the business partners were of the opinion that it was not important. However, both groups thought that it was important to learn about the organizational culture and environment. The requisite business knowledge and skills noted are presented in the table below.

122

Val Hooper and Beverley Bunker Table 3: Requisite business knowledge and skills Broad business knowledge Business perspective or acumen

General Management Skills

Customer focus Corporate view & understanding of priorities

Staff/people management

Managerial skills & experience

Effective Delegation Financial Management & Budgetary skills Decision making skills Vendor Management Strategic & business planning Resource Management

Leadership Skills Leadership skills Business case in relation to IT(cost-benefit) Ability to negotiate (between contending bus. Interests) Strategic perspective - big picture focus Future vision

Interpersonal Skills People skills (Emotional Intelligence) General communication skills Ability to communicate in a non-technical language Ability to communicate with technical staff/vendors Relationship building skills Team Work (with peers)

Regarding broad business knowledge, both the CIOs and particularly their business partners felt that it was important for the CIO to possess a business perspective or acumen. The business partners also felt it was important for CIOs to have a corporate view and understanding of business priorities, although this did not receive any mention by the CIOs themselves. As far as general management skills, both groups of interviewees stressed the importance of the CIO possessing managerial skills and expertise, and especially financial management and budgetary skills. Of the necessary leadership skills, once again the CIOs and their business partners had similar views: that it was important for CIOs to have a future vision. The business partners also emphasized the ability of CIOs to negotiate between contending business interests. In terms of people skills, the business partners focused on people skills and general communication skills, while the CIOs had a more specific focus, and emphasized the ability of CIOs to communicate in non-technical language, and the ability to communicate with technical staff and vendors. Some interviewees mentioned the personal attributes that would facilitate interpersonal skills. These included patience, tenacity, having a practical approach, being politically savvy, being result driven, and having the ability to prioritize.

4.3 Organizational competence Despite not being involved in the strategic planning, the CIOs, and to a slightly lesser extent the business partners, felt it was extremely important for them to have a good understanding of the organizational goals and objectives. Although the business partners did not think it was necessary for the CIOs to be involved in setting those goals, they and the CIOs felt that the CIOs could provide meaningful input in the form of advice on technological innovations and solutions, and on how technology could enable the organizational strategies that would achieve the goals. Both groups of interviewees felt that it was important for CIOs to understand the legal framework, but more at a higher level of understanding. It was, however, important for the CIOs to know where to get the information when needed, and this was usually from business managers and experts or a legal advisor. Regardless of size their council, the CIOs had to provide ICT services (and support applications) across the breadth of the business activities. Both groups of interviewees felt it was very important for CIOs to have a good understanding of the business goals and drivers of the different business units. In addition, the CIOs emphasized the importance of using the business language of the relevant unit in their communication with them. Although the CIOs felt that their involvement in the business unit planning was not sufficient, the business partners felt that it was sufficient for CIOs to be informed of potential IT projects from the business plans.

4.4 Roles of the CIO The interviewees noted seven distinct, integration roles for the CIO. These fell into two categories: the strategic value-adding aspects of the CIOâ&#x20AC;&#x2122;s task; and the more operational IT management and delivery.

123

Val Hooper and Beverley Bunker 4.4.1 Value add Three areas of value-added roles were identified: communication of innovative uses of technology; communication of the impact of IT legislation and standards; and the CIO having a vision for IT. Both interviewee group felt it was important for the CIO to promote the innovative uses of technology, as well as new developments which might assist the organization; and that the best way to do this was for the CIO to communicate with the relevant business unit manager, in a variety of ways, and for the CIO to attend seminars with the business managers. In terms of relevant IT legislation and standards, the business managers were more emphatic that this be discussed with the relevant business managers, rather than in more general forums. Both CIOs and their business partners thought it was particularly important for the CIO to have a clear vision for IS and to communicate that vision, which needs to fit in with the business vision. 4.4.2 IT management and delivery With regard to the management of the IT organization and the delivery of their services, four aspects emerged: prioritization of IT projects and expenditure; managing and reporting IT projects; communication with key stakeholders; and management of and communication with vendors. In terms of prioritization, particularly the CIOs and their business partners of the larger organizations felt it was important to have an IS governance group in place to make the key decisions. The CIOs also felt that they should prioritize infrastructure projects, and that this should be done via formal procedures. CIOs also felt particularly strongly that formal steering groups should be used for major projects and that key stakeholders should be involved in these. In terms of communicating IS/IT matters, the CIOs felt that this was best done through formal reports to senior management but by informal meetings with their business peers. The business partners agreed on the latter channel. Both interviewee groups felt that informal discussions with vendors which sometimes included the business partners was the best way to go.

5. Discussion There has been considerable research into the competencies required by CIOs, and to a large extent, those noted in the interviews reflect the literature. While this research does not claim to have identified an exhaustive list of competencies required by the CIO in the government sector, what is particularly important is the respective focus of the CIOs and their business partners, the extent to which the competencies relate to the different categories of roles identified, and the impact of the public sector context. Regarding technical competencies, on the whole, high-level broad industry skills were the most often mentioned as important for the CIO role. It was also important that this knowledge was current and businessfocused. The CIOs themselves raised both IT management skills and technical skills more often than their business partners. The business partners more commonly mentioned those skills that would have a direct impact on them for example consultancy/advice, project management, business analysis and application support. In terms of business competencies, those specific competencies identified support the (Basselier & Benbasat, 2004) model of business competence which comprises business and commercial knowledge, management and leadership as well as personal attributes. While local government experience was considered more important by the CIOs, a broad business knowledge or perspective was considered extremely important by both CIOs and their partners. A strategic and visionary perspective, and communication and people skills were valued, although the CIOs focussed more on the specific IT details than their business partners. They were very conscious, too, of the need to communicate in language that the business partners could understand. The interviewees agreed on the importance of a CIO having a broad high-level organizational knowledge, in particular with regard to the organizational goals and objectives, organizational culture and environment, legal framework, and business unit goals and drivers. They suggested that this is gained through discussion with business managers favouring informal discussion over more formal arrangements. A key theme that emerged is the importance of the CIOâ&#x20AC;&#x2122;s role as an advisor to the business, both during organizational strategic planning and at the business planning level. The roles of the CIO were divided into two areas:

124

Val Hooper and Beverley Bunker 

A set of strategic roles performed by the CIO to identify where IT can add value to the council



A set of operationally focused roles the CIO performs to ensure IT delivers the benefits within the budget constraints.

Regarding the value-add aspect of the IT-business relationship, there was a clear indication that the business looked to the CIO for advice and guidance on new technologies and IT-related legislation. Informal two-way communication was preferred. The majority of the interviewees felt that it was important for the CIO to have a vision for IT and that this be well communicated and aligned with the business vision. Within the IT management and delivery area, informal structures were favoured when dealing with business peers, while more formal arrangements suited to communications with senior management. Large councils tended to have governance groups and formal processes around IT investment decisions, while smaller used existing reporting structures. Combining competence with roles, the relative emphasis on the strategic versus the operational is depicted below. 25 20 15

Strategic Roles

Operational Roles

ro ad

In du an stry Te age ch m e ni ca n t lS Le kills ad er M an ship O ag rg em an I nt is en er at t pe B us io n rs al in on Kn es al s o w un l e it kn d g ow e le dg e

Figure 1: Areas of competence relevant to roles The two categories of roles require different sets of competencies: 

Broad industry knowledge, leadership skills and high-level organizational knowledge were identified as more important for the strategic value-add roles



IT management skills, technical expertise, general management skills and business unit knowledge were identified as more important for the operational value-for-money roles



Interpersonal skills and personal attributes are important for both roles although different attributes may be required. In particular, both technical and non-technical communication were valued in both categories of roles.

As with the CIO’s competencies, the findings largely reflect the competencies relevant for the various roles as identified in models such as those of Feeny and Wilcocks (1998), Broadbent and Kitzis (2005) and Lane and Koronos (2007). However, the local government context places certain demands and constraints on the role of the CIO. Some of the key points relating to local government are: 

Local government CIOs are not usually included in senior management, and consequently are not involved in setting strategic direction which is based on the LTCCP. Rather, they are involved at a later stage advising how technology can enable the business outcomes. This can be constraining.



Vendor relationships are usually operational in nature rather than strategic as models like Feeny & Willcocks (1998) IS capabilities would suggest. CIOs are more focused on managing vendor issues than seeking new opportunities with vendors.

125

Val Hooper and Beverley Bunker 

There are two key (lynch-pin) roles that are considered the most important by all the participants: to promote the IS plan; and deliver the IS plan.



The management of a particularly wide range of stakeholders, or client groups, places very large demands on the CIOs, not only on the need for excellent communication skills, political skills, and the ability to prioritise between often conflicting demands, but also requisite knowledge of the frameworks within which the groups operate. Additionally, when resources are stretched, the CIO is often expected to adopt a “hands on” role, and demonstrate excellent technical skills.



Local government CIOs communicate formally with senior management and with peers in larger organizations, but informally with peers in other organizations.

6. Conclusion The research aimed to explore the role and competency expectations of CIOs in the public sector context, and to determine the impact of that context. A dyadic approach, involving both CIOs in New Zealand local government and their business colleagues, was adopted in order to gain more meaningful insights. Three categories of competence were noted: IT competence, business competence, and organizational competence. Two categories of roles were identified: strategic, value-add roles, and operational IT management and delivery roles. The respective competencies were matched against the roles. In general, the competencies and roles identified reflect those noted in the literature. However, the CIOs and their business partners differ significantly in their views of required knowledge, skills and competencies. The business partners require a business knowledge and focus similar to theirs, and most manifest scant regard or concern for the technical expertise necessary or the technical requirements of the organization. IT is there to support, advise and guide them. CIOs, on the other hand, are aware of the necessity of possessing not only the broad business and organizational knowledge, but also the more detailed and wide IT knowledge required to cover all the many client groups and activities of the local government. The local government context places decided constraints on the CIO and consequently the on the range of competencies and roles required of the position. The findings from this research extend the application of the RBV and provide greater understanding of the competencies and roles of the CIO. It also provides insights for recruiters of public service IT professionals and CIOs, human resources managers, and providers of training programmes.

References Andreu, R. and Ciborra, C. (1996) "Organisational learning and core capabilities development: the role of IT", Journal of Strategic Information Systems, No. 5, pp 111-127. Basselier, G. and Benbasat, I. (2004) "Business Competence of Information Technology Professionals: Conceptual Development and Influence on Business-IT Partnerships", MIS Quarterly, Vol. 28, No. 4, pp 673-694. Bharadwaj, A.S. (2000) "A resource-based perspective on information technology capability: An empirical investigation:, Management Information Systems Quarterly, Vol. 24, No. 1, pp 169-196. Broadbent, M. and Kitzis, E. (2005) The new CIO Leader (1 ed.), Havard Business School Press Boston, Massachusetts. Chan, Y. E. and Reich, B. H. (2007) "IT alignment: what have we learned?", Journal of Information Technology, No. 22, pp 297-315. nd Cresswell, J.W. (2003) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (2 ed.). Sage Thousand Oaks, CA. Duhan, S. (2007) "A capabilities based toolkit for strategic information systems planning in SMEs", International Journal of Information Management, No. 27, pp 352-367. Emiliani, M. L. (2003) "Linking leaders' beliefs to their behaviors and competencies", Management Decision, Vol. 41, No. 9, pp 893-910. Feeny, D. and Willcocks, L. P. (1998) "Core IS Capabilities for exploiting Information Technology", Sloan Management Review, Vol. 39, No. 3, pp 9-21. Feeny, D., Willcocks, L. P. and Olson, N. (2006) "Implementing Core IS Capabilities: Feeny-Willcocks IT Governance and Management Framework Revisited", European Management Journal, Vol. 24, No. 1, pp 28-37. Garavan, T., N. and McGuire, D. (2001) "Competencies and workplace learning: some reflections on the rhetoric and the reality", Journal of Workplace Learning, Vol. 14, No. 4, pp 144-163. Gottschalk, P. (2000) "Information Systems Executives: The changing role of new IS/IT leaders", Informing Science, Vol. 3, No. 2, pp 31-39.

126

Val Hooper and Beverley Bunker Klein, H. K. and Myers, M. D. (1999) "A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems", MIS Quarterly, Vol. 23, No. 1, pp 67- 94. Kotter, J. P. (1990) "What Leaders Really Do", Harvard Business Review, pp 103-111. Lane, M. S. and Koronos, A. (2007) "Critical Competencies required for the Role of the Modern CIO", Paper presented at the 18th Australasian Conference on Information Systems, Toowoomba, Australia, 5-7 December. Luftman, J. and Brier, T. (1999) "Achieving and Sustaining Business-IT Alignment", California Management Review, Vol. 42, No. 1, pp 109-121. Mingers, J. (2001) "Combining IS research methods: Towards a pluralist methodology", Information Systems Research, Vol. 12, No. 3, pp 240-259. Myers, M. D. and Newman, M. (2007) "The qualitative interview in IS research: Examining the craft", Information and Organisation, No. 17, pp 2-26. Orlikowski, W. J. and Baroudi, J. J. (1991) "Studying Information Technology in Organizations: Research Approaches and Assumptions", Information Systems Research, No. 2, pp 1-28. Peppard, J. and Ward, J. (2004) "Beyond strategic information systems: towards an IS capability", Journal of Strategic Information Systems, No. 13, pp 167-194. Preston, D. S., Leidner, D. E. and Chen, D. (2008) "CIO Leadership Profiles: Implications of Matching CIO Authority and Leadership Capability on IT Impact", MIS Quarterly Executive, Vol. 7, No. 2, pp 57-69. Remenyi, D., Grant, K.A. and Pather, S. (2005) "The chameleon: a metaphor for the Chief Information Officer", Journal of Information Management, Vol. 30, No. 3, pp 1-11. Willcocks, L. P. and Feeny, D. (2006) "IT Outsourcing and Core IS Capabilities: Challenges and Lessons at Dupont", Information Systems Management;, Vol. 23, No. 1, pp 49-56. Willcoxson, L. and Chatham, R. (2006) "Testing the accuracy of the IT stereotype: Profiling IT managersâ&#x20AC;&#x2122; personality and behavioural characteristics", Information & Management, Vol. 43, No. 6, pp 697-705. Wu, J.-H., Chen, Y. C. and Sambamurthy, V. (2008) "The Impacts of BTM Capability and CIO Role Effectiveness on Firms' Information Technology Assimilation: An Empirical Study", Paper presented at the Twenty Ninth International Conference on Information Systems, Paris, December.

127

Trusting Your Bank in a Digitally Connected World – a New Zealand Investigation Into Customers Perceptions of Ethical Business Practice Tony Hooper and Shivonne Londt Victoria University of Wellington, Wellington, New Zealand Tony.Hooper@vuw.ac.nz Shivonne@gmail.com Abstract: As the use of online social networks (OSNs) grows, people place more of their personal information online, and others find different ways of exploiting it. While there is a lack of awareness around the privacy implications of these actions, there remains an expectation of privacy with regards to OSN data that may not entirely be justified. In a heightened competitive environment, banks are using data from OSNs for lending decisions. They expect to reduce their risk significantly on lending decisions through accessing the personal information on clients OSN accounts. Customers in New Zealand may perceive such use of OSN data by New Zealand banks to be a privacy violation, as their OSN data would be used for purposes other than those for which it was provided. As a result customer demand for ethical and compassionate use of their data is growing. Customers expect greater corporate social responsibility, especially from banks after the recent global financial crisis. This research explores whether customers would perceive the use of their OSN data to be a violation of their privacy. It is intended to be a way of evaluating whether banks’ use of data found about customers on public OSNs to improve their lending decisions would be perceived by customers as a violation of their privacy. Through a critical review of the literature on social exchange theory and communication privacy management theory, followed by a discussion of the research methodology used, this research employed an anonymous online survey and a series of semi‐ structured interviews to develop conclusions about whether a perceived privacy violation could occur from the use of customers’ data on OSNs. The research found that respondents who were less informed about OSN privacy policies, had a greater expectation of privacy, even though they had not actually read the privacy policies of the OSN sites. This may contribute to an over‐expectation of privacy on OSNs and a high likelihood that a perceived privacy violation would take place should the New Zealand banks decide to emulate banks in the USA and use OSN data to influence lending decisions. This study has thus highlighted the need for greater care and transparency, as well as careful management of customers’ expectations and trust if New Zealand banks wish to pursue this method of influencing lending decisions. The study revealed a high likelihood that should New Zealand banks decide to follow banks in the USA and use OSN data to influence lending decisions, their customers would perceive such behaviour as unethical and an invasion of privacy. In terms of privacy on OSNs, the research highlighted a greater need for users to read and understand the implications of both the privacy policies and the privacy settings of their accounts in order for them to ensure that the information that they wish to keep private does, indeed, remain so. Keywords: banks, privacy invasions, social networks

1. Introduction As a direct consequence of the growing use of online networking services, people are placing more personal information about themselves online, often without regards for the consequences of these actions or the use to which this information may be put (Chen, Ping, Xu, & Tan, 2009). At the same time, the banking environment is becoming more competitive (Boot & Marinĉ, 2008) and banks in the United States of America (USA) have been using personal data found on OSNs to influence their lending decisions to gain a competitive advantage (Baird & Gonzalez‐Wertz, 2011; Finney, 2010). Although banks already have an intensely personal picture of an individual because of the data provided when that individual enters into a contract with the bank, there may still be certain areas of that individual’s behaviour relevant to lending decisions that are not transparent. This would not be the first occurrence where data from OSNs has been used for purposes other than that for which it was posted. Employers are increasingly using OSNs to perform “informal online background checks” on potential employees and make decisions on their eligibility for the jobs offered based on what is revealed by these background checks (Clark & Roberts, 2010). Clark & Roberts (2010) indicate that employers often assume that using OSNs for character checks of job applicants is an acceptable process, especially when considered in the light of “protecting themselves and shareholders”. OSNs allow these background checks to take place and can reveal the nature of decisions

128

Tony Hooper and Shivonne Londt applicants will make in their personal lives and in their jobs. Similarly, banks may feel that it is acceptable to use OSNs to check potential customers to determine the kinds of financial decisions these customers may make with the intention of protecting themselves and their existing customers (Brandenburg, 2008). There is, however, a growing consumer demand for an ethical and compassionate attitude from companies towards their customers. Using customer data found on OSNs may be seen by customers to be a violation of customers’ privacy even though banks are permitted by the New Zealand Privacy Act to use any publicly available information as a basis for their lending decisions (Privacy Commissioner, n.d.). It is thus necessary for banks to consider this factor before using OSN data in this manner, even though it may grant them a much‐ needed competitive advantage, because of the potential implications for their reputation and the trust of their customers. Nissenbaum (1998) has, however, acknowledged that upholding one party’s privacy imposes disadvantages and potential restraints on others. Upholding individuals’ privacy with respect to the data they have placed on OSNs may prevent banks from taking advantage of a potential source for competitive advantage. This research evaluates whether banks’ use of data found on public OSNs to improve their lending decisions would be perceived by customers as a violation of their privacy. It will be presented as an initial critical review of the relevant literature, followed by a discussion of the research methodology used and an explanation and discussion of the findings. It will discuss whether a perceived privacy violation could occur from the outlined use of customers’ data on OSNs.

2. Literature review Reynolds (2010) indicates that the finance industry has recently been negatively impacted through “unwise and unethical decision making” around mortgage approval and credit extensions. Customer confidence in the entire finance industry in New Zealand, which includes the banking industry, has dropped to the lowest level since the Financial Confidence Index was launched in 2009 (RaboDirect., 2011). Morgan & Hunt (1994) indicate that confidence forms a basis for trust so this loss of confidence can be extrapolated to include a loss of trust by consumers in banks themselves. It is not the intention of this review to discuss at length the literature on trust in electronic commerce. The field has been thoroughly explored. What is relevant is the notion that trust is comprised of competence, benevolence, integrity and predictability and has been defined as a belief that another party has characteristics that may be “beneficial to oneself” (Dimitriadis, Kouremenos, & Kyrezis, 2011). The major New Zealand banks have emphasised their intentions to act ethically and in the best interests of their customers on their publicly accessible websites (e.g. (ASB Bank Limited., 2011; KiwiBank., 2011a; The National Bank of New Zealand., 2011a; Westpac New Zealand Limited., 2008). One can argue that the greater the trust a customer has in their bank, the less they will expect their bank to make use of the customer’s personal data on OSNs. Social exchange theory (SET) indicates that, for relationships to evolve into mutually trusting and loyal relationships, certain “rules” need to be followed (Cropanzano & Mitchell, 2005). The costs of the relationship are weighed against the benefits in order to determine the relationship’s worth. This outcome is further evaluated against the individual’s expectations in order to determine the satisfaction level with the relationship (Luo, 2002). SET also indicates that trust is a key component in building relationships (Luo, 2002). SET thus suggests that a banking customer will weigh up the potential costs (privacy invasion) and benefits (current trust in the bank) and will consider the potential for alternative relationships, (other banks), if faced with the possibility of their bank using OSN data to influence lending decisions. An individual with a strong dependence or attachment to their bank may thus find it difficult to consider moving banks even though they consider the use of their OSN data to be a betrayal of their trust and a violation of privacy. Similarly, an individual may feel that these costs do not actually outweigh the benefits of remaining in a relationship with their bank and would thus not consider leaving, despite the feelings of a betrayal of trust or privacy invasion. Of course, the individual may not experience these costs at all as the determination of costs and benefits in a relationship is highly subjective. Communication privacy management theory (CPMT) deals with the way in which individuals control the access to their private information (Petronio, 2007). It introduces the concept of “boundaries” to separate public

129

Tony Hooper and Shivonne Londt information from information that should be kept private (Bateman, Pike, & Butler, 2010; Child & Petronio, 2010). The “permeability” of these boundaries, or the degree to which information can flow through them, changes depending on the type of relationship and information being communicated (Child & Petronio, 2010). CPMT indicates that there are both risks (increased vulnerability) and benefits (a potential for a deeper relationship through greater intimacy) in revealing private information. If private information is shared, the party with whom it is shared becomes a co‐owner of the information and has certain rights and obligations in terms of it, including how it can be used and to whom it can be revealed (Bateman, et al., 2010; Petronio, 2007). Placing information on OSNs can, in CPMT terms, be considered to be a form of communication (Pike, Bateman, & Butler, 2009). Accordingly, banking customers may feel that a violation of the information boundaries would impact their relationship with the party who violated the boundaries. As SET indicates, this may result in an unacceptable cost in terms of the relationship and may cause the customer to start considering alternative relationships with other banks. Due to the nature of their business and the information that they hold about individuals, banks hold a position of power in relation to their customers. Because of this position of power and the trust that customers hold in their banks, customers could feel betrayed by the actions of their primary bank. Caldwell, Davis, & Devine (2009) have indicated that a feeling of betrayal results when an expected behaviour associated with a person’s trust is not followed. They outline five characteristics of betrayal in a corporate sense, namely:

betrayal is voluntary in that a customer chooses to feel betrayed in particular circumstances;

betrayal is a violation of pivotal expectations in the nature of the relationship between the customer and their bank;

both the bank and their customers are aware of these expectations. Banks are largely aware of these as evidenced by the privacy statements of their publicly accessible websites (e.g. ASB Bank Limited., 2009; KiwiBank., 2011b; The National Bank of New Zealand., 2011b; TSB Bank Limited., 2009). Expectations may not always be reasonable, however, as banks are permitted to make use of publicly available data;

the expectations of the customer are violated through the behaviour of the bank; and

the customer perceives that the betrayal has the potential to harm their wellbeing. (Caldwell, et al. (2009)

So, while betrayal by the banks may be either intentional or unintentional, it is entirely likely that the banks have not considered that their actions may result in feelings of betrayal by their customers. Customers will expect the data that they place on OSNs not to be used without their express consent. Customers could feel betrayed by banks’ use of their data resulting in a perceived violation of privacy.

3. Research questions Arising from the literature review the following research questions emerged:‐

Will the trust that bank customers feel towards their bank be affected if the bank uses OSN data when making business decisions?

If the customer feels that their bank betrayed their trust by using OSN data will that translate into a perception of privacy violation?

4. Philosophical approach A post‐positivistic and relativistic approach was taken by analysing of the outcomes of a survey and semi‐ structured interviews to infer generalisations around the research questions (Onwuegbuzie, Johnson, & Collins, 2009). Post‐positivism considers that there is an independent, individual reality that can be studied but that, due to cultural backgrounds, researchers will always have a certain amount of bias in their views of this reality so that they cannot be completely impartial in their observations or interpretations (Onwuegbuzie, et al., 2009). Relativism indicates that the research is influenced by the culture in which the research takes place (Bloland, 1989), which will affect the expectations of privacy and basis for trust in individuals’ primary banks. Constructivism, closely related to relativism, views knowledge as being constructed through experience and the relationship with the world through prior knowledge and other people’s experiences (Ryder, 2008; Talja,

130

Tony Hooper and Shivonne Londt Tuominen, & Savolainen, 2005). This highlights a potential bias in that the interpretation of the survey and interview data will be influenced by personal expectations of privacy and experiences with social networks.

5. Methodology 5.1 Web based survey The survey consisted of a total of twenty questions constructed from the literature review, covering privacy expectations, trust and demographic issues. Customers of the major banks in New Zealand (ANZ, The National Bank, Westpac, ASB, BNZ, Kiwibank and TSB) who are also currently participating in one or more OSNs were invited to take part in the survey and a subsequent interview. The survey aimed to get at least ten responses from customers of each bank, thus looking to achieve at least seventy responses in total. Invitations to participate in the survey were sent out to key contacts and colleagues in the banking industry, as well as postgraduate students at Victoria University. The survey questions were measured against a five point Likert scale, with potential responses ranging from Strongly Disagree to Strongly Agree. No personally identifiable information was collected from the survey. One hundred and twenty four completed responses from the survey were received. Two were discarded due to invalid data.

5.2 Semi‐structured interviews As a follow‐up to the survey, interviews were conducted with selected respondents who had agreed to be interviewed. One customer from each of the major banks, (aside from TSB which was not represented in the survey responses), was selected as an interviewee. The interviews were semi‐structured and all interviewees were asked the same set of questions. All interviews were conducted by the researcher. The interviews were recorded and transcribed and interviewees were asked to check the accuracy of the transcripts. A content analysis on the latent content and underlying symbolism was then performed against the transcripts (Punch 2005). The results of the interviews and subsequent content analysis were reviewed and compared with the results of the survey.

5.3 Limitations For privacy and other reasons, banks in Wellington were reluctant to allow the researcher access to their customers for this survey. As a result the preponderance of respondents were bank employees or graduate students from Victoria University in Wellington. The majority of these respondents considered The National Bank to be their primary bank because it has a branch on the university campus. In order to be able to draw firm conclusions across the New Zealand banking industry, future studies should aim to include customers of more New Zealand banks, including banks with an online‐only presence. Differences in privacy perceptions and associated expectations can be expected to exist between students and other bank customers, highlighting the need to differentiate the student population of the study from the non‐student population and across different OSNs. However, the results were considered worth reporting because of the topicality of the investigation.

6. Survey findings and discussion Of the 122 valid survey responses received, 56% were male and 44% were female. The ages of the respondents ranged between 21 and 64, with the average age being 36, indicative of a largely student‐based population. As explained above, the largest subset of respondents was from The National Bank with ANZ having the fewest respondents. The majority of respondents participated in Facebook, but more than half also used LinkedIn, reflecting the professional nature of the survey population. It is not possible within the specified word limits to reproduce the full data on which these findings are based. Accordingly the analyses will be summarised below. Question 1: Will the trust that bank customers feel towards their bank be affected if the bank uses OSN data when making business decisions? Given that trust is viewed as a “three‐dimensional construct” (Dimitriadis, et al., 2011), 42% of survey respondents indicated a high perceived trust, while only 1% of respondents indicated no perceived trust in their primary bank.

131

Tony Hooper and Shivonne Londt Survey respondents’ trust in their primary bank was then linked with the notion of use of OSN data. Amongst those respondents who have a high perceived trust for their primary bank, the majority (55%) had a low perceived trust for the use of OSN data. Twenty‐four percent of respondents had a high perceived trust for the use of OSN data. This indicates that the majority of respondents who trust their bank consider that bank use of their OSN data has the potential to cause them harm. The subsequent semi‐structured interviews of several respondents confirmed that the use of OSN data in this manner would cause those respondents to lose trust in their bank. Question 2: If the customer feels that their bank betrayed their trust by using OSN data will that translate into a perception of privacy violation? Again, it is not possible to detail the basis for the analysis of this data because of word limits on this paper. However, based on the definition of betrayal from Caldwell, et al. (2009), perceived betrayal as a result of banks’ use of customers’ OSN data would be:

a voluntary choice. The customer chooses whether or not to feel betrayed.

a violation of expectations in the nature of the relationship between the customer and their bank. Banks are considered to have a fiduciary duty to their customers, which indicates that their customers will expect them to act in good faith and for their best interests (Easterbrook & Fischel, 1993). These expectations are pivotal to the nature of the relationship.

understood by both the banks and their customers based on the expectations above.

From the survey results, 23% of respondents believed that banks’ use of OSN data would be a betrayal of trust. Only 3% of respondents believe that the use of OSN data is not a betrayal. The remaining 74% recorded answers that were less decisive. Of the group of respondents who perceived that a betrayal would occur if banks used their OSN data, 72% considered that a perceived privacy violation would also occur. This indicates that a perceived betrayal has a high likelihood of resulting in a perceived privacy violation. Bateman, et al (2010), recorded that, in terms of CPMT, individuals create psychological barriers around their personal information, thus limiting what they share and with whom. Accordingly, OSN data taken without the full consent and awareness of customers, is likely to lead to perceptions of betrayal and privacy violation.

7. Semi‐structured interview findings and discussion 7.1 Banks use of OSN data Respondents’ opinions were evenly divided around the acceptability of banks using data posted on OSNs to influence their lending decisions. Some respondents argued that the information may be taken out of context by the banks. One respondent recorded that there was great potential for misuse by the users of the OSNs as they could manipulate their data to create for themselves a more favourable impression. This confirms Clark & Roberts’ (2010) suggestion that using OSNs to conduct background checks will change users’ actions and behaviours. It could also have a negative impact on the OSNs as a communications medium. Respondents who considered such use of OSN data to be acceptable argued that the customer should be informed up front and that banks should respect any privacy settings that might be applied to the information. One respondent felt that the information was in the public domain and that use of it by banks was entirely fair.

7.2 Perception of bank as a benevolent entity Most respondents did not consider their banks to be benevolent entities. For the single respondent who indicated a perception of benevolence towards their primary bank, there was still the acknowledgement that the bank was a business and would ultimately seek to make money and to make a profit. This did not appear to impact on trust for the bank or perceptions of benevolence, however. One respondent was quite vehement that they did not consider their bank to be “in the least bit benevolent”. This attitude was attributed to bad experiences that the respondent had had with their primary banks and would not necessarily be common amongst all customers of that bank.

132

Tony Hooper and Shivonne Londt

7.3 Expectations around banks’ use of data from OSNs Most respondents expected that their banks would use the information if it was available. Some felt that the possibility of such use should be disclosed up front. Two respondents, however, felt that the bank should not use data from OSNs. One felt that OSN data would provide no additional value while the other felt that OSN data would not be accessible to them due to users’ privacy settings. One respondent observed that OSN data reveals individuals on a personal level, rather than how they deal with their finances.

7.4 Influence of trust on banks’ use of OSN data For certain respondents, their lack of trust for their primary bank meant that the idea that their bank might use their OSN data was entirely repugnant and would result in them “moving banks to a bank that said they didn’t”. Other respondents expected their bank to collect any “information that might be available in the public domain to improve their decision‐making process”. Amongst those respondents who indicated that they did trust their primary bank, one had no issue with their bank using their OSN data in this manner. Another, indicated that the use of their OSN data in this manner would cause that respondent to lose trust in their bank.

8. Conclusion As indicated in the findings above, respondents' expectation of privacy was influenced by their awareness of the OSN privacy policies. The less aware respondents were, the greater the expectation of privacy within OSNs, indicating that respondents may have a false expectation of privacy. The findings above also highlighted the fact that most respondents had not actually read the privacy policies of the OSN sites, possibly contributing to an over‐expectation of privacy on OSNs. Despite the limitations, the study revealed a high likelihood that New Zealand bank customers would perceive a privacy violation should local banks use OSN data to influence lending decisions. Accordingly, New Zealand banks should exercise care and transparency to manage customers’ expectations and trust. Despite users’ expectations and their perceived awareness of privacy policies and settings, the study showed that privacy is not always guaranteed. The study emphasised the need for users to read and understand the implications of both the privacy policies and the privacy settings of their accounts in order to ensure that the information that they wish to keep private does, indeed, remain so. The results of the study suggest that further research is needed into other factors that influence respondents’ perception of third party use of OSN data. Further research is also needed into the factors that influence respondents’ awareness of privacy policies and what respondents perceive the term “awareness” to mean.

References ASB Bank Limited. (2009) “Privacy ‐ ASB Bank New Zealand”, [online], ASB Bank Limited, https://www.asb.co.nz/story335.aspx ASB Bank Limited. (2011). “ASB Careers 4u”, [online] ASB Bank Limited, https://careers.asbgroup.co.nz/asb/our‐people Baird, C. H., and Gonzalez‐Wertz, C. (2011) “How top performers achieve customer‐focused market leadership”, Strategy & Leadership, Vol 39, No 1, pp 16‐23. Bateman, P. J., Pike, J. C., and Butler, B. S. (2010) “To disclose or not: publicness in social networking sites”, Information Technology & People, Vol 24, No 1, pp 78‐100. Bloland, H. G. (1989) “Higher Education and High Anxiety: Objectivism, Relativism, and Irony”, The Journal of Higher Education, Vol 60, No 5, pp 519‐543. Boot, A. W. A., and Marinĉ, M. (2008) “The evolving landscape of banking”, Industrial and Corporate Change, Vol 17, No 6, pp 1173‐1203. Brandenburg, C. (2008) “The Newest Way to Screen Job Applicants: A Social Networker’s Nightmare”, Federal Communications Law Journal, Vol 60, No 3, pp 597‐626. Caldwell, C., Davis, B., and Devine, J. A. (2009). “Trust, Faith and Betrayal: Insights from Management for the Wise Believer”, Journal of Business Ethics, Vol 84, pp 103‐114. Chen, J., Ping, W., Xu, Y., and Tan, B. C. Y. (2009). “Am I Afraid of My Peers? Understanding the Antecedents of Information Privacy Concerns in the Online Social Context”, Paper presented at the Thirtieth International Conference on Information Systems, Phoenix, Arizona, December. Child, J. T., and Petronio, S. (2010). Unpacking the paradoxes of privacy in CMC relationships: The challenges of blogging and relational communication on the internet, Hampton Press, Cresskill, NJ. Clark, L. A., and Roberts, S. J. (2010) “Employer’s Use of Social Networking Sites: A Socially Irresponsible Practice”, Journal of Business Ethics, Vol 95, pp 507‐525. Cropanzano, R., and Mitchell, M. S. (2005) “Social Exchange Theory: An Interdisciplinary Review”, Journal of Management, Vol 31, No 6, pp 874‐900.

133

Tony Hooper and Shivonne Londt Dimitriadis, S., Kouremenos, A., and Kyrezis, N. (2011) “Trust‐based segmentation”, International Journal of Bank Marketing, Vol 29, No 1, pp 5‐31. Easterbrook, F. H., and Fischel, D. R. (1993) “Contract and Fiduciary Duty”, Journal of Law and Economics, Vol 36, No 1, pp 425‐446. Finney, M. (2010) “Banks mining social media sites for personal info”, [online] http://abclocal.go.com/kgo/story?section=news/7_on_your_side&id=7283384 KiwiBank. (2011a) “Code of banking responsibility – Kiwibank”, [online] http://www.kiwibank.co.nz/about‐us/more‐about‐ us/code‐of‐banking‐responsibility.asp KiwiBank. (2011b) “Privacy and Security – Kiwibank”, [online] http://www.kiwibank.co.nz/privacy‐and‐security/ Luo, X. (2002) “Trust Production and Privacy Concerns on the Internet: A Framework Based on Relationship Marketing and Social Exchange Theory”, Industrial Marketing Management, Vol 31, pp 111‐118. Morgan, R. M., and Hunt, S. D. (1994) “The Commitment‐Trust Theory of Relationship Marketing”, Journal of Marketing, Vol 58, No 3, pp 20‐38. Nissenbaum, H. (1998) “Protecting Privacy in an Information Age: The Problem of Privacy in Public”, Law and Philosophy, Vol 17, No 5/6, pp 559‐596. Onwuegbuzie, A. J., Johnson, R. B., and Collins, K. M. T. (2009) “Call for mixed analysis: A philosophical framework for combining qualitative and quantitative approaches”, International Journal of Multiple Research Approaches, Vol 3, No 2, pp 114–139. Petronio, S. (2007) “Translational Research Endeavors and the Practices of Communication Privacy Management”, Journal of Applied Communication Management, Vol 35, No 3, pp 218‐222. Pike, J. C., Bateman, P. J., and Butler, B. S. (2009) “I Didn’t Know You Could See That: The Effect of Social Networking Environment Characteristics on Publicness and Self‐Disclosure”, Paper presented at the Fifteenth Americas Conference on Information Systems, San Francisco, California. Privacy Commissioner. (n.d.) “Purpose for collection of personal information (principle one)”. [online] Privacy Commissioner http://privacy.org.nz/purpose‐for‐collection‐of‐personal‐information‐principle‐one nd Punch, K. F. (2005) Introduction to Social Research: Quantitative and Qualitative Approaches, (2 ed.). SAGE Publications Ltd, London. RaboDirect. (2011) “Public confidence in finance sector drops while people say investments worth more”, [online] RaboDirect http://www.rabodirect.co.nz/rabodirect‐tv‐radio‐ads/2011/financial‐confidence‐index‐press‐release‐ new‐zealand‐april‐2011.aspx Reynolds, G. W. (2010) Ethics in Information Technology (3rd ed.), Course Technology, Cengage Learning, USA. Ryder, M. (2008) “The Cyborg and the Noble Savage: Ethics in the war on information poverty”, [online] http://carbon.ucdenver.edu/~mryder/savage.html#def_constructivism Talja, S., Tuominen, K., and Savolainen, R. (2005). " ‘Isms’ in information science: constructivism, collectivism and constructionism”, Journal of Documentation, Vol 61, No 1, pp 79‐101. The National Bank of New Zealand. (2011a) “National Bank – Commitments”, [online] The National Bank of New Zealand, http://www.nbnz.co.nz/promos/customercommitments/ The National Bank of New Zealand. (2011b) “Privacy Policy”, [online] The National Bank of New Zealand, http://www.nbnz.co.nz/terms/privacy.aspx TSB Bank Limited. (2009) “TSB Bank ‐ Privacy Statement’, [online] TSB Bank Limited, http://www.tsb.co.nz/Info/PrivacyStatement.aspx Westpac New Zealand Limited. (2008) “Westpac New Zealand Working for Westpac”, [online] Westpac New Zealand Limited, http://www.westpac.co.nz/olcontent/olcontent.nsf/Content/Working+for+Westpac

134

Changes in use, Perception and Attitude of First Year Students Towards Facebook and Twitter Kevin Johnston, Mei‐Miao Chen and Magnus Hauman University of Cape Town, Cape Town, South Africa kevin.johnston@uct.ac.za Abstract: As social computing systems persist over time, many elements such as user experience, perceptions, attitudes and interactions may change. Facebook and Twitter are two social computing systems that have become increasingly popular among university students. This research replicates studies by Lampe, Ellison and Steinfield (2008), and Johnston, Tanner, Lalla and Kawalski (2011) to assess how Facebook and Twitter use, perception and attitude have changed among first year students at the University of Cape Town (UCT) in South Africa. Because online social networks, social networking sites and micro‐blogging sites are relatively young as an area of academic research, there is limited research into the impacts of these social networking and micro‐blogging sites, particularly on South African university students. A sample of 486 students completed the survey. Compared to research data from previous studies by Lampe et al. (2008) and Johnston et al. (2011), this study found that the percentage of students using Facebook increased to 95%, Facebook daily usage and the number of Facebook friends had doubled. This study also found that UCT first year students are more dependent on using Facebook, in comparison to using Twitter; that their perception of Facebook privacy has led to a decrease in personal information shared on Facebook as well as a change in audience perception. The data also shows that UCT first year students perceive friends and total strangers to be their main audiences on Twitter; the attitude of UCT students towards Facebook remained positive, on the other hand, a less positive attitude was experienced from the students using Twitter; and Facebook is a more popular method for communication between students. All findings are within the South African context. One can see there have been changes in usage, attitude and perception of Facebook over time. Keywords: Facebook, Twitter, use, perception, attitude, students

1. Introduction This research replicated aspects of studies into Social network sites (SNSs) at Michigan State University (MSU) by Lampe, Ellison and Steinfield (2008), and of South African (SA) university students by Johnston, Tanner, Lalla and Kawalski (2011), adding an additional element of micro‐blogging (Twitter). The aim was to establish the usage, user perception of and attitude to Facebook and Twitter among first year students at the University of Cape Town (UCT) in 2011, and compare the data to the Lampe et al. (2008) and Johnston et al. (2011) studies. A positivist philosophy was adopted, a survey instrument adapted with permission from Johnston et al. (2011) was used, and all respondents remained anonymous. Limitations included the fact that 85% of the student respondents were first‐year students in Information Systems. Hence respondents were likely to be more computer literate, and have greater interests and access to computers and the internet in comparison to other students.

2. Literature review Two main research papers were used for Facebook, the first by Lampe et al. (2008) was conducted between 2006‐2008, and the second by Johnston et al. (2011) was conducted in 2008. Lampe et al. (2008) focused on the change in use and perception of Facebook among students at Michigan State University (MSU). Johnston et al. (2011) focused on the use of Facebook amongst students at seven South African (SA) universities, and the impact of Facebook friends on social capital. For Twitter, the primary papers were by Java, Song, Finin, and Tseng (2007), and Zhao and Rosson (2009) who focused on how and what micro‐blogging was used for. Social Network Sites (SNS) are an internet‐based media platform that has grown in popularity in recent years (Roblyer, McDaniel, Webb, Herman, & Witty, 2010). Lampe et al. (2008, p721) defined SNSs such as Facebook, to have three main characteristics which allowed users to “(1) construct a public or semi‐public profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within the system”. SNSs create an online community that relies on user participation and contribution (Waters, Burnett, Lamm & Lucas, 2009). Typical SNSs provide users with profile space, and allow users to upload content (such as photographs, statuses and interests) about themselves. SNSs also provide a platform where users can connect and communicate with one another (Joinson, 2008). Furthermore, an SNS acts like an online bulletin board allowing users to view other people’s posts and content (Pempek, Yermolayeva & Calvert, 2009).

135

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman Micro‐blogging is a variant of blogging where short messages are posted instantly (Fischer & Reuber, 2010; Java et al., 2007). Micro‐blogging is used by both consumers and businesses to share news, post status updates and carry on conversations (Reinhardt, Ebner, Beham & Costa, 2009). Messages can be restricted to a certain number of individuals, sent exclusively to a specific contact, or made available to the Worldwide Web (Reinhardt et al., 2009). “Twitter is an Internet social‐network and micro‐blogging platform with both mass and interpersonal communication features for sharing 140‐character messages, called Tweets, with other people, called followers” (Chen, 2011, p. 755).

2.1 Facebook Facebook is an SNS used primarily to connect, interact and stay in touch with contacts that the user knows personally, such as friends, family and colleagues (Ellison, Steinfield, and Lampe, 2007). Facebook is one of the most popular SNSs (Lampe et al., 2008), and has been widely adopted by students (Roblyer et al., 2010). A typical Facebook user spends 10‐20 minutes a day on the site, and around two‐thirds log in more than once a day (Ellison et al., 2007). Hew (2011) stated that there are four elements within a Facebook profile, namely: control elements, referents elements, preference elements, and contact elements. Control elements include a users gender and time they have been a member, referent elements are fields such as home town and high schools, where common points of reference among users can be established. Preference elements are fields that express the personal interests of the user and short biographies (Hew, 2011). Contact elements are fields such as mobile contact numbers and e‐mail addresses (Hew, 2011). Furthermore, users can set their privacy settings to include or exclude certain users from viewing parts of their profile (Pempek et al., 2009). Joinson (2008) found various heterogeneous uses of Facebook such as keeping contact awareness, photo sharing, group organisations, and application participation. Several studies have shown that the use of Facebook is associated with social capital gains (Ellison et al., 2007; Johnston et al., 2011; Ryan & Xenos, 2011). Studies conducted by Lampe et al. (2008) on MSU students, and Johnston et al. (2011) on SA students showed responses to the question “I use Facebook to...” were similar. The main uses of Facebook were– “keeping in touch with old friends” and to “check out someone I met socially”.

There are seven motives why users use Facebook. Firstly, to maintain existing relationships, to stay in touch and maintain relations with people (Hew, 2011; Roblyer et al., 2010). Secondly, to meet new people, find out information about people previously met, and meet others online with the intention of meeting them offline later (Hew, 2011; Joinson, 2008). Thirdly, using Facebook is perceived as “cool” and “fun”, as people enjoy following trends and don’t like to feel left out (Hew, 2011). Fourthly, to raise public awareness of oneself, where users can gain popularity through having Facebook friends (Hew 2011; Pempek et al., 2009). Fifthly, the ability to express or present oneself, by updating their profiles and statuses (Hew, 2011; Smith & Kidder, 2010). The sixth motive – for learning purposes – lets users communicate with one another, ask and share questions and information (Hew, 2011; Roblyer et al., 2010). The seventh motive –enabling of student activism – lets students provide general information about activities, and express their viewpoints (Hew, 2011). The perception users have of their audience plays an important role in the usage of Facebook, as if the user perceives no audience for his or her profile, he or she is less likely to make use of Facebook (Lampe et al., 2008). The type of perceived audience also determines the type of information and content users are likely to share on their profiles (Pempek et al., 2009). Studies conducted by Lampe et al. (2008) on MSU students, and Johnston et al. (2011) on SA students showed that as the offline relations become weaker, the less students consider them as perceived audiences. The overall student attitude towards communicating on Facebook is positive (Hew, 2011). Lampe et al. (2008) observed a significant increase in positive attitudes towards Facebook between 2006 and 2008, despite a small increase in some negative effects.

2.2 Twitter Twitter is a micro‐blogging platform that allows users to share their thoughts, information and links in real time, and to communicate directly, privately or publicly with other Twitter users (Thomases, 2010). Users can tweet about any topic within the 140‐character limit (Fischer & Reuber, 2010; Thomases, 2010). A follower is

136

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman someone who has opted to receive tweets from a specific person, whereas following is whom the person has opted to follow and receive tweets from (Java et al., 2007; Thomases, 2010). Pennacchiotti and Popescu (2011, p.281) stated that the success of Twitter has lead Twitter to “become an integral part of the daily lives of millions of users”. Users update their daily life activities with friends, families and co‐workers; share information, news, and opinions with interested observers; and seek knowledge and expertise in public tweets (Java et al., 2007; Pennacchiotti & Popescu, 2011). Honeycutt and Herring (2009) stated that Twitter has the potential to be used for idea sharing and coordination of activities, similar to instant messaging in a more dynamic way. The main types of user intentions on Twitter are: daily conversations, sharing information and website URLs, and reporting news (Honeycutt & Herring, 2009; Java et al., 2007). Twitter is also used for keeping in touch with friends and colleagues, sharing interesting information within one’s network, seeking help and opinions, and releasing emotional stress (Ehrlich & Shami, 2010; Zhao & Rosson, 2009). A Twitter user can create a public profile which entails full name, location, a web page, a short biography and the tweets of the user (Kwak, Lee, Park & Moon, 2010; Pennacchiotti & Popescu, 2011). The people who follow the user and those that the user follows are also listed (Pennacchiotti & Popescu, 2011). Ehrlich and Shami (2010) indicated that users are less likely to post confidential information on Twitter.

2.3 The South African context One cannot conduct a study on South Africa without understanding the social context of the country. Apartheid in SA played a major role in creating gaps and inequalities between the different races (Johnston et al., 2011). Apartheid actively humiliated and oppressed races that were considered ‘non‐white’, limiting the ‘non‐whites’ in their education and the ability to accumulate and use assets, only 9% of South Africans are white (Johnston et al., 2011). Although South Africa was not the only country to propose and implement racial segregation, it has been “a distinctive feature through which communities and neighbourhoods in South African cities have been understood” (Oldfield, 2004, p.190). The racial segregation within South Africa had a formative impact on urban economic, social and political form (Oldfield, 2004). The racial segregation made South Africa a country with a high GINI coefficient, and one of the most unequal societies in the world (Armstrong, Lekezwa, & Siebrit, 2008).

3. Methodology The study was done on the 2011 cohort of UCT students in the Faculty of Commerce, and aimed to understand the use, attitudes and perceptions of Facebook and Twitter. The philosophy was positivist. The questionnaires relied on the subject’s subjective observations of themselves. Hence, there was bias in the data; however it is assumed that the biases will average out, and compensate for individual differences. The data analysis was quantitative to correspond with the studies done by Lampe et al. (2008) and Johnston et al. (2011). As in the two studies, five‐point Likert scales were used extensively in the questionnaire with 1 = strongly disagree or very unlikely, and 5 = strongly agree or very likely. Data could therefore can be compared and analysed. The sample comprised all 1231 registered first year Commerce students at UCT in 2011. An online survey was made available to students and obtained 488 responses. The raw data was cleaned and formatted, two responses were not fit to be used, reducing the sample to n=486.

4. Data analysis The results of the research (referred to as ‘2011 UCT’) were compared with results from previous studies done by Michigan State University (referred to as ‘2008 MSU’) (Lampe et al., 2008) and Johnston et al. (2011) (referred to as ‘2008 SA’ as it was completed in 2008). The Facebook intensity scale which attempted to obtain a measure of Facebook usage and duration of use was adopted from the 2008 SA study (Johnston et al., 2011). In the 2008 SA study, 67% of students were members of Facebook (Johnston et al., 2011), whereas 95% of the UCT students surveyed in 2011 were members of Facebook. Table 1 compares the statistics of Facebook (FB) intensity for the 2008 MSU students (Lampe et al., 2008), the 2008 SA students (Johnston et al., 2011), and the 2011 UCT students.

137

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman Table 1: Summary statistics for Facebook intensity Individual items and scales About how many Facebook friends do you have? 0=<10, 1=11‐50, 2=51‐100, 3=101‐150, 4=151‐200, 5=201‐ 250, 6=251‐300, 7=301‐400, 8=401‐500, 9=>501 In the past week, on average, approximately how many minutes per day have you spent on Facebook 0=<10, 1=10‐30, 2=31‐60, 3=1‐2 hours, 4=2‐3 hours, 5=>3 hours Facebook is part of my everyday activity I am proud to tell people I am on Facebook Facebook has become part of my daily routine I feel out of touch when I haven’t logged onto FB for a while I feel I am part of the Facebook community I would be sorry if Facebook shut down My Facebook has caused me problems

2008 MSU Mea Std. Dev n 7.33

2008 SA Mea Std. Dev n 3.17 2.31

2011 UCT Mea Std. Dev n 6.78 2.35

3.37

1.69

1.44

2.18

1.44

3.85 3.34 3.66 3.72 2.20

1.12 0.85 1.19 1.34 1.12

3.03 3.53 3.07 2.74 3.36 3.58

1.31 1.00 1.27 1.37 1.08 1.23

3.72 3.68 3.59 3.14 3.53 3.64 2.37

1.05 0.91 1.04 1.23 0.96 1.17 1.13

The 2008 MSU students had an average of 333 friends (mean of 7.33), whereas the 2008 SA students had an average of 134 friends (mean 3.17). In the 2011 study of UCT students, students had an average of 251–300 friends (mean of 6.78). All groups of students in Table 1 leaned towards agreeing that Facebook was part of their daily routine and everyday activity. Similarly, all groups are leaning towards agreeing that they would be sorry if Facebook had to shut down, thus students have an overall positive attitude towards Facebook. When students were asked about problems caused by Facebook, the means were below 2.50 indicating that the majority disagreed with the statement. Figure 1 shows that 72% of the 2011 UCT students indicated that they used Facebook to view the profiles of other students in one of their classes; 55% used Facebook to contact another student with questions relating to class work; 40% used Facebook to arrange study groups and meetings; and 25% were given Facebook‐related class assignments. Facebook was used by students to assist in their education by this group of students, and thus could be used as a communication channel to assist academics.

Figure 1: Usage of Facebook within an educational context Table 2 shows that students were less inclined to use Facebook to meet new people (overall mean of 2.41), and strongly inclined to use Facebook to maintain existing contacts. When asked to rate the statement “I use Facebook to keep in touch with my old friends,” 2008 MSU students had a 4.50 mean, the 2008 SA study had a 4.47 mean, and 2011 UCT study had the highest mean of 4.56. The second highest set of means related to the statement, “I have used Facebook to check out someone I met socially.” Table 2 shows that the main uses of Facebook by all groups are the same, namely: to keep in touch with old friends, and to check out someone they have met socially, in other words to connect with offline contacts, rather than to meet new people.

138

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman Table 2: Facebook use for prior contacts and meeting new people Individual items and scales Off to Online: Use Facebook to connect with offline contacts I have used FB to check out someone I met socially I use FB to learn more about other people living near me I use FB to keep in touch with my old friends On to Offline: I use Facebook to meet new people I use FB people to find people to date I use FB to find people to add to my ‘friends’ list I use FB to meet new people I use FB to find out things that’s happening around me I use FB to find out worldwide news or facts I use FB to find out about specials and promotions

2008 MSU Mea Std. Dev n 3.79 0.90

2008 SA Mea Std. Dev n 3.56 1.17

2011 UCT Mea Std. Dev n 3.75 0.99

3.94 2.95

0.89 1.15

3.57 2.64

1.33 1.29

4.03 2.65

1.05 1.25

4.50 1.88 1.62 2.14 3.51

0.67 1.04 0.95 1.12 1.10

4.47 2.55 2.25

0.89 1.45 1.45

4.56 2.41 1.50 2.02 2.05 3.72 2.74 2.26

0.66 1.17 0.92 1.23 1.29 1.14 1.27 1.26

Figure 2 shows which elements students included in their Facebook profiles. The most commonly selected elements in 2008 were contact details (78%) and a photo of the student (66%); whereas 88% of the 2011 students included educational details, and 83% a photo. Although 80% of 2011 students included their home town, only 14% included an address. There has been a decrease from 78% to 65% in sharing contact details such as email and mobile numbers.

Figure 2: Profile elements included for Facebook The 2008 MSU, 2008 UCT and 2011 studies revealed similar results with regards to perceived profile view (Figure 3), with friends perceived as the main audience. ‘High school friends’ are still being perceived as the main profile audience (2008 MSU had 94%, 2008 UCT had 88% and 2011 UCT had 97%). The biggest variation was the perceived profile audience of ‘someone I met at a party or social’, 2008 MSU study had 72%, the 2008 UCT study had 40%, and the 2011 UCT study had 64%. When asked about perceived professional audiences viewing the profile, only 7% of the 2011 UCT students perceived that companies would view their Facebook sites. Table 3 provides a comparison of Facebook members across the studies. The number of Facebook members within South Africa increased from 67% to 95% in 2011. In both the 2008 SA study and the 2011 UCT study, over 26% of the Facebook respondents were white, whereas only 9% of the country’s population is white. This is a reflection of the inequality in the country. Of the students surveyed, 34% (163 respondents) were members of Twitter. Table 4 shows that most spent less than 10 minutes per day on Twitter, and claimed to have between 11‐50 followers, while following 11–50 Twitter users. Respondents did not consider Twitter a part of their daily activity (mean of 2.38), or daily routine (mean of 2.46) as they did with Facebook.

139

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman

Figure 3: Perceived profile viewers for Facebook Table 3: Comparison of Facebook members

2008 MSU

2008 SA

2011 UCT

Number of respondents Facebook members Facebook use (minutes per day)

n = 286 94% 10‐30 minutes

n = 572 67% 10‐30 minutes

n = 486 95% 31‐60 minutes

Facebook friends Ethnicity of FB members (White/Black)

151‐200 82%/18%

101‐150 28%/72%

251‐300 27%/73%

Table 4: Statistics for Twitter intensity (n=163) Individual items and scales About how many Twitter followers do you have?

2011 UCT Mean Std. Dev 1.08 1.59

0=<10, 1=11‐50, 2=51‐100, 3=101‐150, 4=151‐200, 5=201‐250, 6=251‐300, 7=301‐400, 8=401‐500, 9=>501

About how many people are you following on Twitter? same scale as above In the past week, on average, approximately how many minutes per day have you spent on Twitter 0=<10, 1=10‐30, 2=31‐60, 3=1‐2 hours, 4=2‐3 hours, 5=>3 hours Twitter is part of my everyday activity I am proud to tell people I am on Twitter Twitter has become part of my daily routine I feel out of touch when I haven’t logged onto Twitter for a while I feel I am part of the Twitter community I would be sorry if Twitter shut down My Twitter has caused me problems

1.62

1.77

0.81

1.22

2.38 3.33 2.46 2.06 2.56 2.59 1.80

1.41 1.14 1.36 1.21 1.24 1.31 1.02

Figure 4 shows that 7% of Twitter respondents had used Twitter to arrange study groups and meetings, while 4% had contacted other students relating to class work. However, 18% of 2011 UCT students indicated that they had used Twitter to view the profile of a fellow student, and 9% had used Twitter to view the profile of a professor, lecturer or tutor. Twitter was mainly used by UCT students to find out worldwide news or facts (mean of 3.43), secondly to find out local news and events (mean of 3.16), thirdly to find out about specials and promotions (2.38), and only fourthly to keep in touch with old friends (mean of 2.35) as seen in Table 5. Twitter was used to find out about issues, rather than to socialise. Sixty‐five percent of the students using Twitter indicated that they had included their real name in their profile, and 60% also included a pen name; 42% included a profile picture of themselves and 44% included

140

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman location within their profile. Students are more likely to share personal confidential details on Facebook than on Twitter, as stated by Ehrlich and Shami (2010).

Figure 4: Usage of Twitter within education context Table 5: Twitter use for prior contacts and meeting new people Individual items and scales Off to Online: Use Twitter to connect with offline contacts

2011 UCT Mean Std. Dev 2.19 1.31

I have used Twitter to check out someone I met socially I use Twitter to learn more about other people living near me I use Twitter to keep in touch with my old friends

2.13 2.09 2.35

1.33 1.22 1.39

On to Offline: I use Twitter to meet new people I use Twitter people to find people to date I use Twitter to find people to add to my ‘friends’ list

2.40 1.48 1.96

1.26 0.86 1.22

I use Twitter to meet new people I use Twitter to find out things that’s happening around me I use Twitter to find out worldwide news or facts

1.96 3.16 3.43

1.23 1.50 1.50

I use Twitter to find out about specials and promotions

2.38

1.44

Figure 5: Profile elements included for Twitter vs. Facebook Sixty‐one percent allowed anyone to view their Twitter profile, 15% did not indicate their privacy setting, and only 24% restricted views to approved audiences. Similarly to Facebook, high school friends were perceived as the main audience (75%), whereas only 14% perceived someone they have met socially to be their audience.

141

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman Figure 6 shows who respondents perceived as viewers of their Twitter and Facebook profiles, and as expected the percentages of Twitter viewers were lower than the percentages of Facebook users.

Figure 6: Perceived profile viewers for Twitter vs. Facebook Out of the total number of Twitter members in the current study, 25% indicated that they were white. The ethnicity among students who were members of Twitter and Facebook was unsurprisingly similar.

5. Discussion Of the students surveyed (N=486), 95% were Facebook users and 33% were Twitter users. The 2008 MSU study indicated that 6% of the students surveyed were not registered Facebook members, whereas 33% of 2008 UCT students surveyed were not registered on Facebook. In the current study only 5% of respondents were not registered on Facebook. With regard to Twitter, 67% of 2011 UCT respondents were not Twitter members, while 95% of these respondents were members of Facebook. The current study indicated that time spent using Facebook per day (24%) fell within the 30 to 60‐minute category, higher than the MSU and 2008 UCT study. Similarly, the number of Facebook friends increased, 2008 MSU students had 150 – 200 friends, the 2008 UCT students had 101 – 150 friends, whereas the 2011 UCT students had between 400 – 500 friends. This may be because the 2011 students have been using Facebook for a longer period. With regards to Twitter, the average time spent was under 10 minutes (28%), and most members are following and are being followed by 11 – 50 fellow Tweeters. The main goal UCT students used Twitter for was to ‘find out about things happening around them and in the world’ rather than to socialise. The study has shown an increase in registered Facebook members when compared to previous studies (Johnston et al., 2011), indicating that Facebook is becoming more popular amongst South African tertiary students. South African students have been slow to adopt Twitter. The research established that there has been an increase in popularity of Facebook within the South African tertiary student community. The main use of Facebook remained consistent with the literature, Facebook was mainly used for “keeping in touch with old friends” and to “check out someone I met socially” (Ellison et al., 2007; Hew, 2011; Johnston et al., 2011; Lampe et al., 2008; Pempek et al., 2009; Roblyer et al., 2010). Hew (2011) and Pempek et al. (2009) both stated that although ‘learning purposes’ was listed as a motive for students making use of Facebook, Facebook was not widely used for learning purposes. However, our study proved the opposite, with 55% of the students surveyed using Facebook to contact other students for questions relating to class work, and 47% making use of Facebook to discuss class work. The literature indicated that Twitter is mainly used for daily conversations, sharing information and website URLs and reporting news (Honeycutt & Herring, 2009; Zhao & Rosson, 2009). Our study revealed that students at UCT mainly used Twitter to find out world‐wide facts and events happening around them. Ehrlich and Shami (2010) and Zhao and Rosson (2009) stated that users made use of Twitter to keep in touch with friends and colleagues. However, our findings proved the opposite; students at UCT in 2011 did not make extensive use of Twitter to keep in touch with friends. The literature revealed that the perception users have of their audience plays a role in determining the type of information and contents which users are more likely to share on their profiles (Pempek et al., 2009). The

142

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman current study established that 2011 UCT students are less likely to share confidential information such as their contact details in comparison to 2008 students (Johnston et al., 2011). Lampe et al. (2008) and Johnston et al. (2011) both identified that student Facebook users were more likely to consider their peers as their main audience. Furthermore, Lampe et al. (2008) found that as Facebook became more popular, student Facebook users perceived more family members as their profile audience. The current study found similar results with regard to perceived Facebook audience. The students’ attitudes towards Facebook and Twitter at UCT in 2011 was positive. The literature (Lampe et al., 2008) stated that Facebook had slowly become an important part of the lives of the students and would be missed if it had to close down. The statement was supported by the current study. Lampe et al. (2008) indicated that Facebook had become a popular method for communicating between college‐age users; this was supported by the current study. The main use of Facebook was for communication with friends and to keep in touch with old friends (Pempek et al., 2009). Although literature stated that Twitter is the most popular micro‐blogging tool among other existing equivalents (Zhao & Rosson, 2009), and that Twitter has seen a dramatic growth in the number of registered users, our findings showed a low percentage of respondents (34%) were Twitter users. The sole purpose of Twitter users at 2011 UCT was to be kept updated with world‐wide facts and happenings around them.

6. Conclusion The research replicated aspects of studies performed at MSU in 2008, and in South Africa in 2008. The study aimed to assess the change in Facebook usage, perception and attitude among UCT students and whether or how it is used as a communication channel, drawing comparisons to the 2008 MSU and 2008 SA study. The study included Twitter as there appeared to be a gap in the academic literature. The research questions were to establish the usage of Facebook and Twitter at UCT, and the user perception and attitude towards Facebook and Twitter at UCT. The main use of Facebook remained consistent with the literature, “keeping in touch with old friends” and to “check out someone I met socially”, while Twitter was mainly used for finding news and information. Facebook was also used for learning purposes; with 55% of respondents using Facebook to contact other students with academically related issues, and 47% using Facebook to discuss class work. The student’s perception and attitude towards Facebook was extremely positive, and the vast majority are members, and would be sad should Facebook close down. UCT students in 2011 are less likely to share confidential information such as their contact details in comparison to students in prior years. This study was conducted only on registered UCT students and is not an accurate sample of the entire South African populace. The research questions provide new avenues for future research regarding Facebook and Twitter, or more generally, online social networking within South Africa. Future research could attempt to replicate the Twitter study to see the growth of Twitter within the South African tertiary institution context. Future studies could also address the use of mobile technology within social networking in the South African context.

References Armstrong, P., Lekezwa, B., & Siebrit, K. (2008). Poverty in South Africa: A profile based on recent household surveys. Stellenbosch: Economic Research at the University of Stellenbosch. Chen, G. M. (2011). Tweet this: A uses and gratifications perspective on how active Twitter use gratifies a need to connect with others. Computers in Human Behaviour, 27(2), 755‐762. Ehrlich, K., & Shami, S. (2010). Micro‐blogging Inside and Outside the Workplace. ICWSM (pp. 42 ‐ 49). Association for the Advancement of Artificial Intelligence. Ellison, N. B., Steinfield, C., & Lampe, C. (2007). The Benefits of Facebook "Friends:" Social Capital and College Students' Use of Online Social Network Sites. Journal of Computer‐Mediated Communication, 1143‐1168. Fischer, E., & Reuber, R. A. (2010). Social interaction via new social media: (How) can interactions on Twitter affect effectual thinking and behaviour? Journal of Business Venturing, 26(1), 1‐18. Hew, K. F. (2011). Students’ and teachers’ use of Facebook. Computers in Human Behavior, 27, 662–676. Honeycutt, C., & Herring, S. C. (2009). Beyond Micro‐blogging: Conversation and Collaboration via Twitter. Proceedings of the 42nd Hawaii International Conference on System Sciences (pp. 1‐10). IEEE Press.

143

Kevin Johnston, Mei‐Miao Chen and Magnus Hauman Java, A., Song, X., Finin, T., & Tseng, B. (2007). Why we twiter: Understanding micro‐blogging usage and communities. Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 56‐65). New York: ACM. Johnston, K., Tanner, M., Lalla, N., Kawalski, D. (2011). 'Social capital: the benefit of Facebook 'friends''. Behaviour & Information Technology. 0(0), 1‐13. Joinson, A. N. (2008). ‘Looking at’, ‘Looking up’ or ‘Keeping up with’ People? Motives and Uses of Facebook. Conference on Human Factors in Computing Systems (pp. 1027‐1036). New York: ACM Press. Kwak, H., Lee, C., Park, H., & Moon, S. (2010). What is Twitter, a Social Network or a News Media? Proceedings of WWW, (pp. 591‐600). Raleigh. Lampe, C., Ellison, B., & Steinfield, C. (2008). Changes in Use and Perception of Facebook. Proceedings of the 2008 ACM conference on Computer supported cooperative work (pp. 721‐730). New York: ACM. Lampe, C., Ellison, N., & Steinfield, C. (2006). A Face(book) in the Crowd: Social Searching vs. Social Browsing. Computer Supported Cooperative Work (pp. 167‐170). Alberta: ACM. Mischaud, E. (2007). Twitter: Expressions of the Whole Self. An investigation into user appropriation of a web‐based communications platform. Media@lse, London School of Economics and Political Science, London. Oldfield, S. (2004). Urban networks, community organising and race: an analysis of racial integration in a desegregated South African neighbourhood. Geoforum, 35(2), 189 ‐ 201. Pennacchiotti, M., & Popescu, A.‐M. (2011). A Machine Learning Approach to Twitter User Classification. ICWSM‐11 (pp. 281‐288). Barcelona: Association for the Advancement of Artificial Intelligence. Pempek, T.A., Yermolayeva, Y.A., & Calvert, S.L. (2009). College students' social networking experiences on Facebook. Journal of Applied Developmental Psychology, 30, 227–238 Reinhardt, W., Ebner, M., Beham, G., & Costa, C. (2009). How people are using twitter during conferences. Proceeding of 5. EduMedia, (pp. 145‐156). Salzburg. Roblyer, M. D., McDaniel, M., Webb, M., Herman, J., & Witty, J. V. (2010). Findings on Facebook in higher education: A comparison of college faculty and student uses and perceptions of social networking sites. Internet and Higher Education, 13(2010), 134–140. Ryan, T., & Xenos, S. (2011). Who uses Facebook? An investigation into the relationship between the Big Five, shyness, narcissism, loneliness, and Facebook usage. Computers in Human Behavior, [Article in Press]. Smith, W. P., & Kidder, D. L. (2010). You've been tagged! (Then again, maybe not): Employers and Facebook. Business Horizons, 53, 491‐499. Thomases, H. (2010). Twitter marketing: An hour a day. Wiley publishing, Indianapolis. Waters, R,D., Burnett, E., Lamm, A & Lucas J (2009). Engaging stakeholders through social networking: How nonprofit organizations are using Facebook. Public Relations Review, 35(2), 102–106. Zhao, D., & Rosson, M. (2009). How and why people Twitter: the role that micro‐blogging plays in informal communication at work. Proceedings of the ACM 2009 International conference on Supporting group work (pp. 243‐252). New York: ACM.

144

Analysis of Dissatisfiers That Inhibit Cloud Computing Adoption Across Multiple Customer Segments Easwar Krishna Iyer, Arathi Krishnan, Gaurav Sareen and Tapan Panda Great Lakes Institute of Management, Chennai, India easwar@greatlakes.edu.in arathi.krishnan@greatlakes.edu.in gaurav.sareen@greatlakes.edu.in tapan@greatlakes.edu.in Abstract: Cloud computing in many ways can be viewed as both a technology offering and a business alternative. But its adoption today is driven more by economic rationale than by technology justifications. Though in delivery terms, cloud offering is a merger of state‐of‐the‐art concepts like virtualization, server consolidation, interoperability and dynamic CPU provisioning, its risk‐benefit analysis is purely driven by business imperatives. As a technology, Cloud Computing topped the Gartner’s Hype Cycle only as late as 2009. However since then the hype has settled down and ‘computing in the ethereal cloud’ is slowly emerging as a strong cost‐effective alternative to traditional computing. This paper focuses on the customer side perceptions of cloud adoption with a purely India‐centric perspective. Since cloud is a fairly new offering, there is bound to be a lot of inertia in its acceptance. This is because cloud offerings ‐ from the product development lifecycle (PDLC) point of view ‐ are at a nascent stage and hence perceived risks outweigh perceived gains. This paper focusses on the cloud adoption risks across four sectors – SME, BFS, Education and Hospitals. The four key risk categories identified in the context of cloud adoption are vendor related risk, security related risk, no‐gain risk and efficiency related risk. The paper does a relative mapping of these four risks for each of the four mentioned industry clusters. Since cloud technology is only in the process of getting established and main stream adoption is still a few years away, many of the cloud adoption fears are nebulous and will be removed once critical volumes start building up. Till such a maturation happens, cloud vendors will have to assiduously work out ways and means of assuaging the fears that inhibit adoption – real or perceptional. This paper is posited to be a pointer in that direction. Keywords: cloud computing, dissatisfiers, segmented risk profiling, risk perception management, conjoint regression

1. Introduction Cloud Computing being an evolving technology, current global research on it is focused more on technology and less on business. Eventually the acid test for any technology is its market acceptance. As far as market adoption goes, cloud is on the verge of crossing the vital chasm between the early adopters and the early majority. Early adopters have a psychographic profile of being venturesome and they are known to have fewer inhibitions in accepting a completely new technology offering. The current offerings, be it in applications, computing or storage is being lapped up by this target market. On the other hand, the early majority comprises of the pragmatists in the market and they cover the bulk of the real market. They tend to accept new technology only after measures against failure are reasonably in place. Need for referrals is a strong driver for a buy in this segment. Technology adoption by the pragmatic early majority is a crucial milestone for any nascent technology. This paper offers a multi‐sector, emerging market, customer side view of cloud adoption, a technology that is just one step away from the ‘early majority’ buyers. Any new technology comes in with a set of risks – real, latent or perceived. Literature survey supported by previous work done by some of the authors and reported elsewhere indicates that there are four clear risk vectors for cloud adoption. They are vendor related risk, data security related risk, lack of significant cost reduction risk and system efficiency risk. The authors posit vendor related risk as the first dimension of perceived risk. Fear of lock‐in with an incompatible vendor, lack of guarantee of business continuity and service availability, reputation fate sharing with a vendor and unclear licensing issues come under this risk profile. The next posited risk covers data security, data privacy, data confidentiality and loss of governance & control of IT delivery. The third risk posits that the gains that cloud purports to offer in terms of reduced capital and operative costs might not be sufficient enough to move from existing systems to cloud platforms. The last risk sums up the effects of latency, downtime, data bottlenecks and any other efficiency impediments. The paper compares the relative weightage of these four risks across four customer segments who seem to be

145

Easwar Krishna Iyer et al. potential adopters of cloud – SME sector, banking sector, education sector and hospital sector. Each segment chosen is posited to have a different risk perception towards technology adoption. Extensive literature is available today on each of the four vectors of posited risk. In addition, some work has already been done in looking at the risk‐gain profile of different industry sectors that are poised to move onto cloud platforms. Before an inter‐sector risk profiling is undertaken, a ringside literature survey is provided.

2. Literature survey Michael. Ambrust et al (2010) refer to Cloud Computing as a symbiosis of applications delivered as services over internet coupled with the hardware/systems software in the datacenters that provide those services. A paper which has received wide citation, this Berkeley work delineates the roles of the pure cloud provider and the intermediary SaaS model packager. Brian Gammage et al (2009) talk about the strategic possibility of the ‘power of IT’ shifting towards external providers and users. The paper, which is essentially a Gartner report, tries to clearly define core vs. non‐core strategies in the context of IT asset ownership and utilization. Jeanne Capachin (2012) researches and reports on the slow yet steady cloud computing adoption by the banking sector. The paper covers contract management and regulations management in the context of key financial data moving to the public cloud. It has got a predominant US banking sector perspective. Paul L Bannerman (2010) has done an exhaustive comparative survey of all cloud research papers published between 2009 and 2010 and has come out with a comparative analysis of what are the various adoption risks – real and perceptional. The paper discusses various barriers to cloud adoption by reviewing opinions of industry commentators. Sarfraz Nawaz Brohi et al (2011) compare the challenges as well as the benefits in what they call the new Paradigm – Cloud Computing. Chinyao Low et al (2011) investigates the factors that affect cloud adoption by firms belonging to the high‐tech industry. Malden A Vouk (2008) maps the journey of cloud from technology to implementation. Easwar et al (yet to be published in 2013) looks at the drivers and inhibitors of cloud adoption with a specific SME sector perspective. The data in this work is Indian SME data. Tara S Behrend et al (2011) examines cloud computing initiatives in the education sector. The paper is in the US context and examines the factors that lead to adoption of this technology from the perspective of both colleges and student community. Vladimir Vujin (2011) looks at the education industry and cloud computing, but more from a research support point of view. The paper talks about a reliable and scalable cloud environment that can foster scientific research and educational progress. Alec Nacamuli (2010) in what is essentially an editorial piece stresses on the importance of cloud in banking in the days to come. The paper cites that regulation, data recovery, customer trust and innovation would be some of the key thrust areas which come in the cusp of cloud computing and banking. Jeanne Capachin (2010) in another well researched article on banking focusses primarily on security issues that would be on top of rd mind for bankers when they think of 3 party data storage. Chris Chatman (2010) focusses on another sector which has a clear cloud adoption focus – health care sector. The paper focuses on the dual concerns of data security as well as speed of implementation for the healthcare sector. Edward J Giniat (2011) offers more insights on cloud vs. healthcare. Finally Easwar Krishna Iyer et al (2012) studies the Net Present Value (NPV) behavior for full vs. fractional adoption of cloud. The study looks at the unknown fears of cloud adoption which stretches across dimensions like security, privacy, variability, redundancy, down time, contract breach management and the likes and develops a mathematical model to monetize these risks. To sum up, there is a body of literature available today which individually talks about cloud adoption in the context of SME, BFS, Education and Hospitals. This paper proceeds to do a comparative risk profiling of cloud adoption across all these 4 sectors.

3. Problem formulation and research methodology Cloud – as a technology – has already transformed itself from a mere hype to an implementable reality in the last few years. Enlightened technology maturation and acceptance by the pragmatic ‘early majority’ users are the next logical steps towards an all‐embracing acceptance of this ‘pay‐as‐you‐use’ business model. Technology vendors are keenly working on solutions like ease of deployment, interoperability, server consolidation, economics of deployment and level of customization possible in their efforts to demystify the ethereal cloud. All these activities are actually happening today on the product front. The authors of this work

146

Easwar Krishna Iyer et al. feel that there should be a lot of simultaneous research thrust in positioning the cloud offering to potential enterprise level buyers. There has to be aggregate studies on resistance points to cloud adoption. The research methodology that is proposed here is a step in that direction. As previously mentioned, there is a clear inertia at the client end in going for a full‐fledged acceptance of cloud computing. This inertia is partly driven by the fact that many client industries have already invested heavily in technology and are reluctant to even partially abandon their on‐premise traditional IT infrastructure. For a technology intensive company, the IT infrastructure investment would be a significant portion of their assets in rd the balance sheet. Any move to adopt a ‘pay and use’ type 3 party platform would have ramifications on sunken assets and depreciation of these assets. In addition to capital investment, organizations currently running their IT in the traditional non‐cloud mode have invested deeply in people and processes to run the well‐oiled in‐house IT infrastructure. Any significant cloud adoption will have to take into account a major people and process transformation. Another reason for the slack in adoption is that despite the buildup on cloud over the last few years, the ‘real’ gains from the cloud offering look hazy to the buyer. To compound the scenario, the risks of full‐fledged cloud adoption are yet to be fully quantified. With main stream adoption for cloud predicted to happen in the next 2 to 5 years, this is the time for the cloud vendor industry to introspect on how their new offering will be perceived by the customer. The problem formulation of this paper is a step in the direction of sectorial risk profiling and assessment of cloud adoption. The paper posits that different sectors will have different relative risk perceptions and a correct assessment of the same will go a long way in tailoring custom‐made cloud solutions for each sector. Incidentally, the previous subsection on literature survey quotes at least one paper which has done a significant cloud adoption study in each of the 4 sectors that this paper is working on. As mentioned in the abstract, the four broad risk drivers for this study are vendor related risk, security related risk, no‐gain risk and efficiency related risk. They have been arrived at by collating from literature the risk studies that have been previously done. Subsequently, discussions with industry experts are used to condense the risk mapping along the aforementioned four vectors. Once the four key vectors have been identified, they 4 are paired in all possible combinations of twos, thus yielding C2 combinations; i.e. 6 combinations of risk. The six combinations are vendor + security, vendor + no gains, vendor + efficiency, security + no gains, security + efficiency and finally no gains + efficiency. The respondents are asked to distribute their relative risk perception weightages across the six pairs in such a way that the aggregate weightage comes to 100. In this trade‐off scenario, the respondents are forced to streamline their relative perception of risks. In reality, risks do not come in ones, they occur together. Hence, seeking risk weightages at a one‐on‐one level would have ‘disjointed’ the risks in the respondent’s mind. The process of pairing them is to enable the respondent to think in ‘conjoint’ terms before going for relative risk weightage assignment. 4 The C2 possible combinations can be represented in binomial terms as 1100, 1010, 1001, 0110, 0101 and 0011. The 1s and 0s are basically dummy variables which indicate the simple presence / absence of the risk(s) under study. A simple conjoint regression is done between the respondent risk perceptions and the binomial combinations. The part worth of each risk is derived from the regression output. In all, 5 regressions are run ‐ 4 individual regressions for each sectors and a common regression for all respondents to get the mean risk across all sectors. All the 5 regressions yield an R2 value of greater than 0.5 with education sector giving the highest value of 0.815 and SME giving lowest value of 0.574. The aggregate ‘all‐responses’ R2 value is 0.639. Care has been taken to ensure that there are at least 30 respondents from each sector. The study was eventually done on 150 respondents spread over SME (55), Hospitals (35), Education (30) and BFS (30). The entire sample consists of ex‐ante users and hence the picture that emerges is purely one of risk perception and no way reflects a post‐buy usage driven feedback. The questionnaire itself ensures that at the point of submission, the total risk weightage is exactly 100 across the six combinations of risk. Else, submission is not permitted. This ensures data consistency.

4. Analysis and interpretation The relative perception spread of the 4 risks, as given by the respective part worth utility functions, is plotted for all the four sectors in Figure 1. Inferences on this behavior based on the results obtained are given on a sector by sector basis.

147

Easwar Krishna Iyer et al.

Figure 1: Relative risk assessment of cloud adoption: A 4 risk x 4 sector mapping

5. SME sector The SME sector’s risk perception spread indicates that vendor related fears dominate the mindscape of the SME market. Since Cloud – as an offering ‐ is new and there are multiple layers of cloud vendors (ibid Michael. Ambrust et al), there is an ambiguity on who is one’s actual vendor. At one layer we have the cloud utility vendor who provides computing and storage at the hardware level. At another layer, we have the cloud application vendor who packages / bundles applications like CRM and ERP along with pure play hardware. What the user gets is an amalgam of these two, vended out to him by the intermediary SaaS (Software as a Service) provider. Thus there is a perceived ambiguity of escalation point for the user when he needs trouble shooting. It is this dilemma which has given vendor related risk the highest risk ranking. The SME respondents themselves are proactive / reactive problem solvers in their domain. Hence, they instinctively understand the value of trouble shooting. The respondents currently perceive an inability to identify a feedback path by which they can solve their cloud adoption problems. This opinion, which was obtained from random chats with SME respondents after they have filled up the questionnaire, can be posited as one of the reasons for the sector giving highest weightage for vendor risk. The second risk is more easily explained. Data security, data privacy and its associated losses would weigh across all adoption classes and literature quotes the same as the key deterrent for wide spread cloud adoption. Since both vendor risk and security risk can be perceived even before adoption, they garner the top two perceived risk spots. The risk on ‘Efficiency’ will be felt only post adoption and hence it gets relegated to the rd 3 spot. The inability to sense efficiency‐related risk before adoption is not generic across sectors. This will be explained in the context of the next two sectors. Coming to the last perceived risk, the SME respondent market is apparently completely sold on the monetary gain aspect of Cloud Computing. Hence their risk perception of a ‘No Gain’ is just 13%.

6. BFS sector The security risk is obviously the most overwhelming risk for the data sensitive banking sector. Of all the 16 part worth functions that have been arrived at (spread across 4 sectors x 4 risks), there is only one part worth function which has got a value in excess of 50%. This is for Security Risk in the context of BFS sector. Fears of data loss, privacy invasion, confidentiality loss, account hacking and the likes which can critically hamper banking operations add up to ensure that security risk gets a relative weightage of a whopping 61%. Banking – like any other sector in the verge of cloud adoption – definitely stands to gain from the classic cloud benefits of elasticity of usage, granularity of scaling and flexibility in pricing. Yet, this study indicates that the downside due to security fears override logical upside advantages. nd Coming to the 2 ranked risk, efficiency considerations and ‘cost’ of inefficiency are far more visible in banking sector than in a generic SME sector. BFS industry, which universally follows the norm of daily accounts closing

148

Easwar Krishna Iyer et al. and a day to day transaction reconciliation format, recognizes ex‐ante the value of efficiency. In a different context, all online financial markets like stock markets, commodity markets, FOREX markets and call money markets require minimal down time and ‘zero’ latency as hygiene factors. Latency in cloud can be introduced by cloud elements and internet elements. In a global online financial market, where snap shot execution of trade decisions are critical, lack of efficiency will be a significant deterrent. It is for these reasons that BFS gives efficiency related risk the second highest weightage. BFS is a very organized sector and its vendor calls are taken at national level. Given the scale of operations, there will be tight Service Level Agreements (SLAs) to ensure smooth operations. Hence the sector doesn’t th perceive any significant vendor related fears. The 4 fear of ‘no gain’ wasn’t even statistically significant for this sector. This was the only sector for which a ‘statistically insignificant’ risk was obtained. The conjoint regression was re‐run after dropping the ‘no gain’ risk to get the relative mapping of the remaining three risks.

7. Hospital / health sector Health Sector and BFS sector show a remarkable similarity in their perception towards all 4 risks. Since both the sectors are part of the organized sector and remain well informed, their vendor related risk perceptions are low. Hospitals are as wary about data confidentiality as banks. Leakage of patient information can make this sector morally and legally tenable. Today most hospitals have moved to a paperless health record format. Loss, temporary loss, swap or leakage of patient’s medical data can be calumnious to the industry. Thus, like banks, data security takes the top slot for Health sector also. The important of availability of right data at the right time to the right medical person cannot be overstated in the context of health. The possibility of a network / system failure in between a mission critical situation such as a time sensitive surgery might be highly detrimental to the actual outcome of the surgery itself. The very credibility of the medical fraternity is hinged on availability of dynamically updated patient data. Hence, efficiency risk comes a close # 2 to data security for health sector. Here again, as in BFS, there is a clear ex‐ante judgement of efficiency losses. It is worth noting that amongst the 4 sectors, the health / hospital sector records the highest risk perception of efficiency for the reasons cited above. The no gain risk is the least perceived, a rank that it consistently maintains across all 4 sectors.

8. Education sector The education sector’s high pitching of vendor risk could be driven by the fact that in this sector, students are also cloud users [unlike say in a hospital sector where the patients are not exposed to the cloud environment]. In a university, the student population would run into thousands and hence the load variability on cloud usage would be high. This creates problems with provisioning. Under provisioning of cloud services would result in service outages when the load peaks up. In education sector, there would be daily as well as seasonal peaking of load. In the Indian context, post‐dinner hours would be heavy usage hours if the system is configured in such a way that academic material has to be downloaded from the cloud platform. Examination, placement and admissions time would create seasonal spikes in usage. To take care of such a load variation, if the university goes for over‐provisioning, it would lead to capacity under‐utilization during non‐peak hours. This scenario actually translates to a ‘vendor fear’ for the ex‐ante market. We have respondents talking to us about vendor unpredictability in the context of what essentially is a provisioning driven system outage. This fear – like many other cloud adoption related fears – is purely perceptional. Yet, it exists in the potential customer’s mind space as the study suggests. After vendor risk, security and efficiency risks are almost even for education sector. As earlier mentioned, no sector has got a significant ‘no gains’ fear. This can be attributed to the fact that the cloud vendor industry has really ‘sold’ the CAPEX and OPEX gains of cloud computing to the aspiring adopter markets. As mentioned earlier, 5 independent regressions were done – the first four for the 4 independent sectors and th the 5 across all 150 respondents. The relative risk perception of each sector vis‐à‐vis the mean risk of the total sample population is plotted in Figure 2.

149

Easwar Krishna Iyer et al.

Figure 2: Relative sectorial risk vis‐a‐vis the all sector mean risk It can be observed that the BFS sector and the Hospital sector tend to move together for all the risk vectors. Similarly the SME sector and Education sector show some similarity in risk mapping behavior. This inter‐pair correlation is mapped in the matrix given in Figure 3. It is worth noting that except for the pairs mentioned above, the risk perception correlations are not high for any other pair of industry sectors.

Figure 3: Correlation matrix of inter‐sector risk mapping By way of a conclusion, the authors feel that this study will crucially help cloud service vendors to segment their positioning strategy after they understand the key inhibitors of cloud adoption for different markets. Cloud marketing being predominantly B2B, brochures, mailers, ads in sector specific magazines and other promotional campaigns can be tweaked at the last delivery mile to reduce fears and increase acceptance of this nascent technology. Though the authors have fragmented the total risk along 4 vectors only, in reality, each of these risk vectors consist of fairly uncorrelated and independent sub vectors. Fear of service continuity, fear of reputation fate sharing and risk of mismatch between vendor architecture and client business needs are all part of the broad umbrella of vendor risk. Same is the case with privacy, confidentiality and lack of control in the context of data security. A detailed relative analysis of these sub‐risks is missing in this study. That can be construed as one of the limitations of this study at this point. Cloud as an alternative is here to stay and the business gains of cloud adoption in terms of better Net Present Value is measurable. Cloud changes the paradigm from ownership of technology to utilization of technology. The authors of this paper feel that customer‐centered research will facilitate good product as well as positioning strategies in such a way that cloud adoption fears are played down and gains are ushered in.

150

Easwar Krishna Iyer et al.

9. Future directions of research The papers that are quoted from literature offer perspectives of cloud adoption risk only on a sector by sector basis. This paper attempts to capture a reasonably good framework of inter‐sector relative risk comparison. But as mentioned in the previous section, this work has not explored the sub‐risks that reside within each risk category. A segmented and weighted study of each sub‐category of risk will enable the monetization of all risk possibilities. Such a monetary association of risk will go a long way in delineating the actual Net Present Value gains of Cloud Computing (ibid Easwar Krishna Iyer et all 2012). Some of the authors of this paper are currently working on such a study. The second thrust in research would be a country specific research approach. The risk perspectives offered in this paper are highly India centric. The relative risk perception scenario could play out differently in another country. Again, some of the authors of this work are currently working on cloud adoption risk measurement in some neighboring emerging economies.

References Armbrust Michael et al (2010) “A View of Cloud Computing”, Communications of the ACM, Vol. 53, Issue No. 4, April 2010 Bannerman Paul L (2010), “Cloud Computing Adoption Risks: State of Play”, Asia Pacific Software Engineering Conference (APSEC 2010), Cloud Workshop – Nov 2010 Behrend Tara S et al (2011), “Cloud Computing adoption and usage in community colleges”, Behavior and Information Technology, Vol. 30, No. 2, March – April 2011 Capachin Jeanne (2010),”Change on the horizon: The impact of Cloud Computing on Treasury and Transaction Banking”, Journal of Payments Strategy & Systems, Vol. 4, No. 4, 2010 Capachin Jeanne (2012), “How to Manage Vendors in the Public Cloud”, American Banker, Vol. 177, Issue 156, October 2012 Chatman Chris (2010), “How Cloud Computing is changing the face of Health Care Information Technology”, Journal of Healthcare Compliance, May‐June 2010 Gammage Brian et al (2009), “Gartner’s Top Predictors for IT Organizations and Users, 2010 and Beyond: A New Balance”, A Gartner Research Report, ID no. G00173482, December 2009 Giniat Edward J (2011), “Cloud Computing: Innovating the Business of Health Care”, Healthcare Financial Management, May 2011 Krishna Iyer Easwar et al (2012), “Cloud Computing and Modelling of Cash Flows for Full vs. Fractional Adoption of Cloud”, Proceedings of the International Conference on Business Management and Information Systems (ICBMIS), November 2012 Krishna Iyer Easwar et al (2013), “Decision variables influencing Cloud adoption by SME sector: A Conjoint Analysis Mapping”, National Conference on Business Analytics and Business Intelligence, January 2013 Low Chinyao, Chen Yahsueh (2011), “Understanding the determinants of cloud computing adoption”, Industrial Management and Data Systems, Vol. 111, Issue 7, 2011 Nacamuli Alec (2010),”An Editorial on Cloud Adoption in BFS”, Journal of Payments Strategy & Systems, Vol. 4, No. 4, 2010 Nawaz Brohi Sarfraz, Adib Bamiah Mervat (2011), “Challenges and Benefits for Adopting the Paradigm of Cloud Computing”, International Journal of Advanced Engineering Sciences and Technologies, Vol. 8 Issue No. 2, 2011 Vouk Mladen A. (2008), “Cloud Computing – Issues, Research and Implementations”, Journal of Computing and Information Technology, CIT 16, Vol 4, 2008 Vujin Vladimir (2011), “Cloud Computing in Science and Higher Education”, Management, Issue No. 59, 2011

151

Explaining the Adoption of Complex Innovations by Reflexive Agents — A Critical Realist Perspective Marius Mihailescu, Daniela Mihailescu and Sven Carlsson Informatics, School of Economics and Management, Lund University, Sweden marius.mihailescu@jibs.hj.se daniela.mihailescu@jibs.hj.se sven.carlsson@ics.lu.se Abstract: The adoption of innovation is a multifaceted and dynamic phenomenon. It occurs as a result of the interplay between structural influences and agents’ activities. Although existing studies on innovation have recognised the importance of theories that link the structure, the macro level, and agency, the micro level, in explaining changes over time, few theoretical accounts support the integrations of multiple levels of analysis. The purpose of this paper is to develop an explanatory framework based on a realist social theory and underpinned by a critical realist perspective, with the intention of addressing the difficulties that arise from efforts to integrate the macro‐micro levels of analysis. The potential of the framework is empirically illustrated with a case study that examines the adoption of one Enterprise Systems Implementation Methodology. Our qualitative study provides explanatory insights and a rich description of a particular type of complex innovation. Four theoretically and empirically grounded methods of adopting an implementation methodology are identified: selective, aggregated, integrated and infrastructural. The variations in the adoption suggest an evolutionary change in this particular case. Using the framework allow us to achieve four things. First, the framework will support the researchers in identifying particular innovation configurations. Second, it will take into account the embeddedness of innovations that have occurred within broader structural configurations. Third, it will allow the researchers to distinguish the different stances agents might adopt toward particular innovations and structural configurations. Fourth, the researchers will be able to identify variations that have occurred in the adoption of innovations. This study offers a foundation for future work that may contribute to a more coherent view on complex innovations and insights into their potential adoption; as such, the findings presented here can provide guidance for practitioners who seek to adopt complex IS/IT innovations.

Keywords: IS/IT adoption, enterprise systems implementation methodology, realist social theory, critical realism

1. Introduction IS/IT adoption represents a mature research stream within the IS field. Despite an extensive body of research, progress has occurred mainly at either an individual‐ or an organizational level, with less work devoted to linking the two levels (Sarker and Valacich 2010). Moreover, there has been gradual movement toward homogeneity of a type that might weaken innovation adoption research, a field that is dominated by the positivist paradigm (Fichman 2004), and to a lesser extent of the interpretive paradigm (Williams, Dwivedi, Lal and Schwarz 2009). Several authors have suggested alternative paradigms and theories that might lead to a deeper understanding and alternative explanation of the IS/IT adoption (Williams et al. 2009). In line with these recommendations, we suggest an alternative approach that will provide valuable insights and explanatory purchase in the face of IS/IT adoption phenomena. The objective of this paper is to describe and provide a potential explanation for the adoption of an Enterprise Systems Implementation Methodology (ESIM), which we regard as a type of complex innovation technology, by applying a Realist Social Theory (RST) that is underpinned by a Critical Realist (CR) perspective. The paper continues with a presentation of current trends in IS/IT adoption research, followed by a brief description of the CR perspective and RST, which is applied as a way to conceptualize IS/IT adoption. A framework is then developed and applied in order to explain the adoption of AcceleratedSAP, an example of an ESIM, in an ES‐implementation context. The paper concludes with a discussion of the implications of the findings and some promising avenues for future research.

2. IS/IT innovation adoption There is a rich diversity in the research streams and theories available within the body of IS/IT adoption literature. Focusing on the impact of technical and contextual variables on technology adoption, positivism takes a deterministic view that IS/IT adoption and its outcomes can be objectively described and predicted. Empirical findings show unpredictable effects, however, which often cannot be explained from a deterministic viewpoint (Venkatesh, Thong and XU 2012). Unlike those scholars, interpretivists attempted to understand

152

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson IS/IT adoption as it is experienced by actors through their meanings and frames of reference. Due to their reliance on empirical epistemology, the resultant descriptions and explanations of IS/IT adoption have been confined to observable regularities that provide a one‐dimensional view of the phenomenon. Within the positivist approach, the variance models have accorded contextual factors a deterministic influence on the regular occurrence of IS/IT adoption; within the interpretivist approach, the process models have accorded importance to social actors, who differ from one another by virtue of their knowledge and skills. In addition to such one‐dimensional views, these models fall short as well in explaining changes in the occurrence of IS/IT adoption. Moreover, these two paradigms lead to theory‐practice inconsistencies (Smith 2006), and introduce a problematic notion of causality, as they are based on Humean assumptions of regular cause‐effect relationships within positivism and interpretations of meanings and idiographic phenomena within interpretivism (Sayer 2000). Critical realism therefore has been suggested as a potential means of overcoming these inconsistencies (Smith 2006) and enriching IS research (Carlsson 2012).

3. Theoretical framework The insights garnered from a review of the extant literature have yielded a preliminary understanding of the potential issues that might be overcome by adopting a CR perspective. This section provides a brief introduction to CR and to RST.

3.1 Critical realism perspective The central tenets of CR, as developed by Bhaskar (1986) and expanded upon by several authors (e.g., Archer, Bhaskar, Collier, Lawson and Norrie 1998; Sayer 1992), are: stratification, causal explanation, differentiation and emergence. The notion of stratification draws attention to the ontology of CR, which regards reality as being stratified into three separable but interrelated domains: the real, the actual and the empirical (Figure 1).

Figure 1: The three overlapping domains of reality in CR ontology (Mingers 2004) The stratified ontology directs the critical realist toward an explanatory focus on the domain of the real, where unobservable mechanisms generate observable phenomena in the actual and empirical domains. Causal explanation concerns the identification of causal mechanisms that reside in the domain of the real and generate events in the domain of the actual. Although events in the domain of the actual might be unobservable, they are nevertheless experienced. Experiences of the events that populate the domain of the empirical are observable but represent only the starting point of inquiries intended to result in causal explanations. The notion of differentiation emphasises an analytical and temporal distinction between structure and agency since the former is dependent on the activity of the latter ‐ i.e., structure pre‐dates agency. Stratification and differentiation are related to emergence, which “derives from interaction and its consequences which occur in time” (Archer 1995:14). Operative in open systems, emergence is embedded in interaction when structure and agency shape and re‐shape one another over time (Archer 2010).

153

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson

3.2 Realist social theory The tenets of CR underpin RST, which is based on the notions of analytical dualism, as a theorizing tool and a foundation for social analysis; morphogenesis, as an explanatory framework for social change; and, reflexivity, which provides explanatory purchase for agents’ distinctive stances toward their social circumstances. In order to theorize the interplay between structure and agency, Archer (1995) has suggested the notion of analytical dualism, that emphasise the fact that structure and agency are interdependent but analytically separable and temporally sequenced. In addition to an analytical separation, Archer (1995) also has suggested a separation between structure and culture. Both provide insights into the social context that depends on, but cannot be reduced to, agency ‐ i.e., to individuals and their activity. Structure and culture on the one hand, and agency on the other, therefore are not only two separate strata but are also understood to be temporally differentiated. Furthermore, structure and culture are held to have existed prior and to have exerted a causal influence upon agency ‐ i.e., to have enabled or constrained human activity. Agency, which mediates the influence of structure and culture on individuals, has the power to reproduce or transform structure and culture, but it is the individual who possesses the relevant powers that enable agency ‐ e.g., reflexivity, self‐ consciousness and intentionality (Archer 1995). In this study, analytical dualism informs the interpretation of IS/IT adoption as a concrete phenomenon regarded as a manifestation of the interplay between a social context (structure, culture and groups of agents) and human activity (agents’ stances and interactions). The means by which we have operationalized analytical dualism are represented by: morphogenesis, a cycle that consists of the structural/cultural conditioning of groups; interaction; and, a structural/cultural and group elaboration phase (Figure 2).

Figure 2: The basic morphogenetic cycle (Archer 1995) In the first phase, at Time 1 (T1), the cycle begins with and is informed by the structural‐cultural properties available from previous interactions to have occurred in the social context. According to Archer (1995), these are emergent properties and represent relationships which entail: a) material resources such as wealth, power or expertise with regard to structure; and, b) ideational sources such as doctrines, theories or beliefs, with reference to culture. The distribution of material resources and the composition of ideational sources pre‐ groups agents into collectives with under‐privileged positions as primary agents and organised groups with privileged positions as corporate agents. Primary agents are mobilised to advance the interests of corporate agents, who have emergent powers that they exercise in promoting and articulating their vested interests in maintaining or improving their positions. Relationships of compatibility or incompatibility, and necessary or contingent create different situational logics. The four situational logics that provide directional guidance and motivate agents to maintain or alter the status quo are: correction, protection, elimination and opportunism. In the second phase, T2‐T3, the focus is on agency and concerns the interaction and emergent properties of peoples possessing reflexive power. They act by virtue of their subjectively defined concerns with regard to three orders of reality: a) well‐being concerns with regard to natural order; b) performative achievements with regard to practical order; and c) self‐worth with regard to social order. Representing what they most care about, the concerns are organically prioritized and are realized through projects that are adjusted to reflect the objective circumstances created by previous cycles The interplay between people’s configurations of concerns i.e., what they most care about and their contexts generates four distinctive modes of reflexivity and related stances: 1) evasive, as in the case of communicative reflexivity; 2) strategic, as in the case of autonomous reflexivity; 3) subversive, as in the case of meta‐reflexivity; and 4) passive, as in the case of fractured reflexivity (Archer 2007). In the third phase, T4, interaction ensues in the form of structural‐cultural and group elaboration i.e., reproduction (morphostatis) or transformation (morphogenesis). As a result of exchange and power

154

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson transactions, agents undergo: a) re‐grouping, as of initial ideational sources ‐ i.e., propositions, theories, doctrines; and b) re‐constitution of material resources ‐ i.e., wealth, power, expertise are re‐distributed during morphogenesis. Due to differentiated resources, the transactions between agents are realized in terms of power, while evenly distributed resources involve exchange transactions. The results represent the start and structural conditioning of the next morphogenetic cycle, at T1.

3.3 A realist conceptualization of IS/IT adoption Informed by RST, IS/IT adoption is regarded as a manifestation of the interplay between situational logics and stances of interacting agents over time. Figure 3 illustrates a realist conceptualization of it.

Figure 3: A realist conceptualization of IS/IT adoption Working within this framework we regard IS/IT adoption occurrences as having been generated by the interplay between situational logics, with enabled or constrained powers, and agents’ stances, with maintained or altered powers. The morphogenetic cycle provides us the means to examine and provide potential explanations of IS/IT adoption occurrences and their variations over time.

4. ESIM adoption – the case of ASAP The retrospective case reported in this study concerns ASAP, an ESIM adopted by the SAP vendor. The study adopts a qualitative approach, which subsumes a combination of qualitative data collection techniques, documents and semi‐structured interviews. A retrospective analysis is conducted in order to explain the adoption of an ESIM from early 1980s until 2004 through four morphogenetic cycles. Each cycle depicts a particular adoption occurrence as the interplay between the situational logic and the implementer’s stance. The examination of these occurrences follows the conditions, interplay and outcomes phases of the morphogenetic cycle. Four different adoption occurrences: selective, aggregated, integrated and infrastructural, have been identified and are described bellow.

4.1 Selective adoption — early 1980s–1987 Conditionings: Available on the market as early as the 1970s, “off‐the‐shelf” or generic software application suites able to be customized to better meet the buyer’s needs began to attract increased interest among customers in the period beginning in the early 1980s. One of the few suppliers of generic software applications and contributors to the growth of the software industry was SAP AG. By the early 1980s the company worked to develop a novel system application architecture. The understanding of modular software was rather different from the prevailing view on IS development and the use of technology to support and automate an organization’s existing processes.

155

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson Interplay: Situational logic: By the early 1980s, there had been an explosion of corrective repairs initiated via the reinterpretation of IS development methodologies and system development processes in an attempt to resolve the inconsistencies between them. The majority of developers were engaged in corrective efforts directed toward system development content and system development processes. The few organizations that followed the path of Enterprise Systems (ES), such as modular software providers, integrated general guidelines and provided training congruent with the principles and assumptions that underpinned their ES solutions. Shaped by the principle of technology‐enabled Business Process Reengineering (BPR), and a configuration of ES software based on a vanilla implementation strategy, an ESIM began to take shape as a complementary element of the ES implementation process and broader ES solutions. The corrective ideational repairs fostered by ES providers challenged the dominant IS doctrine on the IS market and in IS education, and coincided with a protective integration of ES operations and roles. Implementers’ stance: Although continuously increasing, the number of ES vendors who were both developers and implementers of ES solutions remained limited. Due to fractured reflexivity, delineated by undetermined concerns and disconcerted experiences, ES vendors were passive in their orientation toward ES implementation practice and were reorienting their concerns. They did attract the interest of an increasing number of customers, however, many of whom were seeking to increase the legitimacy of their syncretic endeavours and to improve their market position. They attempted to advance new ideas, but also sought to reuse their expertise in IS development, augmenting their distress in this way and precluding the formation and pursuit of purposive ES implementation activities. The difficulties encountered in articulating and prioritising their ultimate concerns in relation to ES implementation confined ES providers to passivity. Outcomes: The effects of selective adoption can be seen in the limited amount of ESIM content which carried informative potential. Due to limited contributions from passive ES providers in a corrective ES implementation context, the establishment of ESIM was slow; it was, however, complementary to the ES assumptions and to a technical implementation process. The ES implementation context was morphostatic, with passive ES providers whose interests were served by subordinated collectives and an increasing number of customers. As the exploration and integration of operations intensified, the ability to make resilient differences, undermined by a pronounced dependence on subordinated collectives and the need to satisfy customers’ interests, emerged as an unintended side‐effect.

4.2 Aggregated adoption — 1988–1992 Conditionings: By the late 1980s, SAP had seen expansion of its operations and customer base internationally. Established in 1988, SAP America, established itself as the company’s most important subsidiary, focused exclusively on providing complementary services to its partners. The service partners in turn provided consulting, implementation and support services to customers. The relationship between the vendor and its service partners was beneficial for both. SAP was able to devote added focus to product development and installation or customization, with less attention needing to be paid to implementation services; in SAP America’s case, implementation was managed exclusively by its independent service partners. Aside from knowledge of SAP products, the service partners contributed to implementation projects with skills accumulated as a result of having worked on previous projects and know‐how acquired through the process of having developed their own methodologies. These methodologies were specialized to address particular aspects of ES implementation, however, and differed with regard to integration, scope, content and efficiency across individual SAP implementation projects. Despite this variety of complementary services and expertise, implementation was hardly without challenges. Both partners and customers encountered operational obstructions and practical problems. Interplay: Situational logic: Exploding demand for ES solutions and for ES implementation resources, which were limited in terms of the expertise available to the ES vendors in particular, encouraged an increasing number of consulting companies and ES vendors to join together. Due to their differentiated distribution of resources but complementary areas of expertise, defensive alliances were initiated that shared a common interest in legitimizing ES ideas through defensive protection. There was mutual recognition of benefits between the ES

156

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson vendor and its implementation partners, but both actors also retained sectional interests in their own operations and in being able to offer diversified services. Part of their protective efforts entailed the reuse of the available but limited ESIM contents and the adding of their own content ‐ e.g., project management and BPR services, and reproducing their specialized expertise. Implementers’ stance: Differentiated partners were able to mobilise and exchange material resources, particularly human assets and expertise. Due to communicative reflexivity, delineated by an overarching concern for the maintaining of concordant inter‐relationships, ES partners were evasive in their orientation toward the ES implementation context. They conceived of their operations as existing within and utilising available but differentiated resources. Their cooperation was based on shared interests and an active but circumventing response intended to avoid potential constraints due to a lack of or incongruent ES expertise in different implementation areas. They also simultaneously renounced potential opportunities such as those that would have allowed them to augment their positions and resources, however, instead taking a proactive role in differentiating themselves and their results in ES implementation. Contentment with their position insulated against external stimuli and sustained the reproduction and efficiency of their own operations and expertise. Outcomes: The effects of aggregated ESIM adoption were represented by horizontally extended ESIM content with traceable potential, albeit with the potential risk of inconsistencies. As the content relied on the rapid aggregation of complementary but limited ES elements enhanced by the service partners’ proprietary elements, which had their provenance in IS development, the content of ESIM presented an increased risk of inconsistencies. The ES implementation context was morphostatic, with intensified interaction protected by cooperative ES partners with shared interests in interrelationships. They took an evasive stance toward material and cultural differences, as well as toward rewards, and were actively involved in reproducing defensive ES implementation practices. The reproduction of a distributed organization with differentiated roles and the differentiated distribution of resources was reinforced by systematised ES implementation ideas; such ideas were in turn legitimised by an increased number of customers who consolidated the position of ES partners with sectional interests in ES implementation. The selective assimilation of novelty and the reduced variety among implementation partners emerged as unintended side‐effects.

4.3 Integrated adoption — 1993–1995 Conditionings: SAP grew rapidly and, by the early 1990s, had introduced a new generic, integrated and modular application suite, called R/3. Although the application was to some degree adaptable and customizable, the underlying ideas were to reduce the potential problems relating to the development cycle and/or to enable BPR. The expertise necessary to implement the new product, however, was shallow. In addition, service partners used their own implementation methodologies, which often were better suited to system development than to the task of implementing R/3. After the release of the R/3 application in 1992, however, SAP America’s product registered an impressive increase in sales, even though the actual implementation of the product remained a challenge. In order to tackle this challenge, SAP America expanded its cooperation with its partners and consultants and began joint work on SAP implementations. Supported by SAP America’s service division, a team of consultants with past experience having participated in joint SAP implementations were assigned to develop an implementation methodology. The implementation methodology emerged progressively from customer feedback and experiences accumulated during collaborative pilot implementations, and by assembling ideas and theories from software engineering, project management and organization development. Interplay: Situation logic: ES implementation partners joined together on their own terms and were operationally autonomous. Guided by their own unique interests, they pursued the opportunity to differentiate their operations and services. Alternative ES implementation strategies, as well as specialized ideas about the role of ES in organizations and about ES implementation processes began to gain support from among the different groups of ES implementers and customers. Although an ES implementation based on a vanilla strategy entailed the configuration of ES software, for instance, the alternative, “chocolate”, strategy involved considerable customisation of the ES software. Mobilised by the ES vendor, a team of implementation partners extended

157

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson the ideas underlying ES and the implementation process and exposed complementary and congruent ESIM contents. Specialized but complementary operations and services retained salience in practice. Implementers’ stance: Committed to accumulating and integrating differentiated resources in a systematic fashion, ES partners joined together in their efforts. Due to meta‐reflexivity as delineated by organic concerns in exposing best practices, ES partners were subversive in their orientation toward the state of ES implementation practice and concentrated on providing a specialised set of ideas and resources that were compatible with an ES implementation role. They collaborated and integrated diversified resources in an attempt to improve quality and raise value rationality in ES implementation practice. Their commitment to the integration and protection of quality in ES implementation practice came at a price, but long‐term achievements were assigned a greater value. Outcomes: The effects of integrated ESIM adoption have come to be represented by vertically extended ESIM content with the potential for predictability. The ES implementation practice at this stage was morphogenetic, with vocational ideals sustained by collaborative partners with shared interests in value rationality. They organised and adopted a subversive stance toward material differences as well as toward rewards, and were actively involved in a continuous search for value and quality in ES implementation practice. Transformation into a centralised organisation of differentiated roles, with differentiated distribution of diversified resources, was reinforced by specialized ES implementation ideas promoted by a new and well‐positioned ES implementation partner and legitimised by an increasing number of customers and ES implementation partners. A display of the alternative values that might animate the interests of passive and diversified partners has emerged as an unintended side‐effect.

4.4 Infrastructural adoption — 1996–2004 Conditionings: By 1996, the outcome of the initiative adopted by SAP America and the work of the team of consultants was represented by an implementation methodology that had been added to SAP’s product development portfolio and extended the complementary services SAP America was able to offer its customers. The implementation methodology was introduced to SAP’s partners and customers as AcceleratedSAP (ASAP), and was recommended as a de facto standard for all SAP implementations. In addition, the methodology was included in SAP’s professional services ‐ i.e., training and certifications, and was actively promoted. Moreover, a new role was added to all SAP implementation projects, namely, a SAP coach, whose level of involvement in the implementation project could range from that of project manager to quality assurance advisor. In 1999, ASAP was enhanced via the implementation of related support tools such as the SAP Solution Manager platform, which was integrated into all SAP installations free of charge and was thereafter owned by the customer. Moreover, the Solution Manager platform could be used during evaluation and implementation as well as in the normal operation of a SAP solution; the platform came to comprise part of SAP’s NetWeaver platform. Some of SAP’s implementation partners continued to cooperate with SAP in implementing R/3 solutions, but initiated the development and deployment of their own methodologies by extending and/or integrating elements of ASAP. Others deployed ASAP and collaborated with the vendor to enhance it. Positive results in terms of efficiency and effectiveness were indicated by SAP and its service partners after the introduction of ASAP. Moreover, SAP took a more active role in supporting customers’ implementation projects and required the involvement of an SAP representative in all projects. The initiative was met with mixed feelings, in some cases being well received and in others being regarded as an added burden imposed on the already hectic schedules of consultants who first needed to acquaint themselves with ASAP. It was also perceived as another training or resource hiring cost, or as a way to increase partners’ dependence on SAP’s products. Interplay: Situation logic: The passive acceptance and support of opportunistic efforts introduced by ES partners with sectional and diversified interests and differentiated resources were disadvantageous and affected customer satisfaction, performance during the implementation process and the quality of the ES solution. In order to show their commitment to its services and customers, the ES provider was required to take an active position in promoting complementary resources. A platform of diversified and sectional resources was introduced in ES implementation practice.

158

Marius Mihailescu, Daniela Mihailescu and Sven Carlsson Implementers’ stance: In concentrating on sustaining satisfactory implementation performance, ES partners were active in capitalising on availabilities and circumventing constraints. Due to autonomous reflexivity, defined by an overarching concern for proficiency and feasible performance achievements, ES implementation groups took a strategic stance toward ES implementation practice and accommodated specialised and differentiated resources within a comprehensive platform of services and resources. Aware of the limitations and benefits of ES implementation, the group harnessed the compliance of various resources and circumvented certain constraints by coordinating the sectional and diversified efforts of various ES partners. Outcomes: The effects of infrastructural ESIM adoption have been represented by comprehensive ESIM content with the potential for connectivity. The ES implementation is morphogenetic, with the innovative contributions advanced by ES partners coordinated by an ES implementation group with particular interests in task completion and productivity achievements. The ES implementation groups have adopted a strategic stance toward material diversification and cultural sectionalism and are actively mobilising and accommodating specialised and diversified resources. The intensification of sectional and divergent interests and the reduction of interaction among ES partners have emerged as unintended side‐effects.

5. Conclusion This study has provided an alternative conceptualisation of the IS/IT adoption phenomenon, one informed by a RST underpinned by a CR perspective. Despite its complexity, RST facilitates an emergent and stratified account of a complex innovation, thereby providing a potential explanation for the occurrence of IS/IT adoption and its variations over time. Four things have been achieved in applying the suggested framework: First, it has supported the researchers in identifying particular innovation configurations. Second, it has taken into account the embeddedness of innovations within broader structural configurations. Third, it has allowed the researchers to distinguish the different stances of agents toward particular innovations and structural configurations. Fourth, we have been able to identify variations in how certain innovations have been adopted; such variations have been seen to have often occurred as a result of the interplay between structure and agency over time. Our qualitative study has provided explanatory insights and a rich description of a particular type of complex innovation and its adoption over time. Four theoretically‐ and empirically‐grounded implementation methodologies with related causes and effects have been identified: selective, aggregated, integrated and infrastructural. The variations in the adoption of the implementation methodology over time suggest an evolutionary change in the adoption of this particular innovation and case. While there is much research still to be carried out on this subject, this study offers a foundation for future work that may contribute to a more coherent view of IS/IT adoption phenomena. More specifically, the next step will be to enhance the explanatory potential of the realist IS/IT adoption framework by formulating coherent design propositions that provide high‐level guidance for practitioners who find themselves in similar situations.

References Archer, M.S. (1995) Realist Social Theory: The Morphogenetic Approach, Cambridge University Press, Cambridge. Archer, M.S. (2007) Making Our Way through the World: Human Reflexivity and Social Mobility, Cambridge University Press, Cambridge. Archer, M.S. (2010) “Morphogenesis versus Structuration: On Combining Structure and Action”, The British Journal of Sociology, Vol. 61, No. S1, pp 225–252. Archer, M.S., Bhaskar, R., Collier, A., Lawson, T. and Norrie, A. (1998) Critical Realism: Essential Readings, Routledge, London. Bhaskar, R. (1989). Reclaiming Reality: A Critical Introduction to Contemporary Philosophy, Verso, London. Carlsson, S.A. (2012) The Potential of Critical Realism in IS Research, Y.K. Dwivedi, M.R. Wade and S.L. Schneberger (Eds.), Information Systems Theory: Explaining and Predicting Our Digital Society, Vol. 2, pp 281–304, Springer, New York. Fichman, R.G. (2004) “Going Beyond the Dominant Paradigm for Information Technology Innovation Research: Emerging Concepts and Methods”, Journal of the Association for Information Systems, Vol. 5, No. 8, pp 314–355. Mingers, J. (2004a) ‘Real‐izing information systems: critical realism as an underpinning philosophy for information systems’, Information and Organisation, Vol. 14, pp 87‐103. Sarker, S. and Valacich, J.S. (2010) “An Alternative to Methodological Individualism: A Non‐Reductionist Approach to Studying Technology Adoption by Groups”, MIS Quarterly, Vol. 34, No. 4, pp 779–808. Sayer, R.A. (2000) Realism and social science, Sage, London. Smith, M.L. (2006) “Overcoming Theory‐Practice Inconsistencies: Critical Realism and Information Systems Research”, Information and Organization, Vol. 16, No. 3, pp 191–211. Venkatesh, V., Thong, J.Y.L. and Xu, X. (2012) “Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology”, MIS Quarterly, Vol. 36, No. 1, pp 157–178. Williams, M.D., Dwivedi, Y.K., Lal, B. and Schwarz, A. (2009) “Contemporary Trends and Issues in IT Adoption and Diffusion Research”, Journal of Information Technology, Vol. 24, pp 1–10.

159

Adoption of Mobile Technology to Enhance Services at Academic Library Eunice Mtshali1 and Tiko Iyamu2 1 Tshwane University of Technology, Pretoria, South Africa 2 Polytechnic of Namibia, Department of Business Computing, Windhoek, Namibia mtshalie@tut.ac.za connectvilla@gmail.com Abstract: Many universities are struggling to respond to the needs of their users. This could be attributed to the rapid change in technological innovations. The growing interest on mobile technology in organisations is at a fast pace, particularly in institutions of higher learning. Mobile technology could be used in academic libraries to provide a better service to their clients or improve the service that they currently provide. Case study research was conducted at Capital University to understand the factors that could influence and impact the adoption of mobile technology in academic library services. Keywords: academic library, services, mobile technology, mobile devices, technology acceptance model

1. Introduction The adoption of mobile technology in organisation including institution of higher learning is rapidly increasing, more than ever in the last few years (Castells et al., 2007).The increase in adoption and usage of mobile technology has expanded challenges and opportunities for business, social and educational services. This rapid diffusion, particularly among students and teenagers presents numerous opportunities for business, social and educational services (Bicen & Cavus, 2010; Brown et al., 2003; Roblyer et al., 2010). The development and use of information and communications technology (ICT) in libraries has slowly emerge on which, there has been many successes and failures over the past years (Igben & Akobo, 2007). Developments in ICT have impacted all sectors of society, including the education system. In institutions of higher learning, the application of ICT is changing how teaching and learning are carried out in many ways. This includes academic and non‐academic activities and processes. In the past ten years, both public and academic libraries have begun implementing mobile versions of at least some elements of their web sites including catalogues; online journal articles (Bridges et al. 2010).Rapid developments in information and communications technologies (ICT) and evolving learner behaviours require learning institutions to continuously re‐evaluate their approaches to deliver service to their clients. The increasing availability of low‐cost mobile and wireless devices and associated infrastructure usher in both opportunities and challenges for educational institutions and their teachers and learners. Mobile learning and mobile technologies brings the rewards of placing institutions at the forefront and addresses student requirements for flexibility and ubiquity, that is, ‘anywhere, anytime (Clyde, 2004; Gay et al., 2001; Hill & Roldan, 2005; Liu et al., 2003), and any device’ learner engagement. The use of mobile technologies for service delivery is rapidly increasing. Many organisations including institutions of higher learning have in the recent years employed mobile technologies such as cellular phone, specifically, the short messages service (SMS), and multimedia messaging services (MMS) to deliver various services. Many institutions of high learning have adopted the E‐Learning and M‐Learning approaches to enhance flexibility and improve on students and research outputs. When considering the use of mobile technologies in academic library services it is not sufficient to recognize that a large number of library patrons use mobile phones. The particular type of device they use and the degree to which they are web‐enabled must also be considered. The range of device available can greatly impact the use and acceptance of mobile technologies in libraries (Bridges et al. 2010).Paying attention to the types of devices the users have can also help to design mobile services that best meet the user’s needs.

2. Research approach Qualitative research approach was selected mainly because of rationale such as its probing and close interaction nature. The qualitative research approach allows for clarification from respondents to questions

160

Eunice Mtshali and Tiko Iyamu from the researcher. Also, it allows for close interaction with interviewees, enabling the researcher to develop a deeper understanding of the situation. Also, the researcher can observe events and actions as they happen, without removing them from their context (Babbie & Mouton, 2001). The study adopted the case study method as its research design Creswell (2009) define case studies as a strategy of inquiry in which the researcher explores in depth a program, event, activity, process, or one or more individuals. Yin (2009) recommended case study inquiries for “how” and “why” types of research questions, where the researcher has little or no control, and the purpose is to investigate real‐life contexts, phenomena, and situations . Capital University was used as the study. The focus of the study was on the university’s library. Data was collected using semi‐structured interviews and library documentations. The interpretive approach was employed in the data analysis, using Technology Acceptance Model (TAM). The analysis exploited the social context in the adoption and use of mobile technology to enhance library services at the Capital University. TAM attempts to address why users accept or reject technology and how users’ acceptance is influenced by system characteristics. TAM assumes that people tend to use or not to use technology to the extent that they believe it will help them perform their jobs better.TAM consists of four main components, namely perceived ease of use, perceived usefulness, behavioural intention and actual system use. TAM components help to gain better understanding of the factors which influence and impact the adoption of mobile technology in the delivery of the library services.

3. Data analysis: TAM view The analysis was carried out through TAM constructs, which include Perceived Usefulness (PU) and Perceived Ease of Use (PEoU), behavioural intentions (BI) and actual system usage. Perceived ease of use In TAM, Perceived ease of use is regarded as process expectancy primarily because it focuses on a person’s subjective perception about the level of effort which is needed to use a computer system. The academic library of the Capital University provides services to the students and academics. Some of the services that the library provides include availing materials for teaching, learning and research. The availability of the materials is determined by its accessibility. The materials of the University library were accessed by students and academics through different means and channels. Some of these mean include the physical presence at the library premises, telephonic and email inquiry, and online via the library’s website through the electronic resource portal. The means and channels were dependent on the ease to do so according to the individual users. Academics had the option to search for books electronically from their offices but some of them often don’t. This is because they think that the electronic system was complicated and they claim that is easy for them to physically go to the library to ask the librarian for assistance. As expressed by one of the interviewees: “Several at times when I attempt to make use of the systems in my office, I find it challenging. It takes a lot of time because I think the system they use is not easy to use to get what I want. When I physically access the services, especially through the librarian assistance is quicker and easier because the librarians know where to find the information quicker”. However, majority of the users were keen and interested in exploring the library information technology systems in carrying out their personal and university academic obligations, which include teaching, learning and research. This group of users preferred the services because it was computer systems (information systems and technology) supported and enabled.

161

Eunice Mtshali and Tiko Iyamu The users who preferred computer based systems had different background and knowledge pertaining to information systems and technology. Their background and knowledge influenced their ability and interest on how they used current and new technologies which were deployed in the environment to provide them with the services that they required. One of the interviewees expressed herself as follow: “When I search for books I first check them on an online public access catalogue (OPAC), OPAC is a computer system we use to check for books, journals, and DVD”. Some of those users who preferred the computer based systems struggled with self‐service which was supported and enabled by the systems. This could be attributed to factors such as lack of training and systems’ knowledge. The users who prefer the manual system of service were also divided in terms of how they wanted to be assisted with the services that they requested for. Some of them, particularly the academics, find it frustrating, time consuming and wasting to stand in the queue. Others find benefit to it as they could have the opportunity to interact with the librarians and request for more help other than what they came for. One of the academics explained as follows: “When I am physically present at the library, it helps me to understand more about the services they offer, and build a friendly personal relationship with the librarians”. There was the factor of age, some of the users; particularly students, as expected, were younger than the academic staff. These younger users were more interested and technologically inclined than their older counter‐part users. As a result, they were often quick to learn about the new technology. Even though that was their preference, the technology infrastructures such as mobile stations (e.g. laptop and mobile phone) were not always available to them to access the library’s services. These sometimes discourage the users, as well defeat the intention of the library to provide a mobility service to its users. Many of the users intend to make use of the services of the library as much as they could from wherever they were and as flexibly as possible. This obviously requires deployment of technological innovation. An example is using emerging mobile technology artefacts such as iPhone, iPad and kindle. Unfortunately, the library was not responding to technological innovation as much as it should. According to one of the interviewees, “most of the students do not have laptops. They have mobile phones. Because the library does not have anything on mobile phones, students and staffs have to be physically at the library where the space is not enough for everybody”. There were some conservative people who struggled to learn about new technologies and how they work. This could be attributed to lack of interest and knowledge. The interest was influenced by fear and intimidation of the technologies, which was a manifestation of practical unconscious biasness that they were difficult to learn and understand. As a result, this group of users preferred the other systems where the services were manually carried out, or where a librarian has to use the systems on their behalf. Students and academics have access to the electronic resource portal (ERP). The ERP provided access to the full text electronic journal articles on different databases that the library subscribes to. Some of the databases were only accessible on campus within the Local area network but others were accessible off the University’s campus. Also, when the users were not physically present at the University premises, they could not access some of the databases because of the licence agreement between the University and different vendors. One of the students expressed as follows: Sometimes I don’t want to physically go to the library. Unfortunate, I had to be there because of the limited access and internet connectivity which deprives me accessing some of the journal articles at the university residence. The current systems, both manual (physical) and information systems pose various types of challenges for the different users. Even though there were challenges, the systems had to be used. This was because of the usefulness of the services which were offered by the library. The usefulness was however the end. The means was considered useful by various people in different ways, in accordance with the users’ view and perspectives. Perceived usefulness

162

Eunice Mtshali and Tiko Iyamu Perceived usefulness is defined as the extent to which a person believes that using a system will improve his or her processes, activities, as well as job performance within a context. Usefulness in this context is the extent to which individuals and groups of users perceive the systems to be useful in getting required services from the library of Capital University. The users were always granted the opportunity to loan books from the University’s library using their identification (staff or student cards) as provided to them by Capital University. In order for them to loan any of the materials, they search for its availability before loan request was processed. The library provides infrastructure to enable the users carry out the search. This was often done through an electronic catalogue system installed in PCs. Many users physically come to the library to use the system to get what they wanted; fortunately they could use the system. Some of them thought it was useful for them to use the system at the premises of the library primarily because they could the assistance of the librarians whenever they encounter problems with the system. Even though some of the users thought that it was useful to connect at any time and from anywhere, it was unfortunately not the case as there was often no connectivity when they were outside Capital University’s campus. One of the interviewees expressed his views: “It would be better if we could access the library services off campus. It is difficult to access the services when one is off campus, especially with the electronic journal articles”. Due to the manual and fixed nature of the current systems that are used by the library to provide services, there are numerous challenges, as has been revealed above so far. The systems did not support geographical mobility and flexibility in terms of time‐of‐usage, making it difficult to use the system to achieve the objectives. It would be more useful for both students and academics to access the library services from anywhere and at anytime. A common view among the users was expressed by one of the interviewees as follows: “I would like to have access to the library services from anywhere and at anytime without any difficulty or hindrance”. Mobile technology was thought to be of better assistance in an attempt to increase the library’s productivity and improve effectiveness in its academic obligations and activities. This was mainly because many of the students and academics believed that some of the services were of significant usefulness to them. The services included search for library catalogue for books, and online journal articles on databases, using different mobile devices that they owned. As such, it is of critical importance for those services to be available at all times, online and seamlessly. The services which were provided by the library were considered to be either useful or useless by different individuals and groups. Such considerations or conclusions were based on their various interests, which are influenced by their individual intentions and behaviours. Behavioural intentions In using the services, many of the users preconceived expectations and predictions about the systems or the response of the library’s employees. This formed their behavioural response and approach towards whether to use or not to use, and how they use the systems for the services that they required. The behaviours were influenced by certain factors which were of conscious or unconscious acts in nature, which were both positive and negative. This had impacts on how the services offered by the library were used and received. The user’s intention was to use the library computer based system to access library services; to locate different library material such as books, electronic journal articles, theses and dissertations. Unfortunately, many of the users who required the services were challenged factors which could be classified as ignorance. For example, some students formed opinion and belief that certain systems in the library do not work, as a result, they did not bother to check or verify their thoughts or source of information. According to one of the interviewees, “the OPAC system which we were supposed to use to access the library’s services was sometimes off, with no sign whether they were out of order or not”. Some of the users seem to have given up exploring the systems to get the best out of the services. This was because of their mind‐set and attitude. According to Giddens (1984), the mind that makes up the sentiments of the selfhood could be attributed to the behaviour of unconsciousness. Letseka and Iyamu (2011) argued

163

Eunice Mtshali and Tiko Iyamu that an individual’s consciousness was the layer upon which an individual may either know or claim to process knowledge. At some point, some users’ behaviour became negative and manifested to laziness, in their attempt or intention to use the services of the library. For example, for some users it was inconvenient to leave their homes or offices to physically be at the library because of different reasons such as time, distance between their station and the library premises. One of the interviewees briefly explained: “to get to the library, I have to walk a distance of about 10 – 15 minutes”. Time was main a factor which influences the behaviour of the users in attempts to use the services of the library. This could be attributed to the reason why many services which were by both academic and professional domains were offered online. At the study of this study, some universities have deployed mobile technology to improve their services to the users. There was more human mobility than ever before. As response to change, technologies are required to enable the human mobility. Most of the library users use the system off campus more often than they use it on campus for various reasons such as fewer disturbances. This was because some users did not have enough time during University operations’ hours to use the system. They had other commitments such as lectures and teaching. Unfortunately, the intention to use the services outside campus was often hampered by lack of access to the systems. Some of them, especially the younger generation, were prepared to learn how to use mobile technology such as the blackberry to access the services offered by the University library through the internet at convenient times. The thought and notion of the use of emerging technologies influenced some of the users to have a negative attitude towards the use of the current systems which they considered to be obsolete or outdated. According to one of the users, “I own a blackberry phone; I used it for my internet access. It would be preferable to access the library services from my mobile phone”. Library users could be encouraged and motivated to adopt a new system and use it if they perceive that their organisation is providing the necessary resources and organisational support. Organisational infrastructure support is a determinant of perceived usefulness while technical support is a determinant of perceived ease of use. Actual usage of the system Actual system use refers to users’ utilization of the system functionalities. Any information system that is not used has a little or no value in it. It is therefore important to find out why individuals choose to use or not to use an information system through which the library provides its services. This depends on what individuals’ value in a system and how far the system provides those services. Usefulness and ease of use predict the system usage through attitude and behavioural intentions. Actual use is a behavioural response measured by the individual’s actions. Some of the users did not use the system because they think that it was complicated. As a result, they always asked for assistance from the librarian. According to one of the interviewees, “the library does not have a simple, easy and flexible computer catalogue with easy terms and a simple way of using it”. In terms of system usage, location was critical. More students and academics used the system to access library services while on campus because that was when they could have access to the system. As a result, many of them were forced to be on campus even when they did not intend to. One of the users expressed herself as follow: “As a fulltime postgraduate student, it is convenient for me because I am always on campus. I don’t have a personal PC with internet connectivity”. Some of the users, particularly the postgraduate students and academics prefer to use the system while outside the University campus. This was primarily because of the nature of their work. Unfortunately they could not because there was no connectivity to the library’s system from off campus. Some of the journal articles in the databases could only be accessed through the university network, which was only possible while within the campus.

164

Eunice Mtshali and Tiko Iyamu Another deficiency was the limited resources at the library. There seems to be a shortage of staff resulting in a slow response to users’ requests for assistance. For example, only two librarians were dedicated to a Faculty with different departments, a total of seven thousand students and a large number of academics. The users had different interests and zeal to use the systems. The way or manner in which some of them used the systems also differs. Many of the users had preferred a flexible and scalable usage of the system. There were many factors which impact and influence the results and output of the current systems which were used deployed at Capital University’s library. Those factors were extracted from the analysis, using the Technology Acceptance Model. The findings are now presented in the next section.

4. Findings and discussion Based on the analysis as discussed above, factors which influence and impact the current systems used by the library services at Capital University were identified. The factors include technology obsolesce, lack of training, information sharing, technology innovation, lack of resources and organisational structure. These factors could also have a negative impact on any new systems if they are not well managed. The findings are depicted below in Figure 1 and discussed.

Technology Obsolesce

Organizational Structure

Technology Innovation

Information Sharing

Lack of Resources

Lack of Training

Figure 1: System critical factors Technology obsolesce The Capital University library was challenged in striving to meet its user’s needs. This was attributed to the state of the technologies which were currently deployed in the library. The majority of the technologies were obsolete and had reached the end‐of‐life. Technology end‐of‐life means its parts and accessories are prohibitive or no longer exist. The life span of technology infrastructure such as personal computer (PC) and laptop is between and five years. This is in accordance to the definition of vendors such as IBM and Intel. As a result of obsolesce or end‐of‐life nature of the technologies, they were either incompatible or difficulty to integrate with the emerging technologies such as the mobile technologies. For example, it was difficult or impossible to connect to the current system from outside the University campus network. The users had different backgrounds; their needs were demanding and dynamic. Therefore they required appropriate responses. The current technologies which were deployed were not able to fulfil these requests and desires. The Library needs to make contributions as information disseminators, they need to understand and exploit ICT infrastructures and emerging technologies in delivering their services to the users. The library could embrace advances in technology changes so that they could be on path with the changing technology and be able to provide better services to the users. Systems’ Training Capital University’s library offers training to users of it facilities primarily to be able to use their system independently when accessing some of the services such as electronic journal articles. Some of the users did attend the training but others did not because they did not perceived it to be useful. The non‐attendance of the training sessions was attributed to lack of awareness. The training schedules were poorly communicated to the users. In addition, there was no clear indication or plan which was circulated in advance for users’

165

Eunice Mtshali and Tiko Iyamu preparation. Users were therefore not encouraged to attend the training sessions. This has contributed to their difficulty in using and accessing library services with the current system which is deployed in the library. The training sessions which were provided by the library was of vital importance mainly because some of the library staff members do not possess the necessary skill that was required to carry out their expected duties in assisting the users on their request. The training was aimed to enable the users for self‐help. Another factor that made the training vital was the shortage of staff in the library. There was only one staff in each section (per Faculty) of Capital University’s library. Lack of Resources The success of any organisation depends on its ability to provide and maintain products, facilities and services that answer the needs of its users. The library was responsible for fulfilling information needs of its users. The proper performance of the library’s functions requires well organised services which would be supported by infrastructure comprising of an administrative authority in the form of legislation, organisational structure, adequate finance, facilities, manpower and collection of documents. The challenge most users, particularly students, experience was the limited number of PCs. The number of available PCs in the library was not enough in ratio to its users. Students had to queue in order to get an opportunity to use the PCs to access the library’s system. The facilities were important because not all students owned personal computers. The limited number of computers led to students’ inability to access the system, making them perceived that complexity in the system. Also, subject librarians were not enough in number. There were only two librarians per faculty responsible for academics as well as both undergraduate and postgraduate students. The shortage of staff affected the turnaround time of the services they provided to users. Information sharing With the rapid increase in demand, it would always be difficult to acquire all the materials which are needed by users. No single library could be self‐sufficient. As such, networking and resource sharing with other libraries is essential. Partnership and cooperation in local, national and international libraries has become unavoidable for Capital University library in order to improve and enhance on their services. Networking is one of the most effective ways of serving users’ needs comprehensively. Capital university library was able to provide information material such as books and journal articles that the university does not have from other libraries. Capital University achieves cooperation through the establishment of consortiums. The agreement between academic and special libraries to assist each other with books and journal articles through interlibrary library loan helps libraries to provide quality service to their users. Resource sharing is aimed at extending the resources and services of libraries to a wider category of users and to improve the ability of participating libraries to perform their basic function of matching users’ needs and information sources. Organisational Structure The library structure consists of different sections that collaborate to achieve a common goal which was to deliver service to users. The employees include top management, subject librarians, trainee librarian and library assistants in different sections such as the lending counter and interlibrary loan. The structure limited employment in certain areas. For example, there were only two subject librarians per faculty. This created a challenge and frustration for the users. The users would sometimes wait longer for response. This situation influences the users to assume that the subject librarians were either incompetent or do not manage their time well when it came to actual use of the system. Technology innovation Information technology, as it has emerged in recent decades, constitutes an ongoing process of innovation. Innovation requires an investment in resources. Therefore responsibility lies with senior management to ensure that investments are made wisely and add value to the organisation at large. The continuous changes in technology influence users’ perceptions as well as competitiveness in many organisations. With regard to

166

Eunice Mtshali and Tiko Iyamu library and information services, the technological changes are high. Research tools are making it more user‐ friendly for new ideas to be disseminated and shared very quickly, easily and at real‐time. As revealed in the analysis, the current technological infrastructure limits the users from applying certain technologies. This influenced users’ perceptions and intentions when it came to using the services which the University’s library offers. Latest technologies such as mobile technology have reshaped the way users’ thinks and behaves. Understanding the rapid change of new technology is central to the future of the library in improving the library services.

5. Conclusion The adoption of mobile technology influenced and impacted the factors that have been revealed by empirical study. The factors would guide the adoption and use of the mobile technology to enhance the services which are provided by institutions such as the library, leading to efficient and effective productivity. However, without the appropriate infrastructure facilities, equipment and staff expertise will be impossible to adopt and make use of mobile technology in academic libraries. Mobile devices through mobile technology will enhance and improve students and academics performances and productivities. This makes this study very important to libraries, students and academics.

References Babbie, E. & Mouton, J. (2001) The practice of social research, Oxford University Press, Cape Town. Bicen, H. & Cavus. (2010) “The most preferred social network sites by students”, Procedia Social and Behavioral Sciences, Vol 2, pp 5864‐5869. Bridges, L., Rempel, H.G. & Griggs, K. (2010) “Making the case for a fully mobile library website from floor maps to the catalogue”, Reference Service Review, Vol 38, No. 2, pp 309‐320. Brown, I., Cajee, z., Davies, d. & Stroebel, S. (2003) “Cell phone banking: predictors of adoption in South Africa ‐ an exploratory study” International Journal of Information Management, Vol 23, pp 381‐394. Castells, M., Fernandez‐Ardevo, l. M., Linchuan Qiu, J. & Sey, A. (2007) Mobile Communication and Society: A Global Perspective, MIT Press, Cambridge, MA. Clyde, L.A. (2004) “M‐learning”, Teacher Librarian, Vol 32, No. 1, pp 45‐60. rd Cresswell, J.W. (2009) Research design: qualitative, quantitative and mixed methods approach. 3 Ed., Sage Publications, Thousand Oaks, CA. Gay, G., Stefanone, M., Grace‐Martin, M. & Hembrooke, H. (2001) “The effects of wireless computing in collaborative learning” International Journal of Human‐Computer Interaction, Vol 13, No. 2, pp 257‐76. Giddens, A. (1984). The constitution of society: Outline of the theory of structure, University of California Press, Berkley, CA. Hill, T.R. & Roldan, M. (2005) “Toward third generation threaded discussions for mobile learning: opportunities and challenges for ubiquitous collaborative environments” Information Systems Frontiers, Vol 7, No.1, pp 55‐70. Igben, M.J. & Akobo, D.I. (2007) “State of Information and Communication Technology (ICT) in Libraries in Rivers State, Nigeria” African Journal of Library and Information Science, Vol 17, No 2, pp 150‐34. Letseka, M. & Iyamu, T. (2011) The Dualism of the Information Technology Project, in the Proceedings of the 2011 Annual Conference of the South African Institute of Computer Scientists and Information Technologists, Cape Town. October 3 – 5. pp. 294‐297. Liu, J., Chun‐Sheng, Y., Chang, L., & Yao, J. E. (2003) “Technology acceptance model for wireless internet” Internet Research: Electronic Networking Applications and Policy, Vol 13, No. 3, pp 206–222. Poole, C.E. & Denny, E. (2001) “Technological change in the workplace: a state wide survey of community college library and learning personnel” College & Research Libraries, Vol 62, No.6, pp 503‐515. Roblyer, M.D., McDaniel, M., Webb, M., Herman, J. & Witty, J.V. (2010) “Findings on Facebook in higher education: a comparison of college faculty and student uses and perception of social networking sites” Internet and Higher Education, Vol 13, pp 134‐140. Yin, R. K. (2009) Case study research: design and methods, 4th ed., Sage, Los Angeles.

167

A Theoretical Review of the Impact of Test Automation on Test Effectiveness Donovan Lindsay Mulder1 and Grafton Whyte2 1 Kinetic Skunk ITS, University of the Western Cape, Cape Town, South Africa 2 Namibia University of Science & Technology, Namibia donovan.mulder@kineticskunk.com Abstract: In a recent paper by Whyte and Mulder (2011) on factors impacting on Software Test Effectiveness, Test Automation as a test tool was shown not to have a significant impact on overall test effectiveness, a results which seems to contradict the collective wisdom of the IT software testing industry. Therefore, this paper presents a review of literature and attempts to discover within the existing body of knowledge what academic and practitioner actual experience is regarding test automation. This literature review will form the basis of a thorough examination of the impact of test automation on software test effectiveness. Automated testing is seen as a high‐risk activity used to reduce the cost of testing and to realise other benefits such as reduced test execution time, greater test coverage, and improved quality of testing. Ultimately, these benefits lead to increased competitive advantage through greater product quality and reliability and quicker time to market thus resulting in increased customer satisfaction. Increasingly, it has become common to read about failed attempts at automated testing. There are many reasons for this such as under estimation of effort, complexity and automated test suite maintenance. Increasingly, greater resources are being focused on automated test suite maintenance rather than actual software testing. This research attempts to identify factors that both negatively and positively influence upon automated testing. It also investigates the factors that influence automated test suite maintenance, which is seen as a critical challenge that needs to be met in order to achieve successful test automation. Ultimately, this research seeks to identify techniques and methods that could potentially mitigate the risks associated with automated testing. Keywords: software test automation, automated testing, cost‐benefit model, testware maintenance, testability, test design, test objects, test automation complexity

1. Introduction In Bertolino's, (2007) software testing roadmap "100% automatic testing" is an achievable dream. Bertolino, (2007) states that "far‐reaching automation" is a method of testing keeping pace with the "growing quantity and complexity of software". Bertolino, (2007) states that there have been big steps in the direction of realising this dream within the unit‐testing domain. Automated testing is a method of reducing the cost of testing (Grechanik, Xie and Fu 2009; Ramler and Wolfmaier 2006; Karhu, Repo, Taipale and Kari 2009). The advantages gained by automated testing are numerous. However it has become common to read about failed automated testing initiatives (Persson and Yilmaztürk, 2004; Ramler and Wolfmaier, 2006). Persson and Yilmaztürk (2004) state that an automated testing project is a high‐risk activity. Many and varied types of software development projects ranging from large to small scale have attempted to perform automated testing with disappointing results. In today’s environment this should not be the case (Wissink and Amaro 2006). There are various reasons for this such as unrealistic expectation and gross under estimation of complexity (Ramler and Wolfmaier 2006; Karhu et al. 2009). Therefore, this theoretical review answers these questions.

What factors negatively influence automated testing and,

What guidelines and techniques could potentially mitigate the risks associated with automated testing?

The structure of the rest of the paper is as follows: section 3 provides the background for this research; section 4 discusses the factors that can be categorised into unrealistic expectation and gross under estimation and answers the research questions; section 5 provides recommendations and the conclusion.

2. Background Berner, Weber and Keller (2005) discuss what they term as the six most compelling observations and make four recommendations. They researched five projects on which they worked and produced an experience report, which reviewed current approaches to determine good test automation practice in the industry. They made six observations and four recommendations, which are presented in table1:

168

Donovan Lindsay Mulder and Grafton Whyte Table 1: Observations and recommendations from “observations and lessons learned from automated testing”

Persson and Yilmaztürk (2004) reported their findings on two projects where test automation was implemented. Thirty‐four common pitfalls categorized into six phases of the Automated Testing Lifecycle were mentioned. Not all of the common pitfalls were included as risks on the two projects and not all the included risks were mitigated. Some of the common pitfalls are listed in table 2. Table 2: Common test automation pitfalls

The research aimed to demonstrate that formal risk management could be used to mitigate risks or common faults. This was not the case, and hence Persson and Yilmaztürk (2004), suggest a strong correlation between successful test automation and informal risk management. Wiklund, Eldh, Sundmark and Lundqvist (2012) describes "technical debt" as the difference in the current state and the desired state of a test automation solution. They conducted a case study, designed to increase knowledge on the sources of technical debt and evaluate organisational awareness of technical debt. They summarise four test improvement models and conclude that none takes into consideration the minimisation of "technical debt". They make four observations which are listed in table 3: Wiklund, et al. (2012) did not find a definitive automated test development methodology that keeps technical debt within an acceptable range and propose two areas for further research:

Evaluating the impacts of test tool sharing and test solution usability.

169

Donovan Lindsay Mulder and Grafton Whyte

The implementation of software development principles in automated test solution development.

In the research paper, Karhu, et al. (2009) carries out five case studies at different types of software development companies. The purpose was to investigate and compare the implementation of software test automation. In addition, determine in practise the effects upon test automation. Karhu, et al. (2009) made six contrasting observations, which are listed in table 4: Table 3: Sources of technical debt

Table 4: Contrasting test automation observations

Karhu, et al. (2009) concludes that automation by itself cannot achieve better test coverage; human intervention is required for test selection; the dominant disadvantage of test automation were the costs associated with implementation, maintenance and training. Ramler and Wolfmaier (2006) consider and critique deficient test automation cost models. Five observations were made which are listed in table 5: Table 5: Test automation cost benefit model observations

Ramler and Wolfmaier (2006) discuss the opportunity cost of automated testing and not, and present a straightforward model that takes into account risk mitigation. A limitation of the model is its development under experimental conditions, and not in real life. Factors not considered were ‐ tests that reply on automation; productivity changes over time; growing test effort in iterative development, maintenance costs for automated tests and early/late return on investment. In summary, the reviewed literature provides numerous and consistent reasons for test automation difficulties. These reasons can be categorised into two categories, (1) unrealistic expectation and (2) underestimation of complexity. Inadequate test automation approach, inadequate cost‐benefit models, misunderstanding of the goals of manual and automated testing, inadequate resourcing and skills fall within unrealistic expectations. Maintenance of automated test suites is

170

Donovan Lindsay Mulder and Grafton Whyte impacted upon by testability, test design and maintenance of testware. These critical factors fall within the category of underestimation of complexity. In the following section, the reasons for and consequences of the unrealistic expectations and underestimation of complexity are discussed in detail.

3. Theoretical review 3.1 Unrealistic expectation 3.1.1 Inadequate test automation approach Berner et al., (2005) state that there are four common mistakes when designing a test automation approach, that leads to inadequate test automation. The mistakes are the (a) misplacement and non‐implementation of certain test types; (b) mismanagement of management of expectations; (c) poor heterogeneity in test type execution; (d) test automation being confined to test execution. Persson and Yilmaztürk (2004) argue that an inadequate test approach is a common pitfall of test automation approaches. 3.1.2 Inadequate cost‐benefit models Most organisations have disproportionate expectations about the advantages of test automation (Berner, et al., 2005). Karhu, et al. (2009) postulate that the greatest perceived benefits of test automation when compared with manual testing are greater test coverage resulting in greater quality and more testing in less time. Earlier defect detection, reduced costs of defect resolution and enhanced resource allocation effectiveness. Berner, et al. (2005) state that test case quality increases as test analysts have more time to enhance test cases and develop new test cases. However, there are new costs such as direct investment costs, extra implementation time, and the need for extra human resources which is a disadvantage (Berner et al., 2005; Karhu, et al., 2009). A quick return on investment is expected, and if not met, test automation can be shelved quite quickly (Berner, et al., 2005). Ramler and Wolfmaier (2006) suggest that only costs are analysed and not benefits. Berner, et al. (2005) state that if an automated test is not cost‐effective is it not due to the amount of times the automated test is executed, but rather to an incongruous test automation strategy or hard to test applications or abject test automation architecture. To overcome unwarranted expectations, these benefits have to be factored into the test automation cost‐benefit analysis (Ramler and Wolfmaier, 2006). 3.1.3 Automated testing cannot replace manual testing Another common misconception, which also contributes to inadequate cost benefit modelling, is that automated testing will replace manual testing. This is not the case (Berner, et al., 2005; Ramler and Wolfmaier, 2006; Persson and Yilmaztürk, 2004). Manual and automated testing has different objectives. In the inceptive phase, automated test development is a manual task. Most defects are detected during this stage (Berner, et al., 2005). Upon automation of a test, its purpose is to revalidate the system under test and discover regression defects. 3.1.4 Inadequate resourcing and skills Other misconceptions that contributes to unrealistic expectations, are that the automated test team does not require programming skills and automated testing will reduce human resources (Persson and Yilmaztürk, 2004). In both cases this not true. Test automation is a software development project in its own right; therefore, appropriate resources and skillsets are required (Fewster, 2001).

3.2 Gross underestimation of complexity 3.2.1 Maintenance of automated test suites Persson and Yilmaztürk (2004), Ramler and Wolfmaier (2006) and Karhu, et al. (2009) agree that the burden of automated test suite maintenance is a common reason for test automation failure. Maintenance of automated test suites is no doubt a critical success factor with respect to test automation. Simple changes to the system under test can incur disproportionately greater changes in an automated test suit (Skoglund and Runeson, 2004; Kalinowski, et al., 2007). Increasingly, maintenance activities consume more resources than testing activities (Berner, et al., 2005). Costs of test suite maintenance of automated test suites are greater when

171

Donovan Lindsay Mulder and Grafton Whyte compared to the equivalent manual test suite (Kalinowski, et al., 2007; Skoglund and Runeson, 2004). If the maintenance of an automated test suite is ignored, amending the automated test suite can cost as much or even more than the initial implementation (Berner, et al. 2005; Fewster 2001). Karhu, et al. (2009) states that automated tests suites require constant maintenance. Berner, et al., (2005) also state that automated test suite maintenance easily becomes a herculean task for most organisations. Fewster (2001) warns that when developing a test automation solution that has as a non‐functional requirement, then expect implementation costs to be higher, and implementation time greater, even though the burden of the test suite maintenance is minimized. Furthermore that the converse to this is also true. Therefore, the question is, 'what risks increase the burden of the test suite maintenance of automated test suites and what can be done to mitigate these risks?' In the next two sub‐sections, we take a closer look factors that if not governed by explicit rules in a test automation approach could severely increase the burden of automated test suite maintenance. 3.2.2 Testability Berner, et al. (2005) considers testability to be the design of the system under test and the configuration of test environment. Bertolino (2007) defines "design‐for‐testability" as essentially software design focused on enhancing input control and feedback. Berner, et al. (2005) claim that testability of the system under test is the primary cost driver and thus the main enabler or disabler of automated testing. It can significantly affect the automated test case development and maintenance effort. It is also a forgotten non‐functional requirement. Fewster, (2001) and Berner, et al. (2005) , Wiklund, et al. (2012) identify test environments as a source of testability challenges especially when migrating tests from one environment to another. Software engineering practices such as iterative developments generates problems for testability (Lamoreaux et al. 2004). This is especially true in graphical user interface (GUI) based systems (McMaster and Memon 2009; Xie and Memon 2008). Increasingly, software users interact with a software based system via a GUI. Functional testing of GUI’s is essential to realising quality and reliability goals. Two reasons for this are: the code that composes current GUI based software systems, critically influences the client’s perception of quality (McMaster and Memon 2008). GUI development is driven by fast prototyping modelling; therefore, regression testing is critical to ensuring quality in GUI based systems (Memon, Pollack and Soffa 2001). Simple changes to the GUI could result in large numbers of GUI tests failing due to failed object recognition or change in execution paths. This constitutes a large amount of the test suite maintenance (McMaster and Memon 2009). Iterative development of this nature and its impact on automated testing is often misunderstood (Persson and Yilmaztürk 2004). The resultant GUI changes inevitably have a negative impact upon test suite maintenance, uncontrolled changes to the GUI results in significant rework of automated tests (McMaster and Memon 2009; Karhu, et al. 2009). This is usually due to the implementation of automated testing too early in the development cycle when the system under test is unstable (Persson and Yilmaztürk 2004; Karhu, et al. 2009). This results in greater utilization of time and resources by automated test suite maintenance. 3.2.3 Test design Persson and Yilmaztürk (2004) expound that an automated test development without guidelines puts at risk modularity, repeatability, maintainability and reusability of testware. Karhu, et al. (2009) states that reuse has long‐term advantages such that the unit price per test case drops significantly at some point. After reusable test components are developed test development productivity increases (Ramler and Wolfmaier 2006). Berner, et al. (2005) found that if test suite maintenance is not a non‐functional requirement and not subject to appropriate architectural guidelines, test suite maintenance quickly became a significant burden. 3.2.4 Testware maintenance Berner, et al. (2005) state that testware comprises of everything required for automated testing and lists four reasons why testware maintenance is difficult. These are (a) undocumented architecture, (b) missed opportunities for reuse, (c) poorly structured testware and (d) untested testware; and the predominant reason for test automation failure is not running automated tests often enough.

172

Donovan Lindsay Mulder and Grafton Whyte

4. Discussion Given the proliferation of test automation in industry there is a need for practical solutions to the issues of unrealistic expectation and gross underestimation of complexity.

4.1 Managing expectation How does one mitigate the risk imposed by unrealistic expectation to testing goals? From the theoretical review, there are four factors to take into consideration. These are 'Test Automation Approach', 'Cost‐Benefit Models', 'Purpose of Manual Testing versus Automation Testing' and 'Resourcing and Skills'. 4.1.1 Test automation approach To mitigate the risk imposed by certain misconceptions about test automation the following three guidelines must be explained explicitly in a test automation approach. The role of test automation is complimentary to manual testing and is not a substitute for manual testing; what the goals of automated regression testing are; and the overall test strategy must take into account both manual and automated testing. An adequate test automation approach will include sections that clearly define a cost‐benefit model, the purpose of manual testing versus automated testing and the required resourcing and skills. 4.1.2 Cost benefit models Ramler and Wolfmaier (2006) propose a model that maximizes investment in automated testing by realizing the maximum test benefit instead by decreasing the cost of testing. This model balances opportunity cost between manual and automated testing. One of the characteristics of the model is that it recognises that manual and automated testing mitigate different kinds of risk. Manual testing addresses the need to expose new or undiscovered defects and automated testing addresses the need to detect regression defects quickly. They suggest calculating the contribution to risk mitigation of a test case against the functionality to be tested. This is in keeping with risk based test case selection, which is a common practise due to time constraints (Whyte and Mulder, 2011). This allows for a more real‐world comparison between manual and automated testing and prevents the assumption that all test case and test cycles are of equal significance from prevailing. It also allows for realistic opportunity cost estimation and the model has a fixed budget as a production possibility frontier. However, the model is straightforward given that certain additional cost factors are omitted. These factors are maintenance costs; time related changing productivity; increased test effort due to evolvement; early or late ROI; and defect detection capability. Ramler and Wolfmaier (2006) suggest that, in practise, these additional factors should be included in the cost‐benefit model; however, this could become impractical given increased complexity and data collection constraints. Therefore, additional factors, specific to a project should be taken into account, as they will better support decision‐making. 4.1.3 Manual testing versus automated testing The purpose of manual testing is to test new functionality (also the first phase of automated test development), find new ways to stress or break a system under test and find new defects. The use of automated testing is to discover regression defects (Ramler and Wolfmaier, 2006; Persson and Yilmaztürk, 2004; Skoglund and Runeson, 2004; Kalinowski, Teixeira and van Oppen 2007). Therefore, automated testing cannot replace manual testing. 4.1.4 Resourcing and skills An automated testing initiative requires more resources than manual testing and even more human resources for analysis and maintenance (Persson and Yilmaztürk, 2004). A common mistake is confusing testing with automation. Test development and test automation require mutually exclusive skill sets (Fewster, 2001). Table 6 presents a summary of how to mitigate the impact of unrealistic expectation of test automation.

4.2 Considering complexity Understanding the factors which contribute to the complexity of test automation and the subsequent risk is essential to minimizing the burden of automated test suite maintenance. Key factors to consider are

173

Donovan Lindsay Mulder and Grafton Whyte testability, test design and testware maintenance. Potential methods of mitigating the impact of these sources of complexity on software test effectiveness is discussed. Table 6: Mitigating the risk imposed by unrealistic expectations

4.2.1 Testability System architecture is notoriously difficult to retrofit, therefore, testability should be a non‐functional requirement from the beginning (Berner, et al., 2005). Testability is a reason why testing should be brought early into the development lifecycle (Persson and Yilmaztürk 2004). The most serious complications are a result of not executing automated tests often enough and from an inadequate test automation strategy. Berner, et al. (2005) suggest the implementation of daily builds and the execution of the automated tests. This practise will keep automated tests synchronized with the system under test and will highlight any environment changes as those described in Wiklund, et al. (2012). 4.2.2 Testability ‐ summary of a case study Lamoreaux, Ofori‐Kyei and Pinone, (2004) determined that during the test tool selection process that most of the software products from a test automation perspective were not testable. The company then decides to develop all its software products to be testable by setting development standards that satisfy the defintion of testability as defined by Bertolino (2007). Lamoreaux, et al. (2004) developed an eight step process to achieve a high level of testability. This enabled automated tests to function effectively and reduced test suite maintenance. 4.2.3 Test design In the study by Wissink and Amaro (2006) keyword based test automation is proposed as an appropriate solution in most software maintenance environments. Wissink and Amaro (2006) also state that significant reduction in the test suite maintenance can be realised by implementing a keyword approach. The is a view supported by Wu et al. (2009) which they espouse in their three‐tier data driven testing framework (Wu et al.

174

Donovan Lindsay Mulder and Grafton Whyte 2009). There are some significant advantages to be gained such as abstraction, which supports domain‐specific languages (DSLs). Bertolino, (2007) states that DSLs afford domain experts the ability to develop expressions of abstract specifications, which can be translated into optimized implementations. Hartman, Katara and Paradkar, (2007) present several methods of expressing domain knowledge within a test design that can be converted into optimized implementations without coding knowledge. Keyword‐driven testing has emerged as a method of providing the abstraction required by DSLs. An advantage of a keyword‐based test is that it is a manual (human readable and executable) and automated test at the same time. Therefore the by its nature it is a test specification for automated testing. However the manual test will require some effort to rework but not as significant as the work required to develop an automated test from the ground up (Pedemonte, et al. 2012). Test automation frameworks have become popular mechanisms in which keyword‐driven testing is supported. A test automation framework is a set of assumptions, concepts and tools that provide support for automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is a change to any test case, then only the test case file needs to be updated. This characteristic mitigates the negative impact of rapid iterative development on test suite maintenance by limiting the number of test execution paths. The combination of a test automation framework and keyword automated test development is considered superior to other test automation development methodologies (Wissink and Amaro 2006; Wu et al. 2009). 4.2.4 Testware maintenance With regards to "undocumented architecture", testware is not developed with the same pertinacity as conventional software development projects (Berner, et al. 2005; Wiklund, et al. 2012; Persson and Yilmaztürk, 2004). It is common that an automated test produce unexpected results where there is a fault in the test code. Where reuse is a goal, test code and utilities will become complicated. Given that test automation is a software development exercise, it should be subject to the same standard software development principles (Fewster, 2001; Berner, et al., 2005). This will address the issue of "poorly structured test ware" and "untested testware". It is of utmost importance that tests are structured such that identification and script location are easy. This will mitigate the risk of duplication of both tests and test data. Reusable automated test scripts increase automated test development productivity. In summary adequately mitigating the complexity of test automation on software test effectiveness requires a multi‐dimentional approach. The dimensions are defining testability requirements, employing a test design which supports domain testing languages through abstraction and applying standard software development principles when developing testware. In the next section recommendations for practitioners and researchers as well as the conclusion are presented.

5. Recommendations It is clear that test automation is no trivial task. It requires a collaborative approach from management, development and testing teams. A first step towards test automation success is implementation of a test automation strategy. Test automation must as a project goal, be defined from the outset and cost‐benefit models must take into account opportunity costs. Success criteria defined as non‐functional requirements such as testability in the system under test and the test environment must be clearly defined. Quality levels or levels of system stability must be met before automated testing can successfully take place. Test design approaches that minimise the effort of moving from manual to automated testing must be employed and the role of automated testing must be understood by all stakeholders. Finally, attention must be paid to skills and resourcing. In section 4, managing expectation is adequately discussed therefore recommendations squarely focuses on managing test automation complexity.

175

Donovan Lindsay Mulder and Grafton Whyte

5.1 Managing complexity 5.1.1 For practioners An approach to test automation should take into consideration testability, test design, testware development and maintenance standards. An arcane approach to test automation usually results in increased maintenance overhead of automated test suites (Berner, et al. 2005). From a testability perspective the "simple 8 step" solution for enhancing software testability is a good start (Lamoreaux, et al., 2004). The solution is summarised below in table 7. Table 7: Eight testability solution (Source: Lamoreaux, et al 2004)

In this research, two sources of testability challenges are discussed: the architecture of system under test (SUT) and the test environment (see table 8). Table 8: Sources of testability challenges

From a test design perspective, practitioners should consider test design methodologies, which reduce the burden of test suite maintenance. Methodologies, which have abstraction as a characteristic and thereby support the implementation of domain testing languages (DTL). Keyword‐driven testing has emerged as an approach that supports the development and implementation of DTL's. The use of test frameworks which are built upon the following strategic guidelines discussed by Nagle, (2000) and Wu, et al. (2009) present an example of such a framework that supports a very high level abstraction. The guidelines proposed by Nagle, (2000) are listed in table 9: Table 9: Test automation framework development strategic guidelines

176

Donovan Lindsay Mulder and Grafton Whyte From a testware maintenance perspective, practitioners should consider implementing software development principles to the development of test automation code. The combination of keyword driven testing and test frameworks affords the ability to separate test automation code from the actual test cases thus the test code can be exposed to the same rigors of software development. Wiklund, et al. (2012) state that testware must be "supported by configuration managment, modularization and independence".

5.2 For researchers ‐ future work From the reviewed literature, the approach to research in software test automation seems to be haphazard. There are studies that accomplish the same thing and some that accomplish very different goals. Research in test automation needs to address certain areas, what many researchers have defined as first principles. Questions that should be answered that will provide practical solutions in industry are: Research questions towards managing expectation:

What makes up an effective test automation approach?

What constitutes a realistic test automation cost‐benefit model?

Do organizations understand the key benefits and the role of test automation?

What roles and skills should a test automation team possess?

Research questions towards understanding test automation complexity: Skoglund and Runeson, (2004) and Kalinowski, et al. (2007) state that despite the fact that automated test suite maintenance is expensive, possibly costing more than the initial implementation there are not many studies that take in account the impact of automated test suite maintenance. Persson and Yilmaztürk (2004), Ramler and Wolfmaier (2006) and Karhu, et al. (2009) agree that the maintenance burden of automated test suites are is primary reason why test automation projects fail. The question to ask here is what are the drivers of this burden? This research paper suggests three key areas: testability, test design and testware maintenance. Berner, et al. (2005) suggest that they know of few studies dealing with testability which is the primary cost driver for test automation. Wiklund, et al. (2012) propose further research in the implementation of software development rigors for testware maintenance in order to reduce technical debt in test automation. Given these statements, the three areas of research towards reducing the burden of test suite maintenance are:

The impact of testability on test effectiveness.

The impact of test design on test effectiveness.

The impact of testware maintenance on test effectiveness.

Towards effective testability, research should focus on developing a universal software testability definition and defining the testability requirements. The research should focus on:

Object Recognition and Object Manipulation, requirements are from a test automation tool perspective.

Reducing the impact of Test Environment Management on testability.

Enhancing testability from a Control and Feedback (system design) perspective.

Towards reducing the burden of test suite maintenance, test design and testware, maintenance plays a critical role. Keyword driven testing which supports domain testing languages through abstraction, have been shown in research and practice to reduce this burden. Model based test automation is a burgeoning research area which potentially, could further reduce test suite maintenance from test input and test case generation (Bertolino, 2007). Research should focus on evaluating test design methodologies from an industry perspective. Key areas to evaluate are:

The impact of domain testing languages from a keyword‐word driven perspective on test effectiveness.

The impact of test frameworks on test effectiveness.

The impact of applying the rigors of software development to testware maintenance on test effectiveness.

177

Donovan Lindsay Mulder and Grafton Whyte Figure 1 presents a graphical view of the direction of research as suggested in this paper.

Figure 1: Towards managing complexity of automated testing

6. Contributions to industry and research This study makes five contributions:

In section 2 a high level theoretical review of the literature was presented. In these review common pitfalls of test automation was presented. These common pitfalls were categorized into Unrealistic Expectation and gross Underestimation of Complexity.

In section 3 the common pitfalls with the greatest negative impact upon test automation as determined by the theoretical review was discussed.

In section 4 was a discussion of possible mitigating or avoidance actions for each of the common pitfalls.

In section 5 recommendations;

for practitioners wanting to implement test automation are made and,

For researchers, on the direction future research should be taking.

7. Conclusion From the reviewed literature, the research questions (a) 'What factors negatively influence automated testing?' and (b) 'What guidelines and techniques could potentially mitigate the risks associated with automated testing?' are answered; and recommendations are made for both practitioners and researchers on the future direction of automated testing.

References Anon, Test automation framework. wikipedia. Available at: http://en.wikipedia.org/wiki/Test_automation_framework [Accessed December 11, 2012]. Berner, S., Weber, R. and Keller, R.K., 2005. Observations and lessons learned from automated testing. In Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. ST. Louis, Missouri, USA: ACM, pp. 571–579. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1553603. Bertolino, A., 2007. Software Testing Research : Achievements , Challenges , Dreams. In IEEE Transactions on Software Engineering. Fewster, M., 2001. Common mistakes in test automation. Grove Consultants, pp.1–8. Available at: http://techwellstage.com/sites/default/files/articles/XDD2901filelistfilename1_0.pdf [Accessed December 3, 2012].

178

Donovan Lindsay Mulder and Grafton Whyte Grechanik, M., Xie, Q. and Fu, C., 2009. Maintaining and evolving GUI‐directed test scripts. Software Engineering, 2009. ICSE …, pp.408–418. Available at: http://portal.acm.org/citation.cfm?id=1555055 [Accessed November 25, 2012]. Hartman, A., Katara, M. and Paradkar, A., 2007. Domain specific approaches to software test automation. Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering ‐ ESEC‐FSE ’07, p.621. Available at: http://portal.acm.org/citation.cfm?doid=1287624.1287730. Kalinowski, M., Teixeira, H.V. and Oppen, P.J.H.V., 2007. ABAT: An Approach for Building Maintainable Automated Functional Software Tests. In XXVI International Conference of the Chilean Society of Computer Science (SCCC’07). Iquique, Chile: Ieee, pp. 83–91. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4396980. Karhu, K. et al., 2009. Empirical Observations on Software Testing Automation. In 2009 International Conference on Software Testing Verification and Validation. Ieee, pp. 201–209. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4815352 [Accessed April 17, 2011]. Lamoreaux, T., Ofori‐Kyei, M. and Pinone, M., 2004. A process for improving software testability. In 20th IEEE International Conference on Software Maintenance, 2004. Proceedings. Ieee, pp. 502–502. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1357850. McMaster, S. and Memon, A., 2008. Call‐Stack Coverage for GUI Test Suite Reduction. IEEE Transactions on Software Engineering, 34(1), pp.99–115. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4378345. McMaster, S. and Memon, A.M., 2009. An Extensible Heuristic‐Based Framework for GUI Test Case Maintenance. In 2009 International Conference on Software Testing, Verification, and Validation Workshops. Ieee, pp. 251–254. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4976393 [Accessed June 17, 2011]. Nagle, C., 2000. Test automation frameworks. URL: http://safsdev. sourceforge. net/ …. Available at: http://pragmaticstory.com/attachment/cfile21.uf@1238AC124A2C977486E725.pdf [Accessed December 11, 2012]. Pedemonte, P., Mahmud, J. and Lau, T., 2012. Towards automatic functional test execution. Proceedings of the 2012 ACM …, pp.227–236. Available at: http://dl.acm.org/citation.cfm?id=2167005 [Accessed December 13, 2012]. Persson, C. and Yilmaztürk, N., 2004. Establishment of automated regression testing at ABB: industrial experience report on’avoiding the pitfalls'. In Proceedings. 19th International Conference on Automated Software Engineering, 2004. Published by the IEEE Computer Society, pp. 112–121. Available at: http://www.computer.org/portal/web/csdl/doi/10.1109/ASE.2004.10036 [Accessed June 4, 2011]. Pfeiffer, M. and Pichler, J., 2008. A comparison of tool support for textual domain‐specific languages. … the 8th OOPSLA Workshop on Domain‐Specific …. Available at: http://www.dsmforum.org/events/dsm08/Papers/1‐Pichler.pdf [Accessed March 29, 2013]. Ramler, R. and Wolfmaier, K., 2006. Economic perspectives in test automation: balancing automated and manual testing with opportunity cost. In Proceedings of the 2006 international workshop on Automation of software test. ACM, pp. 85–91. Available at: http://portal.acm.org/citation.cfm?id=1138946 [Accessed June 6, 2011]. Skoglund, M. and Runeson, P., 2004. A case study on regression test suite maintenance in system evolution. In 20th IEEE International Conference on Software Maintenance, 2004. Proceedings. Ieee, pp. 438–442. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1357831. Whyte, G. and Mulder, D., 2011. Mitigating the Impact of Software Test Constraints on Software Testing Effectiveness. The Electronic Journal of Informaiton Systems Evaluation, 14(2), pp.254–270. Available at: http://www.ejise.com/issue/download.html?idArticle=776 [Accessed December 8, 2012]. Wiklund, K. et al., 2012. Technical Debt in Test Automation. 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp.887–892. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6200204 [Accessed December 3, 2012]. Wissink, T. and Amaro, C., 2006. Successful Test Automation for Software Maintenance. 2006 22nd IEEE International Conference on Software Maintenance, pp.265–266. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4021345. Wu, T. et al., 2009. Study on the Automatic Test Framework Based on Three‐tier Data Driven Mechanism. In 2009 Eighth IEEE/ACIS International Conference on Computer and Information Science. Ieee, pp. 996–1001. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5223223 [Accessed February 3, 2011]. Wybouw‐cognard, L., 2002. Test automation framework. EP Patent 1,179,776. Available at: http://www.freepatentsonline.com/EP1179776.html [Accessed December 11, 2012]. Xie, Q. and Memon, A.M., 2008. Using a pilot study to derive a GUI model for automated testing. ACM Transactions on Software Engineering and Methodology, 18(2), pp.1–35. Available at: http://portal.acm.org/citation.cfm?doid=1416563.1416567 [Accessed June 13, 2011].

179

Contributing Factors of Cloud Computing Adoption: a TechnologyOrganisation-Environment Framework Approach Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw RMIT International University, Ho Chi Minh City, Vietnam mathews.nkhoma@rmit.edu.vn dptdang@gmail.com Anthony.deSouza-Daw@rmit.edu.vn

Abstract: Cloud computing as an emerging high technology has been recognised by organisations and individuals for a wide range of potential applications. Since the concept’s first appearance in 2007, the authors found a dominant amount of studies in the non-technological domains, including attempts to define and categorise cloud computing and the challenges and issues of the technology’s adoption. Nonetheless, few researches are dedicated to determine the drivers of adopting cloud computing thus the literature is limited on this topic. As more adopters are becoming familiar with the technology and implementing cloud computing in their daily work, understanding what drives their adoption decision is essential to create opportunities for future cloud technologies to be tailored and aligned with the consumer’s needs thus promotes exploitations of the technology’s promising applications. This research takes a quantitative approach by developing and validating a theory-based conceptual model. Among the theories that are commonly applied in Information Systems research, the authors found Technology-Organisation-Environment framework can encapsulate the adoption’s factors into one big picture. The authors conducted a secondary data analysis on the recent large-scale survey of IBM to investigate the drivers and barriers of cloud computing adoption. Structural Equation Modelling and Partial Least Square statistical methodologies provide rigid scientific procedures to validate the conceptual model. This study contributes a statistically validated conceptual model of the drivers and barriers of cloud computing adoption. In addition, the research provides a comparison between two different discussions (i.e., industry report and academic research) on the same topic and data. The findings benefits are twofold. First, it seeks to clarify the profound knowledge on the factors surrounding cloud adoption to better understanding cloud computing. Second, it also provides directions for future research by suggesting validations on the proposed model while discussing the limitations of analysing commercial survey. Keywords: cloud computing, adoption, drivers, barriers, structural equation modelling, Technology-OrganisationEnvironment-Framework

1. Introduction Cloud computing offers numerous competitive advantages to today’s businesses. First and foremost, it offers the distributive IT hardware and software which saves the costs of the organisation’s IT infrastructure. This feature is especially beneficial for small and medium size business as they can adopt emerging software easily without requiring to make many purchasing, but rather one purchase and share it on the company’s cloud (Aljabre 2012). Likewise, managers do not need to invest much in high performance computers since cloud computing allows access to centralised applications which do not require to be installed locally. The low costs, minimum technical expertise requirements, flexible and dynamic applications of cloud computing makes it easier for technology adopters to make the switch to cloud computing. Businesses operations also become more agile and effective when they can scale their IT infrastructure which makes entering markets faster as well as meet customers’ demands. Finally, cloud computing offers opened environment which fosters distance online collaboration (e.g., Google Drive) as well as sharing repository (e.g., Dropbox) amongst the employees (Aljabre 2012). While disputes on such benefits still leave rooms for future debates and corrections, the authors believe those advantages can partly explain why businesses chose to adopt cloud computing. More important, investigating this subject matter in details can reveal the current concerns of businesses, including motivations and barriers, on the adoption of cloud computing. In the next section, the authors present the literature review of this study that includes discussions on both theoretical and conceptual backgrounds. After declaring the boundary and the relevant knowledge included in this research, Section 3 presents the hypotheses that are based on such backgrounds. Section 4 describes the selected research methodology – triangular approach of secondary data back with the IBM dataset. Section 5 provides the detailed data analysis and discussions of the validated hypotheses, followed by Section 6 – Limitations and Recommendations – which concludes the study.

180

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw

2. Literature review Cloud computing adoption has great growth potential with the current and predicted total budget to be spent on its services. For example, Gartner reported significant amount of money spent on cloud computing (including Infrastructure-as-a-Service (IaaS), cloud management, security devices and Platform-as-a-Service (PaaS)) totalling $7.6 billion in 2011 worldwide and projected to be $35.5 billion in 2016 (Gartner 2012). Such enormous figures have been encouraging joint efforts from academia and industry to understand the reasons and contributing factors. Indeed, the authors found several relevant research articles that drew conclusions on the challenges and drivers of cloud computing adoption. Nonetheless, the found literature displays a lack of theories applied in overall when presenting their findings thus suggests gaps in the field. The relevant literature reveals that most drivers of cloud computing adoption are benefits-driven. However, not all benefits of cloud computing could drive investment as such decision often requires careful cost-benefits analysis. In other words, organisations invest in cloud computing while expecting its business values in return – but only when they feel cloud computing is needed. This argument also clarifies the possible confusion between disadvantages and barriers of cloud adoption. In addition, it emphasises the necessary criteria of this study when selecting and reviewing literature: the authors are only interested in studies whose findings explicitly indicate factors as drivers or barriers of cloud adoption. Further, some drivers are consistent with those that were surveyed by IBM while some were regarded as challenges. Likewise, (Narasimhan and Nichols 2011) also asserts that cloud computing adopters have different perceptions regarding cloud computing. Table 1 below summarises the factors and their sources that indicate the factors’ roles; Section 3 will elaborate the details of the drivers. Table 1: Drivers and barriers of cloud computing adoption (adopted from IBM Center for Applied Insights 2012) Factor Business scalability Cost flexibility – Allows “pay as and when needed” model Access to industry expertise Cloud security

Driver Carroll, Merwe, and Kotzé 2011; Wood and Anderson 2011; Chebrolu 2011; Pandey et al. 2010; Gupta 2010; Leavitt 2009

Barrier

Carroll, Merwe, and Kotzé 2011; Sultan 2010; Leavitt 2009 Pandey et al. 2010 Carroll, Merwe, and Kotzé 2011; Fortis, Munteanu, and Negru 2012; Wood and Anderson 2011; Chebrolu 2011; Gupta 2010; Dargha 2012; Phaphoom et al. 2012; Leavitt 2009 Carroll, Merwe, and Kotzé 2011; Fortis, Munteanu, and Negru 2012; Low, Chen, and Wu 2011; Leavitt 2009

Carroll, Merwe, and Kotzé 2011 Compatibility with existing applications Reliability and availability Extendibility of existing applications to the cloud Regulatory, governance and compliance policies Proprietary vendor platforms/lack of IT standards

Leavitt 2009 Carroll, Merwe, and Kotzé 2011 Carroll, Merwe, and Kotzé 2011; Wood and Anderson 2011 Carroll, Merwe, and Kotzé 2011; Fortis, Munteanu, and Negru 2012; Gupta 2010; Dargha 2012; Leavitt 2009

3. Hypotheses development 3.1 Theoretical background To add a profound theoretical stance to this investigation, the authors found the Technology-OrganisationEnvironment (TOE) framework (Tornatzky and Fleischer 1990) can encapsulate the adoption’s factors into one

181

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw big picture. More important, the framework was originally developed to link Information Systems innovation adoption decisions with contextual factors, thus can be considered a fit with cloud computing as an emerging technology (Chau and Tam 1997). The TOE framework posits the influences of the factors from three contexts (Technology, Organisation and Environment) on the organisation’s adoption decision. The framework offers a holistic view on the multiple facets of an organisation rather than focuses on an individual’s viewpoint such as the Technology Acceptance Model.

3.2 Conceptual background As the study attempts to conduct a secondary data analysis based on IBM data, this section identifies such factors and discusses their relevant to the study. Such discussions are crucial as they establish concrete concepts that are to be fitted in the theoretical background. Accordingly, the authors excluded the elaborations of Market adaptability, Hidden complexity and Inadequate IT skills as few evidences were found advocating their driver’s and barrier’s function. However, these factors are still included in the study’s hypotheses as well as the conceptual model to conform to IBM survey’s results. 3.2.1 Drivers of Cloud Computing Business scalability can be one of the best reasons for organisations to consider cloud computing adoption (Phaphoom et al. 2012). This factor exploits the virtualising and leveraging capability of cloud computing that allow businesses to scale down its infrastructure to save costs (including hardware, software and labour etc.) while improving IT performance (Kim et al. 2009; Aljabre 2012). Further, organisations can enjoy the green benefits thanks to less power consumption from the scaled infrastructure (Marston et al. 2011; Sultan 2010; Pandey et al. 2010). On the other hand, scalability is desirable for continuous growth as the efficient environment of cloud can host very large volumes of data (Baars and Kemper 2011). At the same time it also supports the rapid deployment of computational tools which help businesses to be more responsive and proactive in operations (Marston et al. 2011). Cost flexibility appears to be a desirable feature of cloud computing that allows businesses to “pay as and when needed”. Indeed, such ability to control costs is beneficial to small and medium businesses, especially those that are new (Leavitt 2009). Furthermore, (Sultan 2010) asserted that cloud computing’s cost flexibility is also appealing to educational institutes. The author provided a case where the University of Westminster can literally utilise Google Mail for free instead of spending huge budget on equivalent data storage (Sultan 2010). Access to industry expertise refers to the ability to share best practices using cloud computing’s communication capability. The authors found a dominant amount of literatures highlighting the capability to foster internal collaborations such as teleworking and external communications with customers; however only one of them mentioned organisations’ need to invest in cloud computing to share best practices within the industry (Marston et al. 2011). Despite the lack of clarification on this factor, one study revealed that the need for growth in business collaboration was one of the top drivers of cloud adoption list with 54% consensus (Pandey et al. 2010). Considering the TOE framework, the authors incorporated the discussed drivers into the construct “Perceived Benefit” as they reflect well organisational needs as a whole. This led to the hypothesis: H1. Perceived Benefit (PB) has positive impact on Intention to adopt (ITA) cloud computing. 3.2.2 Barriers of Cloud Computing Cloud security has always been widely discussed as the main barrier that prevents businesses from adopting cloud computing. Indeed, insecure cloud infrastructure could lead to problems between the adopter and the major stakeholders such as customers, partners, providers or even the government. One common security concern is the organisation’s data ownership and control when leveraging the provider’s cloud services (Chebrolu 2011). By doing so, businesses often feel insecure as they have to entrust their confidential data to the provider’s protection against the emerging cyber-threats. In particular, privacy of employees’ data, consistency and integrity of retrieved data (especially when multiple providers are involved in the process) are the top reasons that make organisations feel reluctant to migrate to the cloud (Sengupta, Kaulgud, and Sharma 2011; Motta, Sfondrini, and Sacco 2012).

182

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw Compatibility with existing applications (and extendibility of existing applications to the cloud) is also an issue when businesses consider adopting cloud technology (Heinle and Strebel 2010; Dargha 2012). This issue occurs when cloud adopters have little controls over the pre-designed computing platform by the provider, thus are required to comprise on flexibility. In addition, the vendor can also change the platform as they feel necessary but without the customer’s consent (Leavitt 2009). As organisations are operating well with their current infrastructure, any changes required by cloud integration can be a daunting task. Reliability and availability are important features of cloud computing that organisations expect to have once they adopted the technology. These features are crucial as businesses often utilise cloud as a platform that supports daily operations and interact with customers and suppliers. As a consequence, any errors or delays that affect reliability and availability can jeopardise the whole process as well. However, the provider may fail to scale up their infrastructure or maintain high uptime and bandwidth to meet customer’s demand in terms of usage, thus affects the stability of the cloud services at some point in time (Kim et al. 2009; Marston et al. 2011; Heinle and Strebel 2010; Leavitt 2009). Regulatory, governance and compliance policies can make businesses feel reluctant to adopt cloud computing due to the lack of regulations governing data ownership and privacy as well as data audit (access) and reporting rights (Kim et al. 2009; Marston et al. 2011; Motta, Sfondrini, and Sacco 2012). In addition, cloud adopters are also concerned about whether they would receive support and protection from the government in case a breach occurred. Proprietary vendor platforms/lack of IT standards is the specific barrier that leads to the issues of Cloud security and Compatibility as discussed above. Likewise, Table 1 displayed that these three barriers share the same literatures. Given the discussed barriers of cloud computing adoption and the remaining components of TOE framework (i.e., Technology and Environment), the authors developed two constructs. Specifically, Regulatory and Vendor appear to be outside of the business and its controls, thus formed the construct “Perceived Environment Barriers”. On the other hand, the remaining barriers fit into “Perceived Technology Barriers”. As a consequence, the authors proposed the following hypotheses: H2. Perceived Environment Barriers (PEB) has negative impact on Intention to adopt (ITA) cloud computing. H3. Perceived Technology Barriers (PTB) has negative impact on Intention to adopt (ITA) cloud computing. In addition to the main components of TOE framework, the authors also utilise data from the IBM survey that describes companies’ adopting style towards cloud computing. Specifically, the surveyed questions include measurement of new technologies’ importance within the organisation’s body, pace and predominant approach of adoption. These questions together indicate the adopter’s styles that are introduced in the IBM report, namely Pacesetter, Follower and Dabbler (IBM Center for Applied Insights 2012). Accordingly, Pacesetters are found to be more proactive at adopting cloud computing while the other two appear to have less intention. In accordance to that finding, we proposed the following hypotheses representing the Follower’s and Dabbler’s styles: H4. Adopter’s style (AS) has negative impact on Intention to adopt (ITA) cloud computing. H5. Adopter’s style (AS) has positive moderating impact on Perceived Technology Barriers (PTB). H6. Adopter’s style (AS) has positive moderating impact on Perceived Environment Barriers (PEB). H7. Adopter’s style (AS) has positive moderating impact on Perceived Benefits (PB). The conceptual model (Fig. 1) illustrates the proposed hypotheses and contributing factors mapping:

183

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw

Figure 1: Conceptual model ( adopter style are connected to perceived barriers and beneficial use hypothesis and elaborated into key categories)

4. Research methodology The authors performed a secondary data analysis using Structural Equation Modelling (SEM) methodology. The data source comes from a survey of 1,200 technology decision-makers conducted in 16 different industries and 13 countries, released in 2012 by IBM. The questionnaire was designed to capture insights about adoption of emerging technologies (e.g., cloud computing, social business, business analytics), particularly about the drivers and barriers surrounding the adoption as well as its current stage within each firm. Furthermore, the questionnaire contributes to this research its variables in terms of drivers and challenges which were crucial for the authors to conceptualise the research model. These variables were grouped and organised within the TOE framework so that the study can be coherent to a profound theory. To validate the model, the authors employed SEM statistical techniques to examine the correlations between the surveyed variables and their respective constructs indicated by the TOE framework.

5. Data analysis and discussion The research model was assessed by using Partial Least Squares (PLS) technique. SmartPLS 2.0 (Ringle, C.M., Wende, S. and Will 2005) was used to assess the research model. PLS is a least squares regression-based technique that can analyse structural models with multiple-item constructs and direct, indirect, and mediating paths. In addition, PLS is considered as a robust estimation method with respect to the distributional assumptions regarding the underlying data and tests of normality. Bootstrapping procedure with the resample of 200 was applied to provide the standard error and the t-statistics of the path coefficients. Measurement model in PLS is assessed in terms of item loadings, internal consistency, and discriminant validity. For construct validity, item loadings and internal consistencies greater than 0.7 (in some cases 0.5 for item loadings) are considered as adequate (Fornell and Larcker 1981; Hair et al. 2006). For discriminant validity, item loadings on their own construct should be higher than on other constructs, and the average variance shared between each construct and its measures should be greater than the average variance shared between the construct and other construct (the squared root of AVE of each construct is greater than all the

184

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw correlation coefficients with other constructs). The structural model and hypotheses are tested by examining 2 the standardized path coefficients. The explained variance in the dependent constructs (R values) is assessed as an indication of the overall predictive power of the model.

5.1 Measurement model It is required that the measurement model is checked for reliability and validity before the structural model is estimated. The test of measurement model includes the estimation of internal consistency and the convergent and discriminant validity of the instrument items. Cronbach’s alpha coefficient, whose value ranges from zero (unreliable) to one (perfect reliable), is used to examine the reliability of survey instrument. A value of greater than 0.7 is optimum. However a value of greater than 0.5 is acceptable, but lower than 0.35 must be rejected (Hair et al. 2006). Constructs that have Cronbach’s alpha being greater than 0.7 are interaction constructs, i.e. PB*AS (0.76), PEB*AS (0.752), and PTB*AS (0.783) indicating their good reliability in the model. At a lesser extent, Adopter’s style (0.659) and intention to adopt cloud computing (0.502) have lower Cronbach’s alpha but both are above 0.5 which are acceptable to be included in the model. In contrast, three main variables in the model have very low Cronbach’s alpha that are Perceived Benefit (0.211), Perceived Technology Barriers (0.072), and Perceived Environment Barriers (-0.04). Such results indicated that these constructs are not adequate reliable in the model. However we still keep these constructs in the model for the purpose of illustration. Convergent validity is adequate when constructs have an average variance extracted (AVE) of at least 0.5. For discriminant validity, the square root of AVE for each construct should be greater than the correlation coefficients between the particular constructs and any other constructs (Chin 1998). Table 2 lists the correlations of the latent variables and the square root of AVE on the diagonal. In all cases, the square root of AVE for each construct is greater than 0.5, indicating sufficient convergent validity of the constructs. In most cases, the square root of AVE is larger than correlation of one construct with all others in the model, except the interaction terms between Perceived Benefit and Adopter’s style (PB*AS) and between Perceived Technology Barriers and Adopter’s style (PTB*AS). Construct validity is further examined by using factor loading analysis. First, items with factor loadings below 0.3 among all factors are to be deleted. Second, items with factor loadings of greater than 0.3 and which appear for more than one factor are also deleted. As the results, eleven items were deleted from the following analysis. These items are AS4, PTB3, PTB4, PTB5, PB1, PB2, ITA2, ITA3, ITA4, ITA5, ITA7, ITA8, ITA14, ITA15, ITA16 and ITA17. Table 2: Correlations of Latent Variables and Square Root of AVE AS

ITA

PB * AS

PEB

PEB * AS

PTB

0.770

ITA

-0.46933

0.584

PB PB * AS

-0.22874

0.195438

0.621

0.154142

-0.04161

0.6003

0.538

PEB PEB * AS

-0.03837

0.040404

0.142589

0.099632

0.700

0.250413

-0.09065

0.030777

0.119612

0.591697

0.670

PTB PTB * AS

0.069347

-0.05452

0.029451

-0.0291

-0.1012

-0.00342

0.696

0.455826

-0.2215

-0.06247

0.047198

-0.09161

0.084662

0.86287

185

PTB * AS

0.690

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw

Figure 2: Structural model results

5.2 Structural model: 2

The estimation of the structural model includes the estimation of the path coefficients and the R values. Path 2 coefficients indicate the impacts of the independent variables on the dependent variable, while R values represent the amount of variance explained by the independent variables or the overall explanatory power of 2 the model. Together, the R and the path coefficients (loadings and significance) indicate how well the data support the hypothesised model. The path coefficients from the PLS analysis are shown in Figure 2. Overall, the estimated model explained for only about 23.3% of the variance in intention to adopt cloud computing. This is considered as quite low explanatory power of the estimated model. According to the statistical results only Adopterâ&#x20AC;&#x2122;s style (AS) has a significant negative direct impact on the Intention to adopt cloud computing (-0.48). This negative sign is indeed strange in common sense. However, this is reasonable in our analysis due to the way how the values of the Adopterâ&#x20AC;&#x2122;s style items were defined. In our dataset, the values of the items are defined from the most (IT) professional to the least (IT) professional, with the corresponding values varied from 1 to 5. As the result, when the adopter style went from more professional to less professional (from 1 to 5), the adopters are less willing to adopt the cloud computing. The remaining variables in the model show no significant impact on intention to adopt cloud computing although all the coefficients have expected signs. Specifically, Perceived Technology Barriers (PTB) and Perceived Environment Barrier (PEB) showed no statistically significant impacts on Intention but both showed negative impacts on Intention as expected. The more the users perceived about the technology and the environment barriers, the lesser extend the users are willing to adopt cloud computing. In contrast, Perceived Benefit (PB) increases the Intention to adopt, though the impact is also not statistically significant. In our model, Adopterâ&#x20AC;&#x2122;s style was hypothesised to have moderating effects on the impacts on PTB, PEB, and PB. However, unfortunately, statistical results showed that all of these moderating impacts were not statistically significant, i.e. all the slope coefficients of the interaction term are not statistically different from zero. The hypothesis testing result can be summarised as Table 3 below:

186

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw Table 3: Hypothesis testing result Hypothesis H1. Perceived Benefit (PB) has positive impact on Intention to adopt (ITA) cloud computing H2. Perceived Environment Barriers (PEB) has negative impact on Intention to adopt (ITA) cloud computing. H3. Perceived Technology Barriers (PTB) has negative impact on Intention to adopt (ITA) cloud computing. H4. Adopter’s style (AS) has negative impact on Intention to adopt (ITA) cloud computing. H5. Adopter’s style (AS) has positive moderating impact on Perceived Technology Barriers (PTB). H6. Adopter’s style (AS) has positive moderating impact on Perceived Environment Barriers (PEB). H7. Adopter’s style (AS) has positive moderating impact on Perceived Benefits (PB).

Result Not supported Not supported Not supported Supported Not supported Not supported Not supported

6. Limitations and recommendations Having presented the scientific analyses on the commercial dataset, we were with regrets for not being able to achieve much significant inferences on the factors surrounding intention to adopt cloud computing. As the reviewed theories shared consensus on the driver and barrier roles of the factors while statistical results displayed otherwise, it suggests that the data’s quality is our most challenging limitation. To produce quality results, we encourage future researches may design better instrument and perform analysis on this study’s proposed model. We reflected this study’s results against the findings from IBM’s Tech Trends report (IBM Center for Applied Insights 2012). The report asserted that information security is a major challenge that organisations need to overcome to adopt cloud computing and the other technologies. Although our hypothesis did not support that finding, information security (as item PTB1) played significant role in the construct Perceived Technology Barriers thus indicated its affection. Similarly, the positive impact of Market adaptability as driver of intention to adopt cloud computing was also consistent with the report albeit its hypothesis was rejected due to low significance level. The most consistent finding from this research to IBM’s report was the negative influence of Adopter’s style on intention to adopt cloud. The supported hypothesis implied that adopters with lower IT are less willing to adopt cloud and vice versa for those with better technological ability. Likewise, IBM reported that more than 70% of Pacesetters, who are more strategic and proactive in exploiting technologies, have been conducting professional training to meet cloud implementation’s requirements. As a consequence, this resulted in their leading position in adopting cloud (34%) in comparison to Dabblers (5%) and Followers (15%). The rest of this paper’s findings slightly confirmed the factors’ relationships with intention to adopt cloud computing thus suggest future empirical studies to re-validate the hypothesised model and clarify our results.

References Aljabre, A. (2012), “Cloud Computing for Increased Business Value,” International Journal of Business and Social Science, Vol. 3 No. 1, pp. 234–239. Baars, H. and Kemper, H. (2011), “Business Intelligence in the Cloud?,” Communication Software and Networks (ICCSN), 2011 IEEE 3rd International Conference on. Carroll, M., Merwe, A. Van Der and Kotzé, P. (2011), “Secure Cloud Computing Benefits, Risks and Controls,” Information Security South Africa (ISSA), 2011, pp. 1–9. Chau, P. and Tam, K. (1997), “Factors affecting the adoption of open systems: an exploratory study,” Management Information Systems Quarterly, pp. 1–24. Chebrolu, S.B. (2011), “ASSESSING THE RELATIONSHIPS AMONG CLOUD ADOPTION, STRATEGIC ALIGNMENT AND IT EFFECTIVENESS,” Journal of Information Technology Management A Publication of the Association of Management, Vol. XXII No. 2. Chin, W.W. (1998), “Issues and opinion on structural equation modeling,” MIS Quarterly, Vol. 22 No. 1, pp. 7–16. Dargha, R. (2012), “Cloud computing: from hype to reality: fast tracking cloud adoption,” Proceedings of the International Conference on Advances in Computing, Communications and Informatics, New York, NY, USA, ACM, pp. 440–445. Fornell, C. and Larcker, D.F. (1981), “Evaluating structural equation models with unobservable variables and measurement error.,” Journal of Marketing Research, Vol. 18 No. 1, pp. 39–50.

187

Mathews Z. Nkhoma, Duy P.T. Dang and Anthony De Souza-Daw Fortis, T., Munteanu, V. and Negru, V. (2012), “Steps towards cloud governance. a survey,” Information Technology Interfaces (ITI), Proceedings of the ITI 2012 34th International Conference on, pp. 29–34. Gartner. (2012), Forecast Analysis: Enterprise Infrastructure Software, Worldwide, 2011-2016, 3Q12 Update. Gupta, A. (2010), “Cloud Computing Growing Interest and Related Concerns,” Computer Technology and Development (ICCTD), 2010 2nd International Conference on., pp. 462–465. Hair, Jr., J.F., Black, W.C., Babin, B.J., Anderson, R.E. and Tatham, R.L. (2006), Multivariate data analysis, Multivariate data th analysis, Upper Saddle River, NJ., Pearson-Prentice Hall, 6 ed. Heinle, C. and Strebel, J. (2010), “IaaS Adoption Determinants in Enterprises,” Economics of Grids, Clouds, Systems, and Services, Springer, pp. 93–104. IBM Center for Applied Insights. (2012), Fast track to the future: The 2012 IBM Tech Trends Report. Kim, W., Kim, S.D., Lee, E. and Lee, S. (2009), “Adoption issues for cloud computing,” Proceedings of the 7th International Conference on Advances in Mobile Computing and Multimedia - MoMM ’09, New York, New York, USA, ACM Press, pp. 2. Leavitt, N. (2009), “Is cloud computing really ready for prime time,” Growth, No. January, pp. 15–20. Low, C., Chen, Y. and Wu, M. (2011), “Understanding the determinants of cloud computing adoption,” Industrial Management & Data Systems, Emerald Group Publishing Limited, Vol. 111 No. 7, pp. 1006–1023. Marston, S., Li, Z., Bandyopadhyay, S., Zhang, J. and Ghalsasi, A. (2011), “Cloud computing — The business perspective,” Decision Support Systems, Elsevier B.V., Vol. 51 No. 1, pp. 176–189. Motta, G., Sfondrini, N. and Sacco, D. (2012), “Cloud Computing: A Business and Economical Perspective,” 2012 International Joint Conference on Service Sciences, Ieee, pp. 18–22. Narasimhan, B. and Nichols, R. (2011), “State of Cloud Applications and Platforms: The Cloud Adopters’ View,” Computer, Vol. 44, pp. 24–28. Pandey, Abhinav, Pandey, Akash, Tandon, A., Kushwaha, U. and Tiwari, V. (2010), “Cloud Computing: Exploring the scope,” 2010 International Conference on Informatics, Cybernetics, and Computer Applications (ICICCA 2010). Phaphoom, N., Oza, N., Wang, X. and Abrahamsson, P. (2012), “Does cloud computing deliver the promised benefits for IT industry?,” Proceedings of the WICSA/ECSA 2012 Companion Volume on - WICSA/ECSA ’12, New York, New York, USA, ACM Press, p. 45. Ringle, C.M., Wende, S. and Will, S. (2005), “SmartPLS 2.0 (M3) Beta,” Hamburg, Germany. http://www.smartpls.de. Sengupta, S., Kaulgud, V. and Sharma, V.S. (2011), “Cloud Computing Security--Trends and Research Directions,” 2011 IEEE World Congress on Services, Ieee, pp. 524–531. Sultan, N. (2010), “Cloud computing for education: A new dawn?,” International Journal of Information Management, Vol. 30 No. 2, pp. 109–116. Tornatzky, L.G. and Fleischer, M. (1990), The Processing of Technological Innovation, Lexington, MA, Lexington Books. Wood, K. and Anderson, M. (2011), “Understanding the Complexity Surrounding Multitenancy in Cloud Computing,” 2011 IEEE 8th International Conference on e-Business Engineering, Ieee, pp. 119–124.

188

IT and Business Value–a Missing gap: A SME Case Study Hiep Pham and Kevin Nguyen Department of Business Information Technology and Logistics, RMIT University Vietnam, Ho Chi Minh City, Vietnam hiep.pham@rmit.edu.vn kevin.nguyen@rmit.edu.vn Abstract: There have been conflicting views on how IT system can deliver sustainable business value. IT in this context refers to structured information management and workflow management. IT has been considered as main tools to improve operational efficiencies in term of cost savings and improved productivity. With the advent of the strategic use of IT in business where IT is expected to contribute to sustainable strategic business value, several approaches can be employed by organisations to realize the business value. Resource Based View (RBV) has been widely used as a framework to guide businesses in their quest for achieving business competitive business value from IT system. Melville et al developed a framework based on RBV theory to identify factors influencing the generation of strategic value from IT system which comprised of three domains: focal firm, competitive environment and macro environment. Focal firms should have IT resources, complementary organizational resources with improved business processes to achieve organizational performance. Two external factors including competitive environment composed of industry characteristics and trading partner resources that firms were operating in and macro factor of country characteristics would shape the IT application and IT business value generation. This study aims to explore the impact of factors identified in Melville’s framework during the implementation of an Enterprise Resource Planning (ERP) system in a Small and Medium Enterprise (SME) – a textile manufacturing firm in Vietnam. The study finds that some factors have clear impact while others especially from macro domain do not have clear impact in the way the potential value generation from the ERP system is planned. The role of IT resources has been considered lightly by the firm, whereas stability of human resources, well defined business process and structure have been viewed greatly more important for the effective use of the ERP system. The model is found to help increase the firm’s awareness of factors that may influence the effective investment of IT. Further research can be done to establish a clearer process to evaluate the transition from one domain to another which is proved difficult to apply in this case. Also a comparative study between industries might explore differences in the key factors in different industries. Keywords: IT business value, sustainable organizational performance, small and medium enterprise (SME), resource based view (RBV)

1. Introduction To stay competitive in today business world with strong globalization and rapid technological changes, firms need to engage in global market, apply the latest technology, and actively structure them to be ahead from their competitors. Investing in information technology (IT) has been considered a good strategy to enhance firm efficiency and effectiveness (Lai et al, 2006). IT has been widely viewed as the crucial tool to generate competitive advantages, productivity as well as flexibility for firms including small and medium sized enterprises – SMEs (Jin, 2007, Moghavvemi et all 2012, Lai and Li, 2005). Studies have showed that ability to apply an appropriate portfolio of IS capabilities is critical to achieve sustained improvement in competitiveness (Doherty and Terry 2009, Valacich and Schneider 2010). Resource Based View (RBV) theory has been widely used as a framework to guide businesses in their quest for achieving competitive business value from IT system. RBV theory points out that adoption of IT is supported through deployment of an appropriate portfolio of IT complementary resources and capabilities which eventually deliver strategic advantage. Furthermore, top management’s knowledge and attitude towards IT adoption together with organisation’s IT skills play important role in producing competences required to achieve sustainable business value from IT system (Caldeira and Ward 2003). SMEs and large firms do have major differences in term of level of resources, flatter/simpler structure, internal power conflict, and bargaining power with suppliers (Cragg et al 2011). SMEs often have low levels of internal IT competences which can be varied by industry (Hashim 2007). In addition, organizational processes are needed to exploit such skills to help them identify and realize IT opportunities in the organizations. Other issues also impose challenges to SMEs are lack of strategic planning, shortage of investment capital in IT (Schubert and Leimenstoll, 2004), lack of clarity around the roles and responsibilities of staff (Dutta & Evrard, 1999), weak organizational culture or unstructured business processes (Buonanno et al., 2005).

189

Hiep Pham and Kevin Nguyen Melville et al (2004) proposed a model to assess internal and external factors that influence how business IT value can be generated using IT resources and complementary organisational resources of a firm under the impact of its trading partners’ processes and macro environment namely industry and country level. This study aims to explore the impact of factors identified in Melville et al (2004) framework during the implementation of an Enterprise Resource Planning (ERP) system in a SME ‐ a textile manufacturing firm in Vietnam. The structure of the paper is as follows. First, a brief review of challenges in creating business IT and RBV theory view in regard of sustained business IT value will be discussed. Second, Melville et al (2004) IT Business Value model will be explained in detail. The next section will be the research question(s) and overview of the firm. Research findings and discussion then follow. The paper concludes with key discussion points and suggestions for future work.

2. Challenges to deliver business IT to achieve sustainable organizational performance SMEs have recognised the increased importance of IT to their business performance. Several studies have revealed that SMEs, with the help of IT, are able to improve efficiency, increase productivity, provide customers with more personalized service in comparison to the larger business organizations and create closer relationship between the supplier and the customer (Chong et al 2001). However, SMEs also find challenges in delivering IT value to sustainable organizational performance (Moghavvemi et all 2012). In the past when IT was used mainly in a productivity improvement tool in small areas of a business, it was relatively straightforward to measure the IT value added to the organizational performance such as the ability to process more transactions per person (Smith and McKeen 2003). But nowadays, IT has grown in scope and complexity, it has become more difficult to measure what and how IT value is delivered. This is one of the challenges for SMEs when they do not clearly identify what is the value proposition of IT and how to implement it. According to Smith and McKeen (2003), the IT value has been increasingly viewed as a multi‐layered concept, which is illustrated in Figure 1 with 5 key questions in an “onion” layout. It suggests that before an IT value proposition can be identified, the nature of IT value itself needs to be examined by “peeling the onion” and answering the 5 key questions: (1) what value will be delivered?, (2) where value will be delivered?, (3) who will deliver value?, (4) when value will be delivered? and (5) how will value will be delivered?.

Figure 1: IT value is a multi‐layer concept (Smith and McKeen 2003)

3. Resource‐based view (RBV) and IT business value model Resource‐based view (RBV) has been widely used as a framework to guide businesses in their quest for achieving competitive business value from IT (Barney 1991, Lockett et al 2009, Newbert 2007). RBV is a theory that explains how a firm can achieve sustainable competitive advantages through its internal resources. RBV

190

Hiep Pham and Kevin Nguyen argues that the competitive advantage of an organisation greatly relies on the key resources owned by the organisation. Barney (1991) specifies four conditions required for a resource to create competitive advantage.

Valuable: the resource can enable a firm to improve efficiency or effectiveness

Rare: few firms can owe the resource, it confers a temporary competitive advantage

Imperfectly imitable: the valuable resource should not be easily imitated by lacking transparency the resource confers sustained competitive advantage

Non‐substitutable: there are no readily available substitutes for the valuable resource

The RBV’s main argument is that if a firm is to achieve a state of sustainable competitive advantages, it must acquire and control resources and capabilities to utilize them (Barney 2002). IT resources and the abilities to employ such resources are considered to be valuable and not totally imitable resources for an organisation. They have potential for creating sustained competitive advantage to organisations. To overcome the challenges in delivering IT value to organizational performance, organizations need to have a framework to create and measure the effectiveness of IT. Based on the RBV theory, Melville et al. (2004) developed an IT Business Value model which identified factors influencing the generation of strategic business value from IT. The model comprises of three domains: (1) Focal Firm, (2) Competitive Environment, and (3) Macro Environment. The model illustrates the relationship between IT resources and organization performance as shown in Figure 2.

Figure 2: IT business value model (Melville et al 2004) The first domain in the IT Business Value Model is the Focal Firm, which is an organization implementing IT resources. Within the focal firm, the model implies that IT resources, combining with complementary organizational resources may improve existing business processes or even create new ones that work with

191

Hiep Pham and Kevin Nguyen current resources and processes, thus it increases business process performance and as the end result, may improve the organizational performance (Brynjolfsson and Hitt 2000). The model suggests that IT resources include technology (TIR – technology IT resource) and human capital (HIR – human IT resource). TIR can be categorized as IT infrastructure, such as computing hardware or computing network, and IT business applications (or software) that utilize the IT infrastructure (Broadbent and Weill 2007). HIR are people who run, develop and manage the TIR resource. HIR’s technical and managerial expertise and knowledge are key factors that contribute to the IT business value (Sok and O’Cass 2011). Complementary organization resources in the model may include non‐IT physical capital resources, non‐IT human capital resources and organizational capital resources, such as organizational structure, rules and policies, workplace practises and culture, etc… (Barney 1991) According to the model, the firm achieve strategic business performance through improving business process performance. Business process is the specific order of work activities across time and space with a beginning and an end to transform a set of inputs into outputs (Davenport 1993). A firm uses many business processes to run its business, thus there are opportunities to use IT to improve the performance of business processes or create new processes that are more efficient to replace the ineffective processes. Business process performance brought by IT is the value added specifically to that process, for example improved cycle time in inventory management process. IT also can improve business process integration across physical and organizational boundaries to improve the firm performance (Basu and Blanning 2003). The second domain in the model is the Competitive Environment. It is the industry environment in which the focal firm operates. The model suggests there are two components in the competitive environment. The first component is industry characteristics such as competitive concentration, supply chain configuration, regulation, IT standards, anything that could impact how IT is used in the focal firm. The second component is trading partner resources and business processes which will also have impact on IT used in the focal firm. The IT resources that are able to integrate with the firm’s trading partners’ IT resources will have positive impact in the organizational performance. Macro Environment is the last outer domain in the model. It represents country even international factors that could shape IT application for the improvement of the organizational performance. These factors could be culture, government regulations, IT workforce, and availability of IT technology or IT infrastructure in the region. These factors could influence the decision on what IT resources to use and how to use them in the firm.

4. Case study in focus ABC Textile and Dyeing Joint Venture Firm (name of the firm has been altered for confidentiality purpose) has undergone considerable growth in recent years. Having become one of the largest manufacturers of Denim fabric in Vietnam market accounting for almost 50% of the Denim market in Vietnam, the company was now facing some organizational challenges. To cope with pressing competition in local and overseas markets, ABC needed to acquire internationally recognized certification for its quality control system and standardized manufacturing process. The company then implemented a quality management process to achieve ISO9000:2008 certification. There was substantial additional paperwork to be handled with the new process. ABC currently used an outdated decentralised computing system in managing its textile manufacturing process. It was time‐consuming and difficult to obtain timely and accurate production information including material planning, production costing, machinery inventory, production reports and sharing information among departments. Furthermore, without a centralized reporting system over a hundred different reports for various stages of the textile manufacturing process were generated and handled by several departments. The company recently decided to invest in the implementation of an Enterprise Resource Planning (ERP) system to ease the documentation tasks required by the ISO 9000:2008, and to manage more effectively the textile manufacturing process and to streamline the reporting system. It is reported that the company has been profitable and there is no urgent need of changing business direction or improving its bottom line. The first author was approached to assist with the investigation of ERP implementation to streamline its production and costing process and to centralise the production reporting system. After several discussions

192

Hiep Pham and Kevin Nguyen with the firm’s Deputy Director, it appeared that the management team of the firm was not clear of the potential strategic value from such IT investment and what IT resources that firm needed to generate long term business value. The main research objective is to understand the concerns and justifications of the firm in the decision to implement the ERP system given it is the largest and most complex IT investment so far of the firm. The IT Business Value Model discussed above was used as the framework for creating interview questions with the management of the firm. Three semi‐structured interviews were conducted with the Deputy Director (from now on referred as Ms Uyen) of the firm who made final decision in the implementation of the ERP system. In this study, the unit of analysis was the IT investment, a major one such as an ERP system. Using factors in Melville et al. (2004) to formulate main discussion questions, Ms Uyen was asked to assess the impact of each factor in her decision before and during the implementation of such a large IT investment. Interview responses from Ms Uyen were recorded and descriptive content extracted and analysed against relevant literature to provide useful insight.

5. Interview findings Table below describes key findings from the interviews and provide discussion in respect of relevant theories. Table 1: Key findings from the interviews and discussion in respect of relevant theories External factors (Macro and competitive domain) Country characteristics

The Deputy Director’s answers No obvious influence from national or international level to how the company should implement and use ERP system for business value creation.

Industry characteristics

ABC’s main competitor uses ERP system. If it is common in the industry that such IT system is used, there will be a pressure to use the same to remain competitive. Textile production process is quite complex and large amount of production information needs to managed properly. Currently it reaches a stage where some information cannot be generated due to required input not being stored and shared across departments.

Trading partner resources and business processes

Currently ABC does not have any integrated system with suppliers or customers. There is no clear impact from this factor.

It is critical to acquire technical and managerial competences for key staff to manage the new ERP system and for the future usage and maintenance of the system. The company is not certain what IT skills

IT capabilities have been recognised as key factor contributing to the generation of strategic business value (Doherty and Terry 2009). Cragg, Caldeira et al. (2011) developed 6 macro Information

Internal factors (Firm domain) IT Resources (Technology and Human)

193

Discussion Most studies in IT business value so far focus on organisational level (Doherty and Terry 2009; Cragg et al, 2011). It is desirable to gain better understanding of how macro factors can impact to the level of IT value creation approach at firm level. Some industries can be more regulated and standardised such as financial and banking services. Kankanhalli et al (2003) found that financial organisations are likely to invest heavily on Information Systems (IS) and rely extensively on IS for their business operations, compared to other organizations. It is also contributed to the industry is information‐intensive.

Hiep Pham and Kevin Nguyen External factors (Macro and competitive domain)

The Deputy Director’s answers required to fully utilise the ERP system once it is operational. A sense of uncertainty has caused some concern to Ms Uyen. As the company does not have in‐house IT expertise or staff, the company is concerned how the system can be developed to satisfy business needs. However, Ms Uyen stated that “lacking IT resources is not a major issue” and that the firm could employ as it was needed. Ms Uyen emphasises that “stable organisational structure, permanent staff force” was more critical factor to the decision to implement the ERP system

Complementary organisational resources (non‐IT resources dedicated to support such IT project and its operation)

Ms Uyen receives full support from the business owner to implement the ERP system. She needs that support to change the current business processes and the organisational culture to adapt with the new system. Staff resistance against the implementation of the ERP system has been found present from various staff groups. Especially from more mature, experienced staff who have been with the company for up to 20 years. ABC has been facing other problems in introducing ERP system which have been clearly identified in previous research including lack of strategic planning, shortage of investment capital in IT (Schubert 2003, Schubert and Leimenstoll 2004), no clear identification of roles and responsibilities of staff in using and managing the new system (Dutta and Evrard 1999), and unstructured business processes (Buonanno et al 2005).

194

Discussion System competences relevant to SMEs at organisational level including Business and IS strategic thinking, Define IS contribution, Define IS strategy, Exploitation, Deliver solutions, and Supply. These competences might give ABC a better starting point to acquire suitable skills to effectively generate business IT value. It is clearly evident that the firm currently does not have proper IT resources to fully benefit from the ERP system. Though the firm does not greatly concern about the lack of IT capabilities at this stage. This can be explained as ABC has not rely much on IT system in their business and low cost access to IT staff in Vietnam. This view is consistent with previous research suggesting that SMEs do not perceive IT linked to business strategy and command less business owner’s respect to properly acquire and utilise IT resources (Dojkovski et al 2007). The finding is consistent with previous studies where top management’s knowledge and attitude towards IT adoption together with organisation’s IT skills play important role in producing competences required to achieve sustainable business value from IT system (Caldeira and Ward 2003). Organizational processes are needed to exploit such skills to help them identify and realize IT opportunities in the organizations.

Hiep Pham and Kevin Nguyen External factors (Macro and competitive domain) Business Processes changed as a result of implementing such IT project

Performance Business process performance Organisational performance (expected business outcome of the project)

The Deputy Director’s answers Material ordering and inventory process have to be changed. Previously each order had to be signed and passed around several departments before the purchasing department could proceed with the purchase. Multiple versions of the order have to be archived manually. Data entry and report generation would change substantially with the introduction of new system. At least 4 data entry and report producing staff have to be made redundant. Real time monitoring of manufacturing process and order fulfilment now can be possible. Though the system is not yet fully implemented, the expected process performance will be in material planning, order fulfilment, information sharing among departments, less paperwork in inter‐ department communication process. Business process performance can be measured in term of accuracy, timely output, and ad‐hoc change to planned production. This factor has been the main reason company wants to introduce some forms of computerised system to replace the currently manual process. It had been frustrating to key staff to get figures from the current manufacturing process. Better control of expenses to focus on management task. It is initially not intended to save labour cost from the ERP system but to ensure timely and accurate control of all expenses incurred in every aspect of the manufacturing process. To free up resources from unnecessary areas and focus on quality control and enhance output quality. Timely and accurate information can be produced from the new system in response to constant change in the competitive market.

195

Discussion Organisational processes are needed to exploit IT skills and resources. RBV points out resources are only potential value, it is the firm’s capabilities including complex coordinated patterns of knowledge and skills embedded as firms’ processes to deploy its resources which create performance differentials (Cragg et al 2011).

Without long term view of the IT business value, the company more focused on the short term goal to improve business processes, rather than overall organisational performance. At ABC, the owner of the business commented at the firm’s effort to implement the ERP system “why to do all these things while the business is doing well right now”. As Ms Uyen expressed about the ultimate organisational performance expected from the ERP investment, “we just have to do it to see what happen. We can’t just wait and expect something would change eventually”. Ms Uyen is quite uncertain during the interview that how long it would take to realise return from the IT investment. Right not it is just incurring cost without clear sight of return. This is a common perception of unclear IT return. It usually takes at least three to five years to really deliver IT value (Seagars and Chatterjee, 2010)

Hiep Pham and Kevin Nguyen

6. Practical implication and future research The study has evaluated the use of Business IT Value model proposed in Melville et al (2004) in understanding how a firm assesses external and internal factors in determining the business value of an IT investment. By interviewing a Deputy Director of a textile manufacturing firm in Vietnam which is in the process of considering implementing an ERP system, it is found that macro domain (national level) and trading partners’ resources and processes do not have much impact to the decision on how to approach and generate business performance from the ERP system at ABC firm. The model is found lacking useful transition process from external environment to focal firm’s IT approach formation. Common to most SMEs, ABC does not have sufficient IT expertise and infrastructure to implement and manage its ERP project. IT resources including infrastructure, application, and IT skills are recognised essential to generate business performance from IT system. Though it is not clear to the firm that what particular Human IT Resources are needed to ensure the ERP system would be well managed and utilised. The firm does not regard lacking of IT resources a major concern to decide the IT investment. Other complementary organisational resources such as top management support, organisational culture, established business processes are found to be more important to whether to proceed with the IT investment. This finding emphasises the underestimate of IT roles in generating strategic business performance which has been found in previous research, especially in SMEs (Dojkovski et al 2007). Several business processes require changes to adapt with the ERP system. This is consistent with most theories in IT business value that emphasise business and IT alignment that support each other in achieving efficiency and effectiveness (Doherty and Terry 2009; Cragg et al, 2011). Improvement of some key business processes has been the main reason that ABC decided to invest in an IT system in the first place due to frustration of not being to control and forecast accurate production costing. The study has found the model proposed in Melville et al. (2004) enables the Deputy Director to be aware of how various factors that can shape the way IT system developed and used in her firm. Some factors such as IT resources and complementary resources are more obvious to influence its IT investment than the others such as national and partner’s resources and processes. This commands additional study to provide clearer linkage from macro factors to internal resources and approaches to business IT value generation. Especially, the model helps ABC to be in better position to achieve organisational performance through effective development and use of IT system. The study has some limitations. Study of one firm in one industry may not collect sufficient evidence to ensure the completeness of research findings. More respondents from the same firm would provide more comprehensive view of factors influencing the business IT value generation. Further research can be done to establish a clearer process to evaluate the transition from one domain to another which is proved difficult to apply in this case. Also a comparative study between industries might explore differences in the key factors in different industries.

References: Barney, J. B. (1991) Firm resources and sustained competitive advantage. Journal of Management, Vol. 17, pp 99‐120. Barney, J. B. (2002) Gaining and sustaining competitive advantage, Upper Saddle River, NJ: Prentice Hall. Basu, A. and Blanning, R. W. (2003) Synthesis and Decomposition of Processes in Organizations, Information Systems Research, Vol. 14, No. 4, pp 337‐355. Bharadwaj, A. S. (2000) A Resource‐Based Perspective on Information Technology Capability and Firm Performance: An Empirical Investigation, MIS Quarterly, Vol. 24, No. 1, pp 169‐196. Broadbent, M. and Weill, P. (2007) Management by Maxim: How Business and IT Managers Can Create IT Infrastructures, Sloan Management Review, Vol. 38, No.3, pp 77‐92. Brynjolfsson, E. and Hitt, L. (2000) Beyond Computation: Information Technology, Organizational Transformation and Business Performance, Journal of Economic Perspectives, Vol. 14, No. 4, pp 23‐48. Buonanno, G., Faverio, P., Pigni, F., Ravarinin, A.,Sciuto, D. and Tagliavini, M. (2005) Factors Affecting ERP Systems Adoption: A Comparative Analysis between SMEs and Large Companies, Journal of Enterprise Information Management, Vol. 18, No. 4, pp 384–426. Byrd, T.A. and Turner, D.E. (2001) An Exploratory Examination of the Relationship between

196

Hiep Pham and Kevin Nguyen Caldeira, M.M. and Ward, J.M. (2003) Using resource‐based theory to interpret the successful adoption and use of information systems and technology in manufacturing small and medium‐sized enterprises, European Journal of Information Systems, Vol. 12, pp 127‐141. Chong, H., White, R.E. and Prybutok. V. (2001) Relationship among organizational support, JIT implementation, and performance, Industrial Management & Data Systems, Vol. 101, No. 6, pp 273 – 281. Cragg, P., Caldeira, M. and Ward, J. (2011) Organizational information systems competences in small and medium‐sized enterprises, Information & Management, Vol. 48, pp 353‐363. Davenport, T. H. (1993) Process Innovation: Reengineering Work through Information Technology, Harvard Business School Press, Cambridge, MA. Dehning, B. and Richardson, V. J. (2002) Returns on Investments in Information Technology: A Research Synthesis, Journal of Information Systems Management, Vol. 16, No. 1, pp 7‐30. Doherty, N. F. and Terry, M. (2009). The role of IS capabilities in delivering sustainable improvements to competitive positioning, Journal of Strategic Information Systems, Vol. 18, pp 100‐116. Dojkovski, S.; Lichtenstein, Sharman; and Warren, Matthew J., "Fostering Information Security Culture in Small and Medium Size Enterprises: An Interpretive Study in Australia" (2007). ECIS 2007 Proceedings, Paper 120. Dutta, S. and Evrard, P. (1999) Information Technology and Organization within European Small Enterprises, European Management Journal, Vol. 17, No. 3, pp 239–251. Flexible IT Infrastructure and Competitive Advantage, Information & Management, Vol. 39, pp 41‐52. Hashim, J. (2007) Information Communication Technology ICT. Adoption among SMEs Owners in Malaysia. International Journalof Business and Information. Vol. 2, No. 2, pp 221‐240. Hitt, M.A., Ireland, R.D, Camp, S.M. and Sexton. D.L. (2001) Introduction to the Special Issue Strategic Entrepreneurship: Entrepreneurial Strategies for Wealth Creation, Strategic Management Journal, Vol. 22, pp 479‐491. Jin, N. (2007) A Study of Information Technology Adoption for Small and Medium Sized Enterprises Strategic Competitiveness, IEEE. 1‐4244‐1312. Lai, F., Zhao, X. and Wang, Q. (2006) The impact of information technology on the competitive advantage of logistics firms in China, Industrial Management & Data systems, Vol. 106, pp 1249‐1271. Lai, V.S. and Li, H. (2005) Technology acceptance model for internet banking: an invariance analysis, Journal of Information and Management, Vol. 42, pp 373‐386. Lockett, A., Thompson, S. and Morgenstern, U. (2009) The development of the resource‐based view of the firm: a critical appraisal, International Journal of Management Reviews, Vol. 11, pp 9–28. Melville, N., Kraemer, K. and Gurbaxani, V. (2004). Information Technology and Organisational Performance: An Integrative Model of IT Business Value, MIS Quarterly, Vol. 28, pp 283–322. Moghavvemi, S., Hakimian, F. and Feissal, T. (2012) Competitive Advantages Through IT Innovation Adoption by SMEs, Social Technologies, Vol. 2, No. 1, pp 24‐39. Newbert, S. (2007) Empirical research on the resourcebased view of the firm: an assessment and suggestions for future research, Strategic Management Journal, Vol. 28, pp 121–146. Schubert, P. (2003) Personalizing e‐Commerce Applications in SMEs, 9th Americas Conference on Information Systems, 4–6 August, Tampa, Florida, pp. 737–750. Schubert, P. and Leimenstoll, U. (2004) Personalization of e‐Commerce Applications in SMEs: Conclusions from an Empirical Study in Switzerland, Journal of Electronic Commerce in Organizations, Vol. 2, No. 3, pp. 21–39. Segars, A.H. and Chatterjee, D. (2010) Diets that don’t work: Where Enterprise Resource Planning goes wrong. Wall Street Journal, Available online at http://online.wsj.com/article/SB10001424052748703514404574588060852535906.html Smith, H.A. and McKeen, J.D. (2003) Developments in Practice VII: Developing and Delivering the IT Value Proposition, Communications of the Association for Information Systems, Vol. 11, pp 438‐450. Sok, P. and O’Cass, A. (2011) Achieving superior innovation‐based performance outcomes in SMEs through innovation resource‐capability complementarity, Industrial Marketing Management, Vol. 40, pp. 1285‐1293. Valacich,J. and Schneider,C. (2010) Enhancing the motivational affordance of information systems: The effects of Real‐Time performance feedback and goal setting in group collaboration environments, Journal of Management science, Vol. 56, No. 4.

197

Usability Testing for an Evaluation of Application‐Learning Tutorials and Document‐Based Instructor Nantika Prinyapol Department of Information Technology for Modern Enterprise, Faculty of Information Technology, Dhurakij Pundit University, Bangkok, Thailand nantika.pri@dpu.ac.th Abstract: In recent years, the demand for the development of three‐Dimensional (3D) animation projects and augmented reality (AR) applications have quickly grown. Programmers who are both able to code and possess the ability for creating 3D artworks are rarely found. It has been suggested that some of the major factors that contribute to this include the need for additional graphical skills and thoughts. In addition, many of 3D software are challenging for the programmers to control the 3D views by different mouse movements in the specific buttons. There are numerous of particular technical terms and complicate menus. That is not easy to understand how to use those multimedia applications for new users in a short period of time. Development of an effective learning system for 3D software needs to be evaluated. The suitable learning tutorial system was expected to help the users by self‐learning for easily using 3D software. This study evaluated the usability of an application multimedia tutorial compare with an instructor manual document. The graphical application that was tested in the study was ‘Autodesk Maya 2011’, 3D animation software for creating visual effects, game development, and 3D animation projects. This research involves heuristic evaluation and observation to the non‐ experience Maya‐users when creating simple 3D object. Comparing between a group that use the multimedia learning movies of Maya, and another group uses the instructor manual to guide ‘How’ to create the same 3D object. The evaluation components that were evaluated are the rate of errors, completed time, speed of user self‐learning, and learning ability. The results showed that the instructor documents were able to increase a speed of learning and reduce errors. However, the learning movies were more applicable for the further tasks. The usability test also showed that the step‐by‐step document is practical for learning 3D software in both novice users and computer‐skilled users. Keywords: learning system, self‐learning, application tutorial, usability test, and evaluation

1. Introduction Many authors (Baharudin, et al., 2011; Minocha and Hardy, 2011; Miller, 2012) considered about developing the 3D multimedia require the graphical‐experienced users or an efficiency self‐learning system for the novice users. The learning tutorial is a help system that assists and guides the apprentice users to self‐understandable to 3D tools and medias. Currently, the demand for developing 3D animation and AR applications has been grown. 3D models are the most popular and widely used in the multimedia web‐based application. Furthermore, 3D multimedia aids to develop the e‐learning systems and create the 3D virtual learning environment (Arhippainen, et al., 2011; Minocha and Hardy, 2011). Valcheva et al (2010) supported that when using 3D models in the e‐learning materials was able to increase effectiveness and personalization of e‐ learning services. To employ 3D applications, programmers require the specific skills and experiences in graphics design. However, creative thoughts are not the common skills of the general programmers. Programmers who are both able to code and have abilities for producing 3D application are rarely found. It is very difficult for typical programmers to create desired 3D models (Miller, 2012). In addition, working with 3D software has challenge the programmers because they have to control the 3D views with specific hand‐gestures. In common, programmers are not memorable when controlling unfamiliar mouse movement and sometimes they require using a mouse button together with some specific keyboard buttons. For example, when we clicked the middle mouse button together with the Alt button, it means panning or tracking the 3D views of Autodesk Maya, or pressed the Q‐W‐E‐R‐T‐Y key on keyboard for manipulating the desired 3D objects. These are very weird gesture skills for programmers who generally not familiar with that unusual hand movement. There are several 3D applications such as Autodesk 3ds Max, Autodesk Maya and LightWave 3D. They have different in shortcut menus and commands. The main reason to select Autodesk Maya as the application used within this study is that the complexity of Maya menu and new to the participants. The remainder of this paper is structured as follows: Section 2 presents the user learning experiences in 3D virtual environments. Section 3 presents the usability inspection technique for testing and evaluating the

198

Nantika Prinyapol study. In Section 4, the evaluation results are presented and discussed. Finally, conclusions and comments on future work are given in Section 5.

2. User learning experiences in 3D virtual world Many industrials nowadays (e.g. commerce, education, entertainment, real‐estate) all use 3D virtual applications to increase the consumer attentions. Successfully 3D applications combine the art with interactive controls. In this section defines the personalization of learning 3D virtual environment and multimedia applications. Numerous works have shown that the effectiveness of 3D usability depends on the better visualize tutorial system (Krull, et al., 2001; Duan and Zhang, 2007; Fernandez, et al., 2012). In commonly, users have three basic individual learning styles to process information to memory (Valcheva, et al., 2010):

Visual, learning by seeing (e.g. demonstrations, movies)

Auditory, learning by hearing (e.g. oral instructions, video with voice, instructions)

Kinesthetic, learning by doing

The effectiveness of learning content is depended on the freedom in choosing learning materials that relevant to learning styles of users. Currently, several 3D applications, for example Autodesk Maya and Autodesk 3DS Max, contained the learning movies in their help systems. Those self‐learning movies provide the general lessons to help users understanding how to create 3D models by using their applications. The lessons, such as controlling 3D view by keyboard shortcut, mouse‐button and scroll movement, are different between.

2.1 Application‐learning tutorials The visualization of the tutorial movies show very clearly lessons (Crozat, et al., 1999). Users can able to repeat the lessons by themselves with the multimedia documents provided by the software itself (Doyle et al, 2007). In a long term, users should understandable and usable the 3D application after interpreting digital document. Visual and sound documents, including textual documents in some explanation parts, are the constituents of the learning movies in the software. In Autodesk Maya 2011 (http://usa.autodesk.com/maya), the application‐ learning tutorials are provided in the initial pop‐up dialog (Figure 1) after launch the application or are showed in the menu of “help > learning movies”. There are seven lessons in this application as following:

Zoom, pan and roll: Navigation essentials,

Move, rotate, scale,

Create & view object,

Component selection,

Discover secret menus,

Keyframe animation, and

Material, Lights and Rendering.

In this study, only four lessons have been used to complete the tasks. However, the audio tracks are limited in some languages only such as English, Chinese, and Japanese. There is no Thai language supported in Autodesk Maya 2011. Because of the limited audio tracks, Maya 2011’s users should have English language skill.

2.2 Document‐based instructors The heuristic evaluation (HE) is conducted in the preparation for this instructor sheet. HE is one of the population evaluation methods within Human‐Computer‐Interactive (HCI) that can prevent errors and help users recognize, diagnose, and also recover from errors (Conte et al, 2007; Conte et al, 2009). The objective of this material was to design guidelines and usability standards to launching usability testing with end‐users by inspectors (experts) guidance. According to Al‐Wabil and Al‐Khalifa (2009) framework of integrating the

199

Nantika Prinyapol usability evaluations are used within the usability test sessions. End‐users were inspected and observed by the testing inspector. In this study, the document‐based instructors provided a sequence of twelve tasks for users to create a 3D chair model in two hours (as shown in Figure 2). The observation investigates the users’ activities when using instructor sheets. Think‐aloud protocol and interview also help to provide the relevant feedbacks for evaluating the usability test. If the participants satisfy and understand the document‐based instructor, the task should complete more easily. The participants’ comments, remarks and questions are reduced. Nonetheless, speed of completeness 3D model is depended on how fast the users can understand their 3D perspective viewpoints.

Figure 1: Learning movies dialog in Autodesk Maya 2011

Figure 2: The instructor sheet with twelve tasks to create a 3D chair

3. Usability test and evaluation As multimedia technology advances, it becomes more complicate to develop and use 3D applications for 3D virtual projects, each of which individually required the practical multimedia programmers. Furthermore after AR and virtual e‐learning are grown for a certain period, programmers cannot let pass to learn and use 3D software. Several programmers complain about current 3D applications with hardly level of usability. In this study, the usability testing (Douglas, 2006) and heuristic evaluating (Anandhan, et al., 2006; Al‐Wabil and Al‐ Khalifa, 2009; Conte et al, 2009; Forsell and Johansson, 2010; Vargas, et al., 2011) have been integrated and conducted. In order to make the effective learning system for 3D application are great importance in learning tutorial development. The usability tests of learning systems focus on Thai‐users as a user‐center development.

200

Nantika Prinyapol

3.1 Characteristics of the evaluation system Document‐based instructor provides twelve sequential tasks that instructed by the experience Maya users. Learning movies of the application have seven basic steps of how to use Maya 2011, but only four movies that relevant to employ the model. The learning movies also require listening skills of English language. In the testing, there are forty‐eight participants who generally had at least a little experience with computer technology and with English language skills. Half of participants are a group of professional users who use graphical software in general. These users came from the Dhurakij Pundit university community (age between 25 and 53 years old), and all of them are Thai nationalities. Within two hours, the users required to finish the chair model by using Maya. The laboratory of Faculty of Information Technology has been used for the test. During the usability testing, there are three examiners observing and collecting data. They have recorded every user’s interactions and assisted users to fix the troubles.

3.2 Case study with IT student The basis of this research is to study the efficiency and effectiveness of self‐learning in 3D application of the Master degree students. And, hypothesis of this paper exists that the multimedia‐experienced users are capable to learn 3D application than general end‐users in both two types of self‐learning tools that are application‐learning movies and instructor’s sheets. In the study, the forty‐eight students of Master of Science in web engineering program were divided into two groups (twenty‐four each). The first group (batch 4th) does the usability test on the 1st semester in year 2011 and the second group (batch 5th) does the unchanged test on the 1nd semester in year 2012. The members in each group have different computer‐skills and multimedia experiences. For example, there were the computer graphic designers, multimedia users, programmers, and general end‐users. Moreover, the participants were also separated by listening skills in English.

Multimedia

Table 1: Categories participants to group A‐H by their skills and experiences Participants General Users Experienced Users Total

Computer‐skilled Novice Users Skilled Users A (3) E (3) B (3) F (3) C (3) G (3) D (3) H (3) 12 12

Total 12 12 24

The twenty‐four participants in each batch are divided into two groups that are twelve of novice users and twelve of skilled users as shown in Table 1. It was depended on different skilled levels in computer backgrounds. For example in computer skilled, the general users, who use just routine applications, e.g., MS office, were grouped in novice users; and practical users, who use particular software in their works and also the programmers, were grouped in skilled users. On the other hand, the participants were also separated by the experiences in using multimedia applications. General multimedia users group included users who have none or less skilled with graphical software. Then Experience multimedia users were the users that have skilled in using the graphical instruments for designing 2D and 3D works. Half of each four sets of users were fluent in listening English (for better understanding the learning movies). In addition, all of the participants are the Maya beginners who certainly not use Maya previously. In the study, all participants had to create a simple 3D chair model (as shown in Figure 3) in Autodesk Maya 2011 for two hours. The evaluation provided two different styles of instructor materials for helping participants to employ the 3D chair. In the usability test, the accepted chair model was a likeable 3D chair model. The size can be different from the original, although, the geometry shape should be the same, which is the cube.

201

Nantika Prinyapol

Figure 3: A chair model as extracted from Maya Evaluation Tasks: Create a 3D chair model (as shown in Figure 3). Time: Two hours. The average time of skilled users to complete this chair model is 3‐5 minutes, though the novice users need more time to understand the instructor(s). Then, the test gave users two hours to complete the chair model, partly to assure that they survived the experience without undue pressure. Instructor materials: Have two different types as the follows:

A multimedia application‐learning tutorial, learning movie, of Maya 2011 (as shown in Figure 1.), is provided to participants A, C, E, and G as shown in Table 1.

A document‐based instructor, instructor sheets (as shown in Figure 2), is provided to participants B, D, F, and H as shown in Table 1.

Novice users expect to understand the simpler instructor, which help them easy to learn how to use the Maya application. Expert users that are familiar with 2D and 3D applications should able to use Maya faster. Learning system for multimedia application has great impact on students’ attitude (Srivastava, 2012). All participants have to create 3D chair by themselves using self‐learning instructor. During two hours, the speed of completed model and number of errors and mistakes are collected. Three monitors observed all the work progresses of each participant and supported the students when they need helps. The evaluation results have been explained in Section 4.

4. Evaluation results As introductions in the first parts of this paper, efficiency self‐learning material will increase the user abilities for using 3D applications. When using evaluation methods involving user participation, usability problems are discovered by means of observation the behaviors of the users, while they perform tasks or request assistance about application usability. Comparison between two material styles, users that participated with the application‐learning tutorial required more English skills than the users that participated with document‐based instructors. As a result, the learning abilities of participants that use English instructor would minus 0.3% extra time to the completed period. Table 2 shows the usability test results of this study. It defines the time to complete a task in average of forty‐eight participants, number of errors made in average of each participants, and amount of assistance requested of each participants to the examiners. Including, it shows the perception and attitude of users with the learning materials after users completed the task by observing and interviewing them.

202

Nantika Prinyapol

Figure 4: Usability laboratory environment Table 2: Usability test results analysis (in average) Participants Usability Specification General (multimedia) users Time to complete a task in minutes Number of errors made Amount of assistance requested Attitude after completing a task Expert (multimedia) users Time to complete a task in minutes Number of errors made Amount of assistance requested Attitude after completing a task

Novice users Document‐ Learning based movies instructor A B 118 107 8 2.5 6 4 Neutral Positive C D 110 106 5 2 2 2 Neutral Positive

Computer‐skilled users Document‐based Learning movies instructor E 112 7 5.5 Neutral G 109 4 2 Highly Positive

F 98 2 3 Neutral H 83 1 0.5 Highly Positive

In this study, usability evaluation comprises two quality criteria ‐ efficiency and effectiveness

Efficiency: the ratio between the number of errors made and the time spent in complete task.

Effectiveness: the ratio between the numbers of errors made and the total number of errors.

The total number of errors was summarized with the number of errors and it was spited in to the group of general users (A, B, E, F) and expert users (C, D, G, H). Table 3 shows the efficiency and effectiveness ratios of errors and mistakes of all groups separated by user skills in multimedia applications. As the results, it has shown that users, who have computer and multimedia backgrounds, have more efficiency and effectiveness to handle the errors and have more abilities in self‐learning for the 3D application. The observation found that the first user who completed the 3D task was group F, as shown in Table 4. The users use only 61 minutes in completed the task. And after interviewing the participant, the results show that the more understand 3D perspective viewpoints, the less time to complete the task. The major mistake in this study was the users repeatedly put the geometry shapes in wrong positions. When moving camera viewports, objects (cubes) did not touch to another cubes. It seemed to be a chair only viewing in one side perspective. The reason was that the users did not understand the 3D perspective and they did not

203

Nantika Prinyapol rotate the 3D view control to see the whole model. Inexperience users requested the assistance from the inspectors to continue their tasks. Table 3: Efficiency and effectiveness ratios of the error rate Groups

Efficiency

Effectiveness

0.068

0.410

0.023

0.128

0.045

0.417

0.019

0.167

0.063

0.359

0.020

0.103

0.037

0.333

0.012

0.083

Table 4: Minimum, maximum speeds and the number of incomplete users Groups A B C D E F G H

Minimum (minutes) 92 82 83 73 80 61 77 66

Maximum (minutes) 132 120 125 123 136 120 137 93

Incomplete Task (person) 2 ‐ 2 2 1 ‐ 1 ‐

Out of forty‐eight participants, there were forty students that able to finish the 3D chair model on time. Other eight participants finished the tasks later after 120 minutes passed. To interview unfinished users, they mentioned that listening to English version of learning movies took a lot of times. The tutorial movies explain the general information of how to create 3D objects in Maya 2011, thus users need to design the 3D chair by their own ideas. Table 4 shows that some users in group A, C, E, and G (who used application‐multimedia movies as the help learning system) were not able to complete the tasks in time. On the other hand, instructor sheets provided the steps‐by‐steps of how to create the 3D chair model. When participants use the instructor sheets, they can finish the assigned task easily. The novice users (group D), who common use multimedia software for their daily behaviors (e.g. edit photos for uploading to social network), explained that even they are not complete the tasks within two hours, but it was not surprised. They did not expect that they could use 3D application at first. However, after practicing for a while, their attitudes with the 3D application were changed. Uses who use document‐based sheet said that the easily explanations in the sheets helped them to create the 3D chair in very short times. In conclusions, the simply learning systems are acceptable and satisfy the outcomes for non‐experience users. The attitudes of the novice users were neutral to highly positive levels.

5. Conclusions and further studies The results of the study showed that the instructor documents were able to increase speeds of learning and reduce the error rates. The novice users who used the document‐based instructors can finish their tasks faster than the users who used another tutorial type. The reason was that within limited of time, users need the step‐by‐step instructor to assist them to complete the particular tasks. Comparing between novice and expert users of the multimedia application (2D and 3D), expert users have more capabilities to create the 3D models. However, the self‐learning movies are acceptable and more appropriate for the advance tasks, to create another freeform 3D models or special 3D characters. The results indicated that participants in the study enjoy

204

Nantika Prinyapol the use of the multimedia as the self‐learning system and believe that it helps to provide additional opportunities for further study of the career paths. The usability test also showed that the step‐by‐step document help the users to implement 3D applications more easily. While the step‐by‐step instructor system has proven to be useful for users self‐learning, the e‐learning version should be developed. Future work entails the fully development of different lessons of 3D application and further studies comparing the learning techniques for evaluate 3D applications usability.

References Al‐Wabil, A. and Al‐Khalifa, H. (2009) “A Framework for Integrating Usability Evaluations Methods: The Mawhiba Web Portal Case Study”, In: Proceeding of Current Trends in Information Technology (CTIT), December 15‐16, 2009, pp 1‐6. Anandhan, A., Dhandapani, S., Reza, H. and Namasivayam, K. (2006) “Web Usability Testing – CARE Methodology”, In: rd Proceeding of the 3 International Conference on Information Technology: New Generations, April 10‐12, 2006, pp 495‐500. Arhippainen, L., Pakanen, M., Hickey, S. and Mattila, P. (2011) “User Experiences of 3D Virtual Learning Environment”, In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environment, ACM digital library, Tampere, Finland, pp 222‐227. Baharudin, S., Ismail, M., Ahmad, S. and Dahalan, N.M. (2011) “Evaluating the Usability and Acceptance of Multimedia Web‐based Education Among Children”, In: Proceeding of International Symposium on Humanities, Science & Engineering Research (SHUSER), June 6‐7, 2011, pp 46‐49. Conte, T., Massolar, J., Mendes, E. and Travassos, G.H. (2009) “Web Usability Inspection Technique Based on Design Perspectives”, IET Software, Vol 3, No. 2, Appril 2009, pp 106‐123. Conte, T., Massollar, J., Mendes, E. and Travassor, G.H. (2007) “Usability Evaluation Based on Web Design Perspectives”, In: st Proceeding of the 1 International Symposium on Empirical Software Engineering and Measurement (ESEM), September 20‐27, 2007, pp 146‐155. Conte, T., Mendes, E., Vaz, V., Massolar, J. and Travassos, G.H. (2009) “Improving a Web Usability Inspection Technique Using Qualitative and Quantitative Data From an Observational Study”, In: Proceeding of the 13th Brazilian Symposium on Software Engineering (SBES), October 5‐9, 2009, pp 227‐235. Crozat, A., Hû, O. and Trigano, P. (1999) “A Method for Evaluating Multimedia Learning Software”, In: Proceeding of the IEEE International Conference on Multimedia Computing and Systems (ICMCS’99), Vol 2, p 9714. Douglas, I. (2006) “Collaborative International Usability Testing: Moving From Document‐based Reporting to Information Object Sharing”, In: Proceeding of International Conference on Global Software Engineering (ICGSE’06), October 2006, pp 114‐118. Doyle, J., Viktor, H.L. and Paquet, E. (2007) “Long Term Digital Preservation – An End User’s Perspective”, In: Proceeding of nd the 2 International Conference on Digital Information Management (ICDIM’07), October 28‐31, 2007, pp 146‐151. Duan, J. and Zhang, N. (2007) “Research on Visualization Techniques for Web Usability Analysis”, In: Proceeding of the 8th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (SNPD), Vol 2, July 30, 2007, pp 788‐791. Fernandez, A., Abrahão, S. and Insfran, E. (2012) “A Systematic Review on the Effectiveness of Web Usability Evaluation Methods”, In: Proceeding of the 16th International Conference on Evaluation & Assessment in Software Engineering (EASE), May 14‐15, 2012, pp 52‐56. Forsell, C. and Johansson, J. (2010) “An Heuristic Set for Evaluation in Information Visualization”, In: Proceedings of the International Conference on Advanced Visual Interfaces (AVI’10), ACM digital library, Rome, Italy, May 25‐29, 2010, pp 199‐206. Krull, R., Friauf, J., Brown‐Grant, J. and Eaton, A. (2001) “Usability Trends in an Online Help System: User Testing on Three Releases of Help for a Visual Programming Language”, In: Proceeding of Professional Communication Conference (IPCC 2001), IEEE International, pp 19‐26. rd Miller, J.R. (2012) “Metaview: A Tool for Learning About Viewing in 3D”, In: Proceedings of the 43 ACM Technical Symposium on Computer Science Education (SIGCSE’12), ACM digital library, Raleign, North Carolina, USA, February 29‐March 3, 2012, pp 135‐140. Minocha, S. and Hardy, C.L. (2011) “Designing Navigation and Wayfinding in 3D Virtual Learning Spaces”, In: Proceedings of rd the 23 Australian Computer‐Human Interaction Conference (OzCHI’11), ACM digital library, Canberra, Australia, November 28-December 2, 2011, pp 211-220. Srivastava, S. (2012) “A Study of Multimedia & Its Impact on Students’ Attitude”, In: Proceeding of IEEE International Conference on Technology Enhanced Education (ICTEE), January 3‐5, 2012, pp 1‐5. th Valcheva, D., Todorava, M. and Asenov, O. (2010) “3D Model of E‐Learning”, In: Proceedings of the 11 International Conference on Computer Systems and Technologies (CompSysTech’10) and Workshop for PhD Student in Computing on International Conference on Computer Systems and Technologies, ACM digital library, Vienna, Austria, June 16‐17, 2011, pp 359‐364. Vargas, A., Weffers, H. and da Rocha, H.V. (2011) “Discovering and Analyzing Patterns of Usage to Detect Usability Problems in Web Applications”, In: Proceeding of the 11th International Confernece on Intelligent Systems Design and Applications (ISDA), November 22‐24, 2011, pp 575‐580.

205

Using PRS to Enhance Student Learning Process: The Application of Revised Study Process Questionnaire‐2 Factors Ondris Pui1, Narumon Sriratanaviriyakul2, Wei Wei 3, Patrick Then4 and Zia‐Yee Tho5 1 Centre of Communication and Design, RMIT University Vietnam, HCMC, Vietnam 2 Centre of Commerce and Management, RMIT University Vietnam, HCMC, Vietnam 3 Learning Skills Unit, RMIT University Vietnam, HCMC, Vietnam 4 School of Engineering, Swinburne University of Technology, Kuching, Malaysia 5 School of Business and Design, Swinburne University of Technology, Kuching, Malaysia ondris.pui@rmit.edu.vn narumon@rmit.edu.vn wei.wei@rmit.edu.vn pthen@swinburne.edu.my ztho@swinburne.edu.my Abstract: Based on Biggs et.al (2001), this study investigated the extent of how Personal Response Systems (PRS) can influence students’ learning experiences in an undergraduate classroom for deep and surface learning. The experiment involved 40 students in Bachelor of Design (Multimedia Systems) at RMIT University Vietnam. The topic presented was Colour Theory. 99% of the participants were local students who used English as second language. The Study Process Questionnaire circulated during PRS sessions were used to inquire into student motives and strategies in achieving their learning outcomes. Factor analysis identified 4 underlying factors: Deep Motive, Surface Motive, Deep Strategy and Surface Strategy. The quantitative analysis suggested that there was a positive relationship between Deep Approach (DA) and Surface Approach (SA) with the four sub‐scales, Deep Motive (DM), Deep Strategy (DS), Surface Motive (SM), and Surface Strategy (SS). In other words, PRS appears to enhance, improve and facilitate students’ learning process by promoting deep motives and strategies. This finding was further validated with data gathered from focus group interviews. 10 participants had been invited to participate in focus group interviews. The prompts of the semi‐structured interview include PRS classroom experience, motives of using PRS and the strategies of using PRS in classroom. Special attention had been given attention to the analysis of the student’s role, lecturer’s role and the interaction between them. The findings from the interviews supported the quantitative analysis and suggested that the traditionally perceived teacher roles, learner roles and the interactions between teachers and learners in the classroom had been transformed tremendously when PRS was used as a teaching pedagogy. In details it suggested that PRS has assisted students in increasing their concentration, participation and contribution in classroom discussion. It also shows that lecturers change their roles from knowledge delivery to classroom discussion facilitators. Further pedagogy implications are also discussed. Keywords: clicker, learning process, learning outcome, active learning

1. Introduction The use of Personal Response Systems (PRS) has been widely investigated in many fields and disciplines, which includes physics, chemistry, education, statistics, psychology, human development, nursing, family and consumer sciences, biology, engineering, and accounting (Ramsay, 2006). The application of this tool in classrooms has attracted much attention because of its significant influence on students’ learning process, which in turn, enhanced their learning outcomes as well as promote an active learning environment (Camey et al. 2008; Gauci et al. 2009; Patterson et al. 2010 ). Biggs’ (2001) classification of learning processes using two scales of learning approaches: surface learning and deep learning, has been considered as the critical factors affecting students’ learning outcomes and performance in traditional classrooms. Gijbels (2005) has applied Biggs’ (2001) theory of classifying surface learning and deep learning into his studies and successfully concluded that it is feasible for students to achieve a higher approach for deep learning than surface approach. To further investigate, the investigation will focus on the effectiveness of PRS as a teaching pedagogy, and attempts to answer the following questions:

To what extent does PRS facilitate or inhibit students’ approach learning in classrooms in terms of their motivations and strategies?

What are the possible reasons for this?

206

Ondris Pui et al.

2. Literature review 2.1 The impact of PRS on learning process PRS is not only a tool that is flexible, and easy to use (Britten 2011) but it also enhances students’ achievement and promote active learning with many advantages in their functions and characteristics such as, allowing students to respond anonymously (Hinde and Hunt 2006), providing immediate feedback that help increases faculty‐student interaction (Revell and McCurry 2010) as well as producing feedback for presenters, assessors and educators (Beringsa 2010). In addition, it enables creative activities, cooperation and conceptual application (Hoekstra and Mollborn 2008). Previous studies reported a positive relationship between the use of PRS in the classroom and the levels of student participation, student engagement, and learning outcomes, although a relatively small numbers of studies suggested negative results. Nevertheless, there is a lack of studies evaluating the influence of using PRS on students’ learning approaches as well as their learning process. This study attempts to understand the influence of using this technology on students’ learning approach based on Biggs et al. (2001) Revised two‐factor Study Process Questionnaire: R‐SPQ‐2F in a Multimedia Design course. Previous studies have provided various evidences about the effectiveness of using PRS on learning processes. From the students’ perspective, PRS helped to motivate and promote their learning process, for instance, students rated lectures using clickers as more interactive, fun, interesting, entertaining, intellectually stimulating, helpful in understanding how people perceived subjects (Shaffer and Collura 2009; Fifer 2012). The past investigations also showed that using clickers for questioning in class helped students to be more engaged with materials, made them more aware of their misunderstandings on course concepts (Hinde and Hunt 2006; Trees and Jackson 2007). Because of all these significant influences on students’ attitude and their engagement in class, PRS is considered as pedagogy to promote active learning and possibly provide immediate feedback to faculty and students (Revell and McCurry 2010; Fifer 2012) in the classrooms. Generally, the greater the volumes of clicker usage, the more favourable student perceptions were in terms of active learning, motivation, and providing feedback which in turn influence their learning outcomes (Nelson and Hauck 2006). This approach will bring higher levels of students’ satisfaction (Beckert et al. 2011). Presumably, the feature of anonymity brought by PRS to the classrooms had a huge potential to transform traditional teacher‐centered classroom to a more learner‐centred environment where students’ engagement and contributions to the classroom activities are highly appreciated. Besides the positive influences of PRS on learning processes, other studies have found evidences of limitations with no significant difference of improvement when using this tool compared to traditional pedagogy in classroom. Although students can participate anonymously, one feels "negatively reinforced" to study more to avoid future embarrassment for answering incorrectly in future (Britten 2011). PRS has been known to be more effective at reviewing areas that were weak. It, however, neither increases the feelings of hope, nor reduces the feeling of anxiety (Fallon and Forrest 2011).

2.2 The impact of PRS on learning outcomes Students of Human Research course found that their examination results improved in comparison to their previous semesters and had rated the use of the PRS in lectures favorably (Gauci et al. 2009; Mayer et al. 2009; Gagne 2011). However, Patterson et al. (2010) found that clickers did not improve learning outcomes as measured by learning objectives achieved in nursing classes. Most of the evidences showed that PRS could be used to create an active learning environment and promote students engagement but it did not always motivate students to achieve better results in final exams. From the instructors’ perspective, PRS seemed to be an effective pedagogy to create efficient and dynamic classes. Particularly, with its immediate feedback feature, which enables instructors to adapt their lectures to address areas of students’ misunderstanding of the concepts (Ramsay et al. 2006), perform instructional interventions (Murphy 2012), as well as manage peer and class discussions more efficiently. Christopherson (2012) suggested some practical methods to improve this pedagogy in class, through the use of questions that attempt to probe at analysis, synthesis, and evaluation. Despite the facilitation of PRS to support active learning, he also suggested that PRS should not be viewed as a magic bullet to improve students’ experience; rather, more focus should be placed on how instructors can improve their pedagogy through the use of this

207

Ondris Pui et al. tool. Furthermore, instructors should consider students’ performance on standardized tests rather than evaluating learning outcomes using only in‐class assessment instruments (Camey et al. 2008).

2.3 Students’ learning approach and the application of the study process questionnaire in PRS classrooms Biggs previously measured students’ learning process using deep and surface learning approaches. Deep approaches require higher order factor analyses usually associated with motives and strategies in achieving a desired learning outcome (Biggs 1987). Surface approaches, on the other hand, are dependent on subjects taught, teaching conditions, and sometimes achieving‐related scores load that are part of assessment rubrics (Biggs 1984). By using PRS in classrooms compared to traditional methods, the teaching environments and pedagogy encourage active learning, boosting significant positive effects on students’ learning processes and outcomes. The application of Study Process Questionnaire (SPQ) was used to investigate learning processes in both deep and surface approaches in PRS classroom, enquiring into the motives and strategies applied by students to achieve learning outcomes. This is the critical purpose of this study. The Study Process Questionnaire (SPQ) (Biggs 1987) was developed from an earlier 10‐ scale Study Behaviour Questionnaire (SBQ). Biggs et al. (2001) defined deep and surface factors each with ten items. Within each of these two factors, it was possible to distinguish the sub‐scales of students’ strategies and motives. Each of the sub‐scales consisted of five items. The final version of the questionnaires therefore has two main scales: Deep Approach (DA) and Surface Approach (SA) with four sub‐scales, Deep Motive (DM), Deep Strategy (DS), Surface Motive (SM), and Surface Strategy (SS). Deep approach is when students would be expected to engage in the highest level learning activities and thus, to handle the task or solve problems effectively with learnt knowledge. While a student using a surface approach would use lower order processes such as memorising. A particularly depressing finding is that most students in undergraduate courses are increasingly achieving surface learning level instead of achieving deep learning level in their studying (Watkins 1986; Biggs 1987; Gow and Kember 1990). In fact, motives and strategies in gaining learning outcomes had a different relationship with the deep and surface learning from the outset (Biggs 1987). Deep and surface strategies describe how the students organise when and where the task will be engaged, and for how long.

3. Methodology The original Study Process Questionnaire (Biggs 1987) was developed from an earlier 10‐ scale Study Behaviour Questionnaire. Biggs et al. (2001) developed the revised two‐factor Study Process Questionnaire (R‐SPQ‐2F) which has fewer items which deep and surface approach scales further identified students’ motives and strategies. In attempt to make clear the question “To what extent PRS facilitate or inhibit students’ approach learning in classrooms?”, the R‐SPQ‐2F was adopted to measure the deep and surface learning as well as learning outcomes in PRS classroom. Two specific questions will be asked: “To what extent PRS facilitate or inhibit students’ approach learning in classrooms in terms of their motivations and strategies? Why?”

3.1 Participants The experiment involved 40 students in Bachelor of Design (Multimedia Systems) at RMIT University Vietnam. The topic presented was Colour Theory. 99% of the participants were local students who used English as second language.

3.2 Experiment design Design students were taught Colour Theory and the PRS were used in facilitating lectures. The length of a lecture session is 2 hours spanning across 12 weeks for the course in Centre of Communication and Design, RMIT Vietnam. The students were introduced and trained in the use of PRS during the first two weeks of semester, and its usage was monitored in week 3. Students were presented with slides presentation as an aid in teaching and comprehension was tested after each topic. The comprehension questions were designed based on Bloom’s Taxonomy keywords to classify learning objectives. Bloom identified six levels of learning within the cognitive domain, from the simple recall or recognition of facts, through increasingly more complex and abstract mental levels, to the highest order which is classified as evaluation, including six levels in order: knowledge, comprehension, application, analysis, synthesis, evaluation. The instructors designed 6 discussion questions that were based on this model which guided students from recall to critical thinking of knowledge, measuring surface as well as deep learning, particularly for the Colour Theory topic. Surface learning was

208

Ondris Pui et al. measured by 4 multiple choices questions of low‐level knowledge. Deep learning was measured by two discussion questions of critical thinking level. During class time, six questions were posted on 6 slides in the classroom and moved gradually from the lowest to the highest level based on Bloom’s Taxonomy model.

3.3 Procedure and measurement At the end of each session, students who participated in the PRS class were invited to complete a Study Process Questionnaire (Biggs et al. 2001) with five Likert‐scales. Qualtrics Online Survey Software was used to support data collection. The correlation analysis supported for investigating the relationship between students’ attitude towards using PRS in classroom and the learning process in two scales, deep and surface approach. At the end of the semester, a Pop Quiz on Colour Theory was conducted in order to assess students’ knowledge in term of analysis, synthesis and evaluation at deep learning level. Two groups of students who were in the top five and bottom five rank scores were invited for a semi‐structured focus group. The interview was conducted with the open questions to help in understanding their attitudes toward using PRS during class time. The prompts of these semi‐interviews include:

experiences of learning before and after the introduction of PRS

attitude towards the use and non‐usage of PRS in the classroom.

The interview was voice recorded and coded in as qualitative data for further conclusion. The purpose of the focus group method was to understand students’ attitude toward using PRS in terms of level of engagement they have achieved and determining the factors affecting their critical thinking.

4. Results Thirty respondents participated in the survey out of the forty students. The quantitative data from the survey and qualitative data from the focus group interviews have strongly supported in understanding the relationship of using PRS and learning processes at surface and deep approach. First of all, an exploratory factor analysis (EFA) was applied in reducing low loading factors. This is an essential first step for validating the next analysis. Factor loadings of .40 and higher are considered significant for interpretative purpose (Hair et al. 1998). 20 items were found to be valid for running correlation analysis and 2 items were removed due to low loading coefficient. Table 1 contains the full results of the EFA test. According to the test, the coefficient of factor loading between items recommended for four component factors. They include “Interested” in deep learning, composed by DM5 and DS14; “Passive”, composed by SM7 and SM19; “Satisfied” with surface learning, composed by DM1, DM9, DM17, DS2, DS10, DS18. Secondly, a reliability and validity test was conducted. The alpha coefficient of items in four constructs (including: Deep Strategies, Surface Motives, Deep Motives, Surface Strategies) were ranged from .621 to .729, suggesting that the items have relatively high internal consistency (table 2). Based on this finding, the Pearson correlation analysis was applied to test the relationship between the attitudes toward using PRS and the four component factors related to learning approaches (table 3). The results showed that the higher the attitude toward using PRS, the higher the level of interest in deep learning is achieved. This suggests that PRS, has a positive effect on students’ attitude and interest in deep learning at a coefficient of .454 and .483, significant at P < 0.05. On Surface Learning approach, the Passive factor does not show any significant relationship with any other component. Finally, the biggest finding from the survey is the positive relationship between Satisfied and Surface learning level. Both Surface Motivate and Surface Strategy have high correlation with Satisfied factor at coefficient .440 and significant at P < 0.05. The mean score had reached 60 percent and demonstrated a significant achievement of deep learning from students’ outcomes. With regard to students’ new role, all participants had indications of interest and curiosity in using PRS during lectures. The PRS had motivated and assisted the students to participate actively in creative ways to achieve critical thinking level. Some strategies of using PRS have been identified, such as using it as an inquiry tool to communicate with teachers. For example, one said that “I know the correct answer, but I want to get more explanation for wrong ones from the lecturer, so I click on the wrong answer….Everyone understands the correct answers but I like to ask more questions about wrong options” (student 1). Another student also

209

Ondris Pui et al. expressed a very similar idea by saying “choosing wrong answers and then understanding why they are wrong help me remember the knowledge longer and deeper” (student 4). Obviously, PRS appeared to offer students an alternative to double check or solidify their understanding of the instructed knowledge, which were unlikely to exist in traditional lecturer‐centred classes. In addition, students feel confident and more motivated in responding because of the anonymous function that the PRS offers. Other students commented that “Using PRS encourages lazy, shy and unmotivated people like me to respond enthusiastically across the questions in class” (student 7), and “in other classes, I am afraid of getting embarrassed by giving the wrong answers, but with PRS, I can respond comfortably because no one knows my choice” (student 2). Table 1: Factor analysis test

210

Ondris Pui et al.

Table 2: Reliability test Component factors

Cronbach’s alpha

Deep Strategies (DS)

.430

Surface Motives (SM)

.621

Deep Motives (DM)

.782

Surface Strategies (SS)

.844

The use of the PRS also initiated a new role for the lecturer. The active response from students required the lecturer to offer feedback immediately and in detail. This is a critical change of pedagogy compared to traditional classroom. By using PRS, lecturers can be more flexible in understanding students’ demand of knowledge and can provide immediate feedback to verify concepts that are misunderstood. A student mentioned that “If I do not understand the concept, I have a chance to listen the explanation again when

211

Ondris Pui et al. choosing the wrong answers in the review questions” (student 8). With this model, the interaction between teacher and student promotes instructors not only as presenter but also as an active leader to motivate active learning when PRS is present. With respect to the relationship between them, students can possibly take advantage of PRS to achieve deep knowledge actively and lecturers on the other hand can promote active learning. However, using this pedagogy requires careful planning in assisting students to be interested, attentive towards the lectures’ content and the ability to bring them to a higher level of thinking. This will include using appropriate comprehensive questions gradually moving from level of recall to critical thinking. The lecturer’s knowledge is also challenged and affirmation of answers would surface when preparing for questions for PRS classes. Furthermore, students commented using PRS is more interesting than traditional class, because “Usually, traditional classes are boring because the teachers just talk and talk, if we do not understand, we skip it. But in PRS classes, the contents of lecture are more solid and interesting by review questions that move gradually from easiest to hardest level” (student 10). Table 3: Correlation coefficients

*. Correlation is significant at the 0.05 level (2‐tailed). Pearson correlation analysis

5. Discussion A number of studies have suggested that PRS is effective in creating an active learning environment. Our study investigates the motives and strategies applied by students to reach a level of Surface or Deep Learning. This study concludes that although students are motivated at Surface Learning while PRS was used, their motives and strategies can facilitate learning process to reach Deep Learning significantly. However, the coefficient does not show the correlation between using PRS and the extra learning variable. Hence, using PRS does not motivate students in applying extended learning towards the topic. This finding is consistent with previous studies where PRS did not significantly improve students’ long‐term retention of material in the course (Karaman 2011), or as a motivating factor to study more for the course (Trees and Jackson 2007).

212

Ondris Pui et al. The findings from the focus group also disclosed that students can be distracted while using PRS and it did not help much in motivating them in group discussion. “Sometime, I could not concentrate on the lecture because of my curiosity in using this tool….I participated in group discussion as usual like in any other classes”. These findings are contrary to previous research, where clicker assisted students in increasing their concentration (Hinde and Hunt 2006; Trees and Jackson 2007), and their participation and contribution in class discussion (Sprague and Dahl 2010; Efstathiou and Bailey 2012). A possible reason for this inconsistent finding was due to the fact that PRS was implemented for the first time in the classroom investigated. Thus, excitement and curiosity in using new tools can lead to distraction. Although students were not motivated to take part in group discussion through the use of PRS, the lecturer initiated reviews of the best answers as a class discussion after each session. This is considered as a new pedagogy in enhancing learning process. Instructors on the other hand, discovered that preparing effective critical questions and activities for PRS classes require time and effort. The learning curve in setting up PRS classes could also be a challenge that requires time in understanding the technology and many practices before performing real classes. There is also the positive aspect of PRS that is beneficial for instructors. By applying Bloom’s Taxonomy together with PRS, the instructor themselves are challenged into a deeper understanding of the topic taught, reinforcing their knowledge when preparing critical thinking questions. The results have also shown a significant impact on the changing roles between teacher and learner in a specific Asian classroom culture. Ku et al. (2004) has mentioned the characteristic of Confucian pedagogy as a one‐way communication teaching mode, where the instructor teaches and students listen attentively without much interaction and communication with the instructor. According to the results, students were motivated and it gave them an active role and opportunity to participate in a classroom when PRS was used. This also enables them to form strategies subconsciously in obtaining more knowledge when the classroom environment encourages participation. The qualitative data indicated there is positive interaction between instructors and students. Students’ active role had manipulated the classroom interaction to achieve Deep Learning for the Colour Theory topic. Our results provide statistically significant evidence that the use of PRS can have an impact on student learning process at Surface and Deep Learning level. However, the sample size is small and the experiment was only applied in a Multimedia Design course. Therefore, it is limited as a representation for the entire population of Vietnamese students. Secondly, the study scales were limited while facilitating the use of PRS. Variables such as course design, students’ characteristics, students’ desire to learn, classroom involvement, students’ motivation, the amount of clicker used and gender were also significant factors to consider (Huang et al. 2011; Kanga et al. 2012). For future research, the investigation will utilize a bigger sample size and consider more factors in getting effective results.

References Beckert, T. E., Fauth, E. & Olsen, K. (2011) Clicker Satisfaction for Students in Human Development: Differences for Class Type, Prior Exposure, and Student Talkativity. North American Journal of Psychology, Vol 11, No. 3, pp 599‐612. Beringsa, D. G. V. (2010) Student response system and learning oral presentation skills. Procedia Social and Behavioral Sciences, Vol 2, No. pp 1786–1789. Biggs, J. B. (1984) Differentiation of Learning Processes within Ability Groups. Educational Psychology, Vol 4, No. 1, pp 21‐ 39. Biggs, J. B. (1987) Student approaches to learning and studying, Melbourne, Australian Council for Educational Research. Biggs, J. B., Kember, D. & Leung, D. Y. P. (2001) The Revised Two Factor Study Process Questionnaire: R‐SPQ‐2F. British Journal of Educational Psychology, Vol 71, No. pp 133‐149. Britten, T. (2011) Using student response systems (clickers) in the history classroom. Teaching History: A Journal of Methods Vol 36, No. 1, pp 14‐21. Camey, J. P., Gray, G. T. & Wert‐Gray, S. (2008) Evaluating the impact of student response systems on student performance. Journal for Advancement of Marketing Education, Vol 13, No. pp 1‐8. Christopherson, K. M. (2012) Hardware or Wetware: What Are the Possible Interactions of Pedagogy and Technology in the Classroom? Technology and Teaching, Vol 38, No. 4, pp 288‐292. Efstathiou, N. & Bailey, C. (2012) Promoting active learning using Audience Response System in large bioscience classes. Nurse Education Today, Vol 32, No. pp 91–95. Fallon, M. & Forrest, S. L. (2011) High‐Tech Versus Low‐Tech Instructional Strategies: A Comparison of Clickers and Handheld Response Cards. Teaching of Psychology, Vol 38, No. 3, pp 194‐198. Fifer, P. (2012) Student perception of clicker usage in nursing education. Teaching and Learning in Nursing, Vol 7, No. pp 6‐ 9.

213

Ondris Pui et al. Gagne, J. C. D. (2011) The impact of clickers in nursing education: A review of literature. Nurse Education Today, Vol 31, No. pp e34‐e40. Gauci, S. A., Dantas, A. M., Williams, D. A. & Kemm, R. E. (2009) Promoting student‐centered active learning in lectures with a personal response system. Advances in Physiology Education, Vol 33, No. pp 60‐71. Gijbels, D., Watering, G. V. D., Dochy, F. & Bossche, P. V. D. (2005) The relationship between students’ approaches to learning and the assessment of learning outcomes. European Journal of Psychology of Education, Vol 20, No. pp 327‐ 341. Gow, L. & Kember, D. (1990) Does higher education promote independent learning? Higher Education, Vol 19, No. 3, pp 307‐322. Hair, J., Tatham, R., Anderson, R. & Black, W. (1998) Multivariate Data Analysis (5th Edition), Prentice Hall. Hinde, K. & Hunt, A. (2006) Using the Personal Response System to Enhance Student Learning: Some Evidence from Teaching Economics. Audience Response Systems in Higher Education: Applications and Cases, Vol, No. pp 140‐154. Hoekstra, A. & Mollborn, S. (2008) Vibrant student voices: exploring effects of the use of clickers in large college courses. Learning, Media and Technology, Vol 33, No. 4, pp 329–341. Huang, Z., Sun, C., Chen, L. & Lei, D. 2011. The Research of Clicker Application on College Physics Teaching. 20ll International Conference on Electronics and Optoelectronics. Justicia, F., Pichardo, C., Cano, F., Berbén, A. B. G. & Fuente, J. D. L. (2008) The Revised Two‐Factor Study Process Questionnaire (R‐SPQ‐2F): Exploratory and confirmatory factor analyses at item level. European Journal of Psychology of Education, Vol 23, No. 3, pp. Kanga, H., Lundeberga, M., Wolter, B., Delmasd, R. & Herreide, C. F. (2012) Gender differences in student performance in large lecture classrooms using personal response systems (‘clickers’) with narrative case studies. Learning, Media and Technology, Vol 37, No. 1, pp. Karaman, S. (2011) Effects of audience response systems on student achievement and long‐term retention.(Report). Social Behavior and Personality: an international journal, Vol 39, No. 10, pp 1431(9). Ku, H.‐Y., Pan, C.‐C., Tsai, M.‐H., Tao, Y. & Cornell, R. (2004) The impact of instructional technology interventions on asian pedagogy. Educational Technology Research and Development, Vol 52, No. 1, pp 88‐92. Mayer, R. E., Stull, A., Deleeuw, K., Almeroth, K., Bimber, B., Chun, D., Bulger, M., Campbell, J., Knight, A. & Zhang, H. (2009) Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, Vol 34, No. pp 51–57. Murphy, K. (2012) Using a Personal Response System To Map Cognitive Efficiency and Gain Insight into a Proposed Learning Progression in Preparatory Chemistry. Journal of Chemical Education, Vol 89, No. pp 1229−1235. Nelson, M. L. & Hauck, R. V. (2006) Clicking to Learn: A Case Study of Embedding Radio‐ Frequency based Clickers in an Introductory Management Information Systems Course. Journal of Information Systems Education, Vol 19, No. 1, pp 55‐64. Patterson, B., Kilpatrick, J. & Woebkenberg, E. (2010) Evidence for teaching practice: The impact of clickers in a large classroom environment. Nurse Education Today, Vol 30, No. pp 603–607. Ramsay, S., Barker, M. & Jones, E. (2006) Academic Adjustment and Learning Processes: a comparison of international and local students in first‐year university. Higher Education Research & Development, Vol 18, No. pp 129‐144. Revell, S. M. H. & McCurry, M. K. (2010) Engaging Millennial Learners: Effectiveness of Personal Response System Technology with Nursing Students in Small and Large Classrooms. ResearchJournal of Nursing Education, Vol 49, No. 5, pp 272‐276. Shaffer, D. M. & Collura, M. J. (2009) Evaluating the Effectiveness of a Personal Response System in the Classroom. Teaching of Psychology, Vol 26, No. pp 273‐277. Sprague, E. W. & Dahl, D. W. (2010) An Evaluation of the Personal Response System Clicker Technology in Introductory Marketing Courses. Journal of Marketing Education, Vol 32, No. 1, pp 93‐103. Trees, A. R. & Jackson, M. H. (2007) The learning environment in clicker classrooms: student processes of learning and involvement in large university‐level courses using student response systems. Learning, Media and Technology, Vol 32, No. 1, pp 21‐40. Watkins, D. (1986) The Approaches to Learning of Australian Tertiary Students: A Replication. Higher Education Research & Development, Vol 5, No. 2, pp 185‐190.

214

Learning From a Failed ERP implementation: The Case of a Large South African Organization Anjali Ramburn, Lisa Seymour and Avinaash Gopaul University of Cape Town, Cape Town, South Africa Ranjali17@gmail.com Abstract: Enterprise Resource Planning (ERP) systems allow enterprises to integrate various processes across different functional areas in an attempt to increase productivity, efficiency and to sustain their competitive edge. However, despite the various positive outcomes of adopting ERP systems, the implementation process poses a number of challenges for organizations. This research reports on the case of a failed ERP implementation in a large South African organization and looks into the details that led to the failure. Consequently, through the lessons learnt, this paper also reports on the different dimensions of Knowledge Management (KM) required during the ERP implementation using Chan’s (1999) framework. Through use of semi‐structured interviews and qualitative data analysis, this study validated Chan’s (1999) model in a large emerging economy. Some of the challenges the organization faced include lack of technical, process and project knowledge. Other concerns include poor understanding of the need for change, lack of management support and poor change management and training initiatives. Managing ERP systems knowledge has been identified as a critical success factor. However, a review of the existing literature reveals a lack of comprehensive research on the role of KM during ERP implementation. While ERP implementation within large organizations has been well researched, there have been relatively fewer studies focusing on the role of KM in emerging economies. The study seeks to close some of the identified gaps in this area, hence contributing to the existing body of knowledge. Through a practical lens, this research should be of immediate benefit to large organizations in the South African economy. Keywords: failed ERP implementation, user experiences, knowledge management challenges and dimensions, large organization, emerging economy

1. Introduction Organizations are continuously facing challenges, causing them to rethink and adapt their strategies, structures, goals, processes and technologies in order to remain competitive (Bhatti, 2005; Holland & Light, 1999). Many large organizations are now dependent on ERP systems for their daily operations. Enterprise Resource Planning systems (ERP) allow enterprises to integrate various processes across different functional areas in an attempt to increase productivity, efficiency and to sustain their competitive edge. ERP is not a new technology, as such; its market penetration within large organizations is considerable. ERP systems have been widely deployed in developed countries all over the world to automate and streamline business processes to achieve global competitive advantage. Over the years, ERP vendors have tried to overcome “the economic, cultural and basic infrastructural barrier” related to emerging economies (Upadhyay, Basu, Adhikary, Dan, 2010). An increasing number of organizations are investing in ERP systems in South Africa. There have been many implementations in the South African public sector such as the SAP implementations at the City of Cape Town and Tshwane Metropolitan Council. Consequently, this research will focus on the implementation of an ERP in a large organization in South Africa. The implementation process is described as costly, complex and risky whereby firms are not able to derive benefits of the systems despite huge investments. Half of all ERP implementations fail to meet the adopting organizations’ expectations (Jasperson, Carter & Zmud, 2005). This has been attributed to the disruptive and threatening nature of ERP implementations (Zorn, 2002). With ERP adoption, comes the requirement of adapting work practices to the global processes inherent in the ERP system. This requires the organization to undergo a major transformation; users have to come to terms with the organizational changes, new ways of work and unlearn previous practices (Robey, Ross & Boudreau, 2002). This process however can be less challenging and more effective through proper use of knowledge management (KM) throughout the ERP lifecycle phases. Managing ERP systems knowledge has been identified as a critical success factor and as a key driver of ERP success (Leknes & Munkvold, 2005). An ERP implementation is a dynamic continuous improvement process and “a key methodology supporting ERP continuous improvement would be knowledge management” (McGinnis & Huang, 2004). However, there has been very little work conducted to date that assesses the practices and techniques employed to effectively explain the impact of knowledge management during ERP systems implementation (Parry &Graves, 2008; Sedera, Gable & Chan, 2004).

215

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul

1.1 Research objective, scope & importance This study seeks to explore the reasons that led to the failure of the ERP system in a large organization. One of the objectives is to investigate the challenges faced by organization during the implementation phase. Furthermore, the dimensions of KM practices required during the ERP implementation phase are investigated. In this context, the implementation phase takes into account an end user’s perspective and focuses on the experiences of the users’ during the training and development phase of the implementation. This research should be of immediate benefit to both academics and practitioners. From an academic viewpoint, this study looks at the ERP implementation phase from a KM perspective. There has not been any prior research in South Africa, hence this research provides a better inference of knowledge .This research also aims to provide a reference for any further future research which might be undertaken in the context of KM and ERP systems in South Africa. Through a practical lens, this research draws a number of lessons learnt from a failed ERP implementation which can be of immediate benefit to large organizations implementing ERPs in South Africa and international organizations with global user bases.

2. Literature review 2.1 Enterprise resource planning systems Enterprise Resource Planning (ERP) systems include a set of software products which are mostly targeted to support day to day operations, decision making and automation, streamlining and improvement of processes in organizations (Sedera et.al, 2004). ERP systems are characterized as large, complex, multifunctional, modular and generic. However, most enterprises need to customize their ERP packages to align their specific business needs to the capabilities of the systems, in an attempt to achieve optimum business value. This process however poses a number of challenges to organizations, leading to high implementation costs, big investments and risky projects. Despite the cost intensive, lengthy and risky process, the rate of implementation of ERP systems has increased over the years. Most of the large multinational organizations have already adopted ERPs as their de facto standard with the aim of increasing productivity, efficiency and organizational competitiveness (Pan, Newell, Huang, Cheung, 2004).

2.2 Knowledge management Knowledge management can be referred to as the diffusion of knowledge. Employees in organizations acquire different types of knowledge and the core purpose of knowledge management is the distribution of this knowledge to those who require it for their tasks within the organization (Parry & Graves, 2008). A number of studies have emphasized the importance and use of KM in the ERP implementation process. Use of KM activities during the ERP implementation phase ensures reduced implementation costs, improved user satisfaction as well as strategic and competitive business advantages through effective product and process innovation during use of ERP (Sedera et.al, 2004). Organizations should therefore be aware of and identify the knowledge requirement for any implementation. This knowledge can be derived from the lessons learnt of people who have suffered from failed ERP implementations as well as those who have gained useful knowledge and experience from successful ERP implementations. Hence, based on past research, experiences and failed implementation cases, Chan (1999) puts forward a three dimensional framework of KM required for successful ERP implementation. These refer to project management, business and management and technical knowledge. 2.2.1 Project management For the previous 12 years, project management has been amongst the key critical success factors required for successful ERP implementation (Huang, 2010; Umble, Haft & Umble, 2003). Project definition, project planning, people/resources, time and cost are key components of this dimension (Chan, 1999). In their research, Umble et.al (2003) categorized poor project management and a wrong selection of the implementation team as the major reasons accounting for ERP implementation failures.

216

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul 2.2.2 Business and management knowledge The business and management knowledge dimension deals with the soft issues of ERP implementation and includes business process engineering, change management, management support, outsourcing, training and education and culture. Management Support Undoubtedly one of the most influential factors for successful ERP implementation, top management support has always been rated the number one critical success factor by most previous studies (Fang & Patrecia, 2005; Motwani, Subramaniam, Gopalakrishna, 2005; Umble et.al, 2003). Senior management commitment plays a fundamental role in the entire implementation process as it forms the basis for a successful implementation (Fang & Patrecia, 2005). Change Management Managing change is arguably one of the primary concerns of ERP implementation. Knowledge about managing change in an organization should be prioritized as implementing change without any prior planning or strategy in organizations will result in confusion and instability amongst employees (Chan, 1999; Motwani et.al, 2005). In effect, half of the ERP implementations have failed to achieve expected benefits as a result of misjudging the efforts involved in change management (Somers & Nelson, 2001). 2.2.3 Technical knowledge The third dimension looks at the technical knowledge which organizations need to consider in order to successfully implement their ERP systems. Chan (1999) identifies the following attributes: process engineering, programming, system administration and hardware/network.

3. Research method This research is qualitative and descriptive in nature. Semi‐structured interviews have been used to allow unrestricted responses from the sample, providing richer insight into the users’ opinions and perceptions of ERP implementation, hence allowing the researcher to understand the subject from different perspectives. An inductive research approach has been used to ensure the derived findings have resulted from the raw data. Thematic analysis has been used to analyze the data (Thomas, 2003; Attride‐Stirling, 2001). Iterative analysis of the collected data has enabled the researcher to understand and investigate the main research problems posed. All the interviews have been recorded and transcribed. The transcripts of the interviews have been read a number of times to identify, conceptualise, and categorise emerging themes. Approval for this research has been obtained from the University of Cape Town’s ethical committee. Participants have been asked to sign a voluntary participant consent form and their anonymity has been assured.

3.1 Case description The case organization has a number of branches throughout South Africa, employing over 39 000 people and is currently launching the SAP Project Portfolio Management module throughout its different branches across the country. Currently in the implementation stage, an organization wide training, involving the employees, has already been conducted. A number of individual interviews have been carried out in one of organization’s division in Cape Town and purposive sampling has been used to select the interviewees. All the interviewed participants had been through the training and were impacted by the SAP implementation process. The sample includes 8 employees, 6 of them are users of the system while the remaining 2 are support staffs. All of the participants are currently working within the engineering department of the organization. The sample consists of 63% males and 37% female. With respect to the age distribution, the sample reflects a relatively young user group: 75% of the respondents fall in the 25‐35 year age group with the remaining 25 % being older than 50 years.

4. Analysis The users experienced a number of challenges throughout the implementation process, leading to non‐use of the system. Most of the issues were experienced by the users during the training period and consequently the ERP system initial ‘Go‐Live’ date has been put on hold. Users were told that the identified issues were being

217

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul addressed. The section below provides a comprehensive account of the aforementioned challenges and experiences of the users that led to non‐use of the ERP system.

4.1 Training The training itself was a two day process and it was conducted by two trainers during which all the users of the new system were trained. Users included people from various departments such as finance, project management and planning. The users perceived the training of the new system inadequate in various ways. They felt they were wasting their time as they saw no value in the attending the sessions. The training consisted of simulation exercises related to user activities, tasks and processes. The simulation exercises did not give them an adequate insight and exposure of the new system. There was no format or structure to follow and they felt that they were only filling random blocks on the system. The training was seen as a click and paste exercise with a number of repetitive data that needed to be entered in the system. “So the training showed us how to click on SAP. This is what we did; ‘Click, Click, Click, Copy Paste, Copy Paste’…It was ridiculous, it was insane to have more than 300 steps of clicking” I1. Moreover, it was reported that there were numerous errors and technical faults in the software that were uncovered during this period. The software did not function as intended and users saw the system as sub‐standard. At the end of the 2 day training, the users were given an assessment to test their acquired competencies. Users perceived the assessments as a mere formality; they did not see the usefulness of the assessments to reinforce or test their skills, instead they claimed anyone could have completed the tests using the information provided in training material without really having a core understanding of the process.

4.2 Customization of training materials Another concern raised was with reference to the lack of customization of the training materials and exercises used resulting in a poor focus on local context. Interviewees could not relate to the given training examples as they were based on the process flow from a different province. Interviewees stated that each province has its own way of operating and has unique terms and terminologies. The fact that the standard examples used came from only one province made it harder for the users to understand the overall process. “The examples used were from Joburg, I could not relate to the examples. It would have been nice to have our terminologies” I7.

4.3 Information overload The training included people from different departments who are associated with a range of diverse processes. As a result, the trainers covered various tasks related to various processes in one training session as opposed to focusing on the specific processes that a specific group of users could understand. There was a lot of information which was not specific to anyone’s job role and this made it difficult to focus. The users got confused with regards to their role definition and the ownership of the different tasks. The trainers were unable to clear this confusion. This caused a certain level of panic amongst the employees; subsequently they lost interest in the training and attributed it as an unproductive process. Employees believed the training would have been easier and beneficial if the training was limited to their specific and individual department and processes. “If you are planners, this is all you need to do. I do not feel we had to do all these steps and a lot of the information needs to be entered before we would see it as planners, someone else would have already entered that data but when we doing the training everyone was confused with what and why we need to do certain tasks” I1. “If the trainers said, this is only 2 pages of information you need to do, we would be calmer. So there was too much of information given which was not specific to anyone and you did not know what to focus on and where this data is supposed to come from” I8.

4.4 System complexity The users described the ERP system as incoherent, the user interface and navigation increasingly complex as opposed to their existing system. They persistently compared the ease of use of their existing system to SAP’s complex design and structure. They have been working with the current system for a number of years and believed it operated in the most logical way, the same way as to how their minds would function. However, they did not understand how the new system functioned, the system made no sense to them. Activities which could be completed without any difficulty on the existing system seemed increasingly complex when using

218

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul SAP. Furthermore, the number of steps taken to complete a task seemed to have increased drastically; hence they felt their productivity would also decrease. “I have not used the system before, so I do not understand it. We struggled with the complexity of the system. The number of steps we had to complete for one task made it worse. No one understood what and why we were doing most of the steps. This will surely affect my productivity” I3.

4.5 Trainers’ process and system competencies The trainers’ knowledge and skills were also questioned. The trainers lacked key ERP skills and did not understand the process from the users’ perspective. Since none of the trainers had any understanding of the system as end users, there were some inconsistencies in what was communicated. Ownership of roles and tasks were not clearly defined and there was no clarification on the information and process flow between the different departments and the individuals as per their role definition. “However what makes it difficult is that the trainers do not work with the project. They do not understand the process entirely and are not aware of what is happening in the background, they only collect data.” I5 Employees also indicated that the trainers’ lack of technical knowledge contributed to their negative experiences. The trainers were the administrator support technicians who are experts in the existing system used by the organization but they did not have enough expertise to tackle the different technical issues that surfaced during the training. “There were a lot of unanswered questions. Data needed to be put in the system, we did not know where…We asked the trainers but they did not know themselves, I felt they were not very qualified either” I3

4.6 Project management Interviewees were unaware of the clear project objectives, milestones and deployment activities. The employees did not have any information regarding the status of the project activities. They were only aware of the fact that they had to be trained in SAP as management had a set deadline set up by which all employees had to complete the training. The training was conducted a year prior to the conduction of the interviews and users claimed that they did not hear anything regarding the status of the system. They believed that there will be another training session in the future as none of them remembered how to use the system. Users were also uninformed of the date they were required to start using the system. They were past the initial system ‘Go‐ Live’ date and there was no information given on the new roll out dates. Some of them believed they were not near the implementation stage, and the training was only a pilot activity to assess the ERP system’s suitability to their environment. Others hoped that the implementation had been cancelled due to the number of issues and faults experienced in the training sessions. “All they said was that now you have had the training for SAP and this is the system we will phase into. No one knows when it will happen for sure. I am hoping it won’t be anytime soon….We do not get frequent emails or updates to inform us of the status of the project.” I8 “We got trained in October now we are in August, I hope they will train us again as I have forgotten so much of what was covered. I did not gain anything from the training.”I5

4.7 Management support Management has neither supported nor encouraged the employees to use the system. Top and middle management were not part of any training activities that took place. Users felt management had not supported the initiative. Consequently, they assumed they did not have to use or understand the system until they got the ‘go ahead’ from top and middle management. They had not received any information or communication on the purpose and benefits of the new system from management. This lack of motivation and support from management resulted in users disinterest and lack of commitment towards the system. “We will only start using it when we get the word from top management; well basically when they tell us, as from now you are expected to use this program. Then I will make an effort to learn the program” I3.

4.8 Change management The results show the lack of importance attributed to this area. Lack of proper communication channels and planning coupled with the absence of change management initiatives resulted in employees’ confusion, instability and resistance. Users said they were unaware of the organization’s goals and objectives. The initial

219

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul implementation training took place in October and one year later, there was still no clarity on the status on the system, future roll out plans and strategies. 4.8.1 Lack of need of system Consequently, the employees did not understand the need of a new system from a strategic perspective or the perceived benefits of the new system. They constantly questioned the implementation of the new system; most of them had their own understanding as to why the system was being implemented. They did not see any value in the new system and felt their previous system could do everything they needed it to. They had never felt the need for a new system. They felt the new system had been forced onto them; the system was seen as a misfit as it did not cater to their needs. The users seemed disinterested in any related activities. They mentioned that occasionally there would be emails sent. However, none of them had opened any of the emails related to the new system. They felt that they have been treated unfairly and some of them had even considered leaving their jobs once the system phased in. “The system was forced onto us. I think they said they wanted a better way of keeping previous information…but I am not too sure” I1

5. Findings The emergent themes of this research have been categorized according the three main knowledge dimensions put forward by (Chan, 1999). These knowledge dimensions provide an indication of the key knowledge required for implementation of an ERP system in large organizations. Figure 1, adapted from Chan (1999), illustrates the different dimensions of KM applicable to a large emerging economy.

Figure 1: Dimensions of knowledge management during the ERP implementation phase

220

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul

5.1 Technical knowledge The findings of this research reveal 3 different aspects which impacted the ERP implementation. Participants struggled to use SAP due to the system’s complexity, their lack of technical knowledge of the system and the trainers’ lack of technical expertise. System complexity has been attributed to the lack of system configuration knowledge which could have been essential in substantially decreasing the number of steps required to perform a particular task. Therefore, trainer’s expertise, participant’s technical know‐how and configuration knowledge are key components on this dimension.

5.2 Project management knowledge The findings of this study show that project knowledge such as project objectives, milestones, deployment activities, people and time are the key project dimensions required for successful implementation of an ERP system. Lack of key project knowledge accounted for employees’ resistance, uncertainty and negative attitude towards the overall implementation process. With statistics showing more than half of ERP projects going over time, knowledge about the time required to install ERP systems is crucial to the organization and is a crucial aspect of project management that needs to be carefully assessed. This particular organization was no exception. The implementation process was put on hold for more than a year due to the number of issues uncovered during the training period and the system ‘Go‐Live’ date was suspended. Identification of the right mix of people and resources required to implement the ERP system is another crucial activity. The findings revealed that the organization incorrectly selected its core training team, hence the incompetence of the chosen training team and their failure to transfer key project knowledge to the employees.

5.3 Business and management knowledge Process knowledge, customization & contextualization knowledge, management support, change management and training have been identified as key components of this dimension. Since the research was carried out in a large organization with multiple branches throughout South Africa, process knowledge and knowledge on contextualization and customization of the training content from the users’ perspective is essential. In this particular case, the process flow was not clear to the users and the training material was not satisfactorily contextualized and customized to suit users’ needs, resulting in information overload. The findings reveal that employees expected management to enforce the implementation of the system and were waiting for management’s go‐ahead and communication to accept and use the system. Employees also unanimously agreed that they would have shown more interest in the new system and they would have been keener to learn and use the system had they been informed of the strategic need for change. Training initiatives and education are usually an integral component of change management. The results indicate that although the organization has invested in the training and education of their employees to enhance their ERP skills and to ensure they are capable of understanding their business processes and functions, the training was not seen as valuable and adequate to create and transfer key knowledge.

6. Conclusions This paper reports on the various reasons that led to an ERP system implementation failure in a large engineering company in South Africa and provides a comprehensive insight on users’ initial experience. This research unveils a number of intra‐organizational barriers to efficient knowledge creation, transfer and use. Inadequate training, lack of technical and process knowledge, lack of knowledge on management and project initiatives and lack of change management have been cited as major challenges. Seemingly, in a large organization with multiple branches throughout South Africa, contextualization and customization of the processes from the users’ perspective and system configuration are key aspects to consider for the ERP implementation process. Based on the above identified challenges, a three dimensional knowledge management framework derived from Chan’s (1999) model has been projected. The resultant framework looks at the key knowledge dimensions required during the ERP implementation phase from an end user perspective. The findings of this study thus validate Chan’s (1999) three dimensional knowledge management model. While the three major knowledge dimensions namely project management, business management and technical knowledge are prominent in this study; some of the derived components of each dimension are unique to this study. Notably, as opposed to Chan’s (1999) original model, outsourcing and cost did not emerge from the findings of this study.

221

Anjali Ramburn, Lisa Seymour and Avinaash Gopaul

References Attride‐Stirling, J. (2001) “Thematic networks: an analytic tool for qualitative research”, Qualitative Research, Vol 1, No.3, pp 385‐405. Bhatti T.R. (2005) “Critical Success Factors for the Implementation of Enterprise Resource Planning (ERP): Empirical Validation”, Paper read at 2nd International Conference on Innovation in Information Technology, Zayed University, College of Business, Dubai, UAE. rd Chan, R. (1999) “Knowledge management for implementing ERP in SMEs”, Paper read at 3 Annual SAP Asia Pacific Institutes of Higher Learning Forum 'Maximizing the synergy between teaching, research and business, 1 ‐2nd November, Singapore. Fang, L. & Patrecia, S. (2005) “Critical Success Factors in ERP Implementation” Masters thesis. Jànkàping International Business School. Huang, Z. (2010) “A compilation research of ERP implementation Critical Success Factors”, Issues in Information Systems, Vol 11, No. 1, pp 507‐512. Holland, C. P. & Light, B. (1999) “A critical success factors model for ERP implementation”, IEEE Software, Vol 16, No.3, pp 30‐36. Jasperson, J. S., Carter, P. E. & Zmud, R. W. (2005) “Conceptualization of Post‐Adoptive Behaviours Associated with Information Technology Enabled Work Systems”, MIS Quarterly, Vol 29, No.3, pp 525–567. Jones, M.C. Cline, M. & Ryan, S. (2004) “Exploring knowledge sharing in ERP implementation: an organizational cultural framework”, Decision Support Systems, Vol 41, pp 411‐434. Leknes, J. & Munkvold, B.E. (2005) “The role of knowledge management in ERP implementation: a case study in Aker th Kvaerner”, Paper read at 14 European Conference on Information Systems (ECIS 2006), 12‐14th June, Göteborg, Sweden. McGinnis, T.C. & Huang, Z. (2004) “Incorporating of Knowledge Management into ERP continuous improvement: A research framework”, Issues in Information Systems, Vol 2, pp 612‐618. Motwan, J., Subramanian R. & Gopalakrishna, P. (2005) “Critical factors for successful ERP implementation: Exploratory findings from four case studies”, Computers in Industry, Vol 56, pp 529–544. Pan, S.L., Huang, J.C., Newell, S. & Cheung, A. W.K. (2004) “Knowledge Integration as a key problem in an ERP nd Implementation” Paper read at 22 International Conference on Information Systems, 16‐19th December, New Orleans, Louisiana. Parry, G. & Graves, A. (2008) “The importance of knowledge management for ERP systems” International Journal of Logistics Research and Applications, Vol 11, No. 6, pp 427–441. Rana, A.B. & Abdullah, S.A.M. (2011) “Justifying ERP Investment: The Role and Impacts of Business Case A Literature Survey”, International Journal of Computer Science and Network Security, Vol 11, No.1, pp. 185‐193. Robey, D., Ross, J. & Boudreau, M. (2002) “Learning to Implement Enterprise Systems: An Exploratory Study of the Dialectics of Change”, Journal of Management Information Systems, Vol 19, No.1, pp 17‐46. th Sedera, D., Gable, G. & Chan, T. (2004) “Knowledge Management for ERP success”, Paper read at 7 Pacific Asia Conference on Information Systems, 10‐13th July, Adelaide, South Australia. Somers, T.M. & Nelson, K. (2001) “The impact of critical success factors across the stages of enterprise resource planning implementations”, Paper read at 34th Hawaii International Conference on Systems Sciences, Maui. Thomas, D.R. (2003) “A general inductive approach for qualitative data analysis”, School of Population Health, University of Auckland, NewZealand. Umble, E., Haft, R. & Umble, M.(2003) “Enterprise resource planning: Implementation procedures and critical success factors” European Journal of Operational Research Vol 146, pp 241–257. Upadhyay, P., Basu, R., Adhikary, R. & Dan, P.K. (2010) “A Comparative Study of Issues Affecting ERP Implementation in Large Scale and Small Medium Scale Enterprises in India: A Pareto Approach”, International Journal of Computer Applications, Vol 8, No.3, pp 23‐28. Zorn. T.E. (2002) “The Emotionality of Information and Communication Technology Implementation”, Journal of Communication Management, Vol 7, No. 2, pp 160‐171.

222

Using YouTube to Support Student Engagement for the Net Generation in Higher Education Sumarie Roodt University of Cape Town, Cape Town, South Africa sumarie.roodt@uct.ac.za Abstract: Using engaging and entertaining teaching tools inside the classroom has become imperative in order to ensure, amongst others, engagement for the scholars of today. These scholars are familiar with a vast array of digital technologies and are able to engage with multiple sources of information at the same time. Traditional teaching methods are not enough to keep these scholars engaged and as such, traditional teaching methods must be supplemented by engaging learning methodologies and interactive learning tools. Many of these learning tools have their origins in Web 2.0 and have enjoyed wide‐spread adoption in the education arena because of their potential as contemporary education technologies. These technologies are not restricted to enriching course content alone and can also stimulate and encourage engagement of students in the classroom. For this research project, the use of YouTube © is described to encourage student engagement and how this innovative teaching tool can enhance students’ engagement with the subject‐matter. The research findings will be supported with outcomes from student questionnaires. The main inspiration for this paper is the research and analysis of an innovative method of teaching, being the use of YouTube for student engagement in the classroom, which has been applied for the first time Department of Information Systems at the University of Cape Town for the INF 2004F course. This will have practical implications for higher education institutions on a global scale as it will assist in the decision making process as to whether to adopt YouTube. Keywords: student engagement, social computing, net generation, technology enhanced learning, YouTube©

1. Introduction The Net Generation is a homogenous group of individuals that have grown up with and are immersed in technology (Kennedy, Judd, Dalgarno, & Waycott, 2010). They are tech‐savvy individuals and interface with technology on a regular basis (C. Jones, Ramanau, Cross, & Healing, 2010). It is for this reason that students from the Net Generation adopt new radical learning approaches (Bennett & Maton, 2010). Furthermore, the use of technological tools inside the classroom is becoming a necessity to ensure the engagement of these Net Generation students (Abell, 2011). Using YouTube and other Web 2.0 technologies in the classroom is an innovative and fresh method of teaching (Roodt & De Villiers, 2011) and these tools have the ability to enrich course content and improve student engagement (Harris, 2011). IT in Business (INF2004F) is a second year information systems course hosted at the University of Cape Town. It is a compulsory course for accounting and finance majors, and has the first year information systems course as a foundation. This is the first year in which a new, innovative method of teaching is applied, namely the use of YouTube for student engagement in the classroom. The purpose of this research paper is to outline the effect of using YouTube in the classroom for Net Generation students. The results gathered will aid in determining the feasibility of continuing this teaching method.

2. Literature review 2.1 Net generation The Net Generation is believed to be a homogenous group of people who have advanced skills in the use of information and communication technologies (Kennedy et al., 2010). They are homogenous, in the sense that everyone part of this generation is believed to be tech‐savvy and immersed digital technologies (Bennett & Maton, 2010). The terms Digital Native, Net Generation and Millennial can be used interchangeably and are the most popular terms for this newer generation (C. Jones et al., 2010). C. Jones et al. (2010) state that the main differences between these classifications are the dates that bracket the new generation. The Net Generation was said to have begun in January 1977 and ended in December 1977 (Tapscott, 2009). Palfrey and Gasser (2010) suggest that the Digital Natives started appearing after the 1980s and the Millennials can be classified as those born in or after 1982 (Oblinger, 2003).

223

Sumarie Roodt For the purpose of this research paper, anyone born after 1982 will be characterized as part of this new generation, and the terms Digital Native, Net Generation and Millennial will be used interchangeably. Those born into the Net Generation are the people who have grown up with technologies that include mobile phones, high powered computers and the internet (Duffy, 2007). They “speak” the digital language of computers and the internet (Helsper & Eynon, 2010, p.504). Blogs, wikis, virtual worlds, instant messaging and authoring videos for www.youtube.com are just a few of the activities enjoyed by the average Millennial (C. Jones et al., 2010). Duffy (2007, p.119) goes on to mention that people from the Net Generation operate at “twitch speed”, which means they respond and expect feedback almost instantaneously. Digital Natives also like to parallel process or multi‐task and they prefer gathering information through pictures and videos over text (Helsper & Eynon, 2010). As has been mentioned during the introduction, the Net Generation (a.k.a ‘The Millennials’), present a material challenge to the academic community, not only because they learn differently from traditional learning methods but also because they want to learn differently (Roodt et al., 2009). In terms of the defining characteristics mentioned above, there are ten features that have a potential impact on higher education (Oblinger & Oblinger, 2005, p. 16):

Digitally Literate – They can operate a variety of technological devices and are familiar with the internet, for example: laptops, cellphones and/or iPods ®.

Connected – They are almost always connected to a technological network of some sort, for example: cellular networks and/or the internet.

Immediate – They have fast response time and multi‐task, for example: playing a game and instant messaging at the same time.

Experiential – They have an exploratory style of learning and have a preference for ‘learning by doing’ which results in better memory retention of the subject matter, for example: creating an animation to teach peers about green IT instead of writing a document.

Social ‐ They seek to interact with others, whether in their personal lives, their online presence, or in class, for example: blogging and having social network profiles on a network such as Facebook ®.

Teams – They prefer to learn and work in teams, for example: a peer‐to‐peer approach where students help each other.

Structure – They like to know what it will take to achieve a goal, for example: rules, priorities and/or procedures for doing a task.

Engagement and Experience – They like interactivity, for example: watching a YouTube ® video on a topic instead of reading slides.

Visual and Kinaesthetic – They are more comfortable in image‐rich environments than text, for example: looking at pictures showing the impact of global warming instead of reading text.

Things that Matter – They readily take part in community activities and want to learn about things that matter, for example: environmental concerns.

2.2 Social computing Social computing, also referred to as social informatics, can be defined as “…the interplay between persons’ social behaviours and their interactions with computing technologies. Social computing involves both science and technology. As a domain of science, we seek to describe the relationships among social behaviours and machines so that we can reduce our uncertainty about how humans and machines will interact. As a domain of technology, we seek to apply social and behavioural science to the design of information technology systems that enable efficient collaboration and support natural social behaviours.” (Dryer et al, 1999:653). Fun and Wagner (2008:243) define social computing as “…any type of computing application that serves as an intermediary or a focus for a social relation…” A layman’s understanding of this term would be that social behaviour, whether desired or otherwise, can be supported through information technology. When focussing on the latter, being the technology domain, then

224

Sumarie Roodt applications such as blogs, wikis and social networks could be used to support collaborative learning as these applications are a form of social software that encourages interaction and collaboration. Initially, social computing had two primary focus points being:

The technological issues, interfaces, user acceptance, and social effects around group collaboration and online communication (Wang et al., 2007:79) and;

The use of computational techniques, principally simulation techniques, to facilitate the study of society and to test out policies before they were employed in real‐world organizational or political situations (Wang et al., 2007:79).

Recently, social computing as a concept has become far broader due to the pervasiveness of information technology (Wang et al., 2007). Wang et al (2007:79) argue that a revised definition of social computing is required in order to include “…computing technologies that support and help analyze social behaviour and help create artificial social agents.” As such, they present a revised definition of social computing as (Wang et al., 2007:79): “The computational facilitation of social studies and human social dynamics as well as the design and use of ICT technologies that consider social context.”

2.3 Youtube© YouTube was founded in February 2005 by Chad Hurley, Steve Chen and Jawed Karim and can be found online at www.youtube.com (Hansen & Erdley, 2009). It is currently one of the biggest hosts for online video content and the third most popular website after Google and Facebook (Tan & Pearce, 2012). YouTube is a popular form of the Web 2.0 technology (Duffy, 2007) and access to YouTube and creating YouTube accounts is free of charge (Agazio & Buckley, 2009). Playlists and channels are some of the popular features which aid in the reduction of search time and playback efficiency (Tan & Pearce, 2012). YouTube has grown steadily from 30,000 viewers in April 2005to 100 million video views per day in July 2006 (Agazio & Buckley, 2009).YouTube was bought by Google in 2006 (Miller, 2010) and in 2007, YouTube had 20 million users and hosted about 60% of all the videos posted on the internet (Hansen & Erdley, 2009).YouTube is a participatory culture, meaning that it does not only host material like sports clips and music videos (Tan & Pearce, 2012), but can also be seen as a repository for user generated video content (Jarrett, 2010). During 2009 YouTube had a total of 258 million users and it was assumed that 65,000 video clips were uploaded on a daily basis, and that every minute, 10 hours’ worth of video were uploaded (Agazio & Buckley, 2009). Currently, YouTube is a website which caters for high volumes of traffic, a platform for broadcasting, a media archive and a social network (T. Jones & Cuthrell, 2011). As a social network, YouTube allows for sharing, uploading and viewing a wide variety of videos hosted online ‐ these videos can be accessed via blogs, handheld devices and websites (Hansen & Erdley, 2009). In 2012, an average 48 hours of video is uploaded every minute and 3 billion views are generated daily (Wattenhofer, Wattenhofer, & Zhu, 2012).

2.4 Student engagement Student engagement has been defined and measured in numerous different ways over the past two decades (Fredricks et al., 2011). Fredericks et al. (2011) state that earlier definitions focus on the behaviour and participation of students. Newer definitions for engagement incorporate the concepts of emotional and cognitive processes (Wolters & Taylor, 2012). Trowler’s (2010, p.7) definition for student engagement is “the time and effort students devote to activities that are empirically linked to desired outcomes of college and what institutions do to induce students to participate in these activities”. From this new definition, scientists have begun to view student engagement as a multidimensional construct which reflects on both internal and external factors (Reeve, 2012). The three dimensions associated to student engagement are elaborated below:

Behavioural engagement refers to a student’s ability to abide by behavioural norms, which include attendance, attention and effort (Trowler, 2010).

Emotional engagement focuses on the extent to which a student experiences affective reactions, which can include interest, enjoyment or a sense of comfort towards educators or extracurricular activities (Wolters & Taylor, 2012).

Cognitive engagement covers a student’s competency and willingness to learn and establish goals (Archambault, Janosz, Fallu, & Pagani, 2009).

225

Sumarie Roodt Reeve (2012) mentions an additional fourth dimension, agentic engagement, which focuses on the student’s willingness to try and enrich a learning experience, as opposed to passively receiving it as a given. For the purpose of this report, when referring to student engagement, the combination of behavioural, emotional and cognitive engagement is meant.

3. Research methodology 3.1 Research approach This study was exploratory in nature, since the purpose of this research was to investigate the impacts of using YouTube in the classroom on student engagement for Net Generation students. Seeing that this study was performed in a South African context, it could assess the phenomena of using YouTube in education in a new light. In order to achieve this goal, a majorly quantitative approach was employed in combination with minor qualitative aspects. Because data was gathered from a fixed sample and analysed, the interpretive paradigm was adopted. The data gathering process was also performed within a certain timeframe, meaning a cross‐sectional approach needed to be followed. The data gathered from the sample aided in concluding whether our findings coincided with those previously reviewed in the literature.

3.2 Research design In order to gather the data relevant to this study, a questionnaire was used. This questionnaire consisted of both quantitative and qualitative questions based on the relevant literature that was previously reviewed. The questionnaire was distributed to the sample via the University of Cape Town’s content management system, Vula. Once the data had been collected, the data was password protected and encrypted. This ensured an adequate level of security in case a breach was attempted. To minimise the risk of theft, the data was only be stored on a maximum of three storage locations at a time. Having the data in more than one place at a time also minimised the risk of loss. On completion of the research paper, the researcher only had the data stored in one location.

3.3 Research sample The data for this research was collected from one sample. This sample consisted of students enrolled in the undergraduate Information Systems course, IT in Business, INF2004F. The class consisted of 616 students, who met the requirements of being potential Net Generation students. These students have also had exposure to YouTube videos as teaching materials in the classroom, which therefore made them ideal candidates for this research. From the 616 students, 156 students completed the questionnaire. Unfortunately not all of the responses were useful – many of the questions vital to this study were either left unanswered, or some of the responses were inconsistent. After having discarded all unusable responses, the sample size decreased to a total of 82. In total, the questionnaire contained 53 questions. Of these 53 questions, 14 are discussed in this paper. These 10 questions were: 1. Were you born between 1982 and 1994? [Yes / No] 2. What is your gender [Male / Female / Prefer not to answer] 3. What is your ethnicity? [White / Black / Coloured / Indian / Asian / Other / Prefer not to answer] 4. Are you digitally literate, i.e. do you interact with technological devices such as computers, laptops or tablet PCs on a regular basis? [Yes / No] 5. If you are digitally literate, how many hours per week do you spend interacting with technological devices? [0 – 5 / 6 – 10 / 11 – 15 / 15 – 20 / 20+ / Unsure]

226

Sumarie Roodt 6. Are you connected to a technological network (mobile phone network or the internet)? (mobile phone network or the internet)? [Yes / No] 7. Did the YouTube videos engage you in the classroom? [Always/Definitely / Frequently/Nearly Always / Occasionally/Seldom / Never / Unsure] 8. In lectures where YouTube videos were not used, were you engaged? [Always/Definitely / Frequently/Nearly Always / Occasionally/Seldom / Never / Unsure] 9. Do you feel that the use of YouTube was successful in class? [Yes / No] 10. Would you recommend the use of YouTube in other courses? [Strongly agree / Agree / Neutral / Disagree / Strongly disagree]

4. Findings and analysis This section details the questions and the results of the questions mentioned in the above section. Question 1: Were you born between 1982 and 1994? All of the respondents answered this question positively, and could therefore be considered candidates for this research, according to Oblinger’s (2003) definition for the Net Generation. Question 2: What is your ethnicity? Table 1 shows that the majority of the respondents, 48.78%, were black, followed by 21.95% of the respondents who were white. There were, however, two respondents that answered “Prefer not to Answer” in order to withhold their ethnicity. Table 1: Ethnicity Ethnicity

Response

Percentage

White

21.95%

Black

48.78%

Coloured

14.63%

Indian

8.54%

Asian

1.22%

Other

2.44%

Prefer not to Answer

2.44%

Total

100.00%

Question 3: What is your gender? Table 2 indicates that the genders of the respondents were perfectly balanced. None of the respondents decided to keep their gender type confidential. Table 2: Gender Gender

Response

Percentage

Male

50.00%

Female

50.00%

Prefer not to Answer

0.00%

Total

100.00%

Question 4: Are you digitally literate, i.e. do you interact with technological devices such as computers, laptops or tablet PCs on a regular basis? All of the respondents indicated that they were digitally literate and did interact with technology regularly, which correlates to the statements made by Kennedy et al. (2010) and Bennet and Maton (2010), who claimed that the Net Generation consists of tech‐savvy individuals.

227

Sumarie Roodt Question 5: If you are digitally literate, how many hours per week do you spend interacting with technological devices? Table 3 summarises the number of hours the respondents spend using the technological devices on average per week. The majority of the respondents, 31.71%, claimed to use technological devices more than twenty hours per week. On the other extreme, 8.54% of the respondents stated that they interact with technology less than five hours per week. Bennet and Maton (2010) use the term digitally immersed – it is difficult to quantify this term, but it is apparent that the majority of the respondents are completely immersed in technology, but there are a few which are only immersed to a minor aspect. Table 3: Hours spent interacting with technology (per week) Hours per week interacting with technological devices

Response

Percentage

Unsure

13.41%

0 – 5

8.54%

6 – 10

8.54%

11 – 15

20.73%

16 – 20

17.07%

20+

31.71%

Total

100.00%

Question 6: Are you connected to a technological network (mobile phone network or the internet)? This question was answered positively by all of the respondents, confirming that they belong to some form of technological network. Question 7: Did the YouTube videos engage you in the classroom? Table 4: Engagement from using YouTube in the classroom Q7: Did the YouTube videos engage you in the classroom?

Unsure

2.44% 5

Never

6.10% 22

Occasionally/Seldom

26.83% 34

Frequently/Nearly Always

41.46% 19

Always/Definitely

23.17% 82

Total

100.00%

The bulk of the respondents, 41.46%, stated that they were nearly always engaged with YouTube. A total of 23.17% of the respondents claimed to have always been engaged when YouTube was used. These findings correlate to the findings made by Harris (2011), who discovered that videos inspire engagement in the classroom. Question 8: In lectures where YouTube videos were not used, were you engaged?

228

Sumarie Roodt Table 5: Engagement without YouTube Q8: In lectures where YouTube videos were not used, were you engaged?

Unsure

3.66% 5

Never

6.10% 52

Occasionally/Seldom

63.41% 18

Frequently/Nearly Always

21.95% 4

Always/Definitely

4.88% 82

Total

100.00%

The majority, 63.41%, stated that they were only occasionally engaged without YouTube. A total of 21.50% of the respondents claimed to have always been engaged when YouTube was used. Only 4.88% could state they were always engaged when YouTube was not used. Question 9: Do you feel that the use of YouTube was successful in class? Table 6 contains the information the respondents supplied on whether or not they found that using YouTube in the classroom was successful. A small number of respondents, 3.66%, strongly disagree with the success of YouTube as a teaching material. Table 6: Success of using YouTube in the classroom Q9: Do you feel that the use of YouTube was successful in class?

Strongly Disagree

3.66% 6

Disagree

7.32% 20

Neutral

24.39% 38

Agree

46.34% 15

Strongly Agree

18.29% 82

Total

100.00%

Question 10: Would you recommend the use of YouTube in other courses? Table 7 contains the information the respondents supplied on whether or not they would recommend further deployment of this teaching method. The majority of the respondents, 40.24%, agreed that this teaching

229

Sumarie Roodt method was worthy of recommendation. A small number of respondents, 4.88%, strongly disagreed with the recommendation of YouTube as a teaching material in other courses. Table 7: Recommendation of YouTube for other courses

Strongly Disagree

Disagree

Neutral

Agree

Strongly Agree

Total

Q10: Would you recommend the use of YouTube in other courses? 4 4.88% 6 7.32% 16 19.51% 33 40.24% 23 28.05% 82 100.00%

5. Conclusion The main objective of this research was to discover the impact of YouTube in the classroom for Net Generation students. This research was conducted by examining the students enrolled in IT in Business, INF2004F, who were all exposed to YouTube videos in the classroom. This sample does not represent the entire South African populace. The instrument used was a questionnaire, which was majorly quantitative with minor qualitative aspects. The use of YouTube videos in class was a success and the majority of the respondents would agree that using videos to illustrate concepts is an effective teaching method and would recommend this teaching method to be used in other courses. The analysis further indicated that YouTube videos contributied to the overall engagement of the students. These findings are consistent with those made by Harris (2011), Berk (2009) and Maragret and Erdley (2009).

References Abell, C. H. (2011). Using YouTube to bridge the gap between baby boomers and millennials. Journal of Nursing Education, 50(5), 299. Agazio, J., & Buckley, K. M. (2009). An untapped resource: Using YouTube in nursing education. Nurse Educator, 34(1), 23. Archambault, I., Janosz, M., Fallu, J. S., & Pagani, L. S. (2009). Student engagement and its relationship with early high school dropout. Journal of Adolescence, 32(3), 651‐670. Bennett, S., & Maton, K. (2010). Beyond the ‘digital natives’ debate: Towards a more nuanced understanding of students' technology experiences. Journal of Computer Assisted Learning, 26(5), 321‐331. Dryer, D.C., Eisbach, C., Ark, W.S. (1999) “At what cost pervasive? A social computing view of mobile computing systems”, IBM Systems Journal, Vol. 38, No. 4. Duffy, P. (2007). Engaging the YouTube google‐eyed generation: Strategies for using web 2.0 in teaching and learning. European Conference on ELearning, ECEL, 173‐182. Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments. issues & answers. REL 2011‐no. 098. Regional Educational Laboratory Southeast, , 88. Fun, R.K., Wagner, C. (2008), “Weblogging: A study of social computing and its impact on organizations.” Decision Support Systems Vol 45 2008, pp 242–250. Hansen, M., & Erdley, S. (2009). YouTube and other web 2.0 applications for nursing education. Online Journal of Nursing Informatics (OJNI), 13(3), 1‐20. Harris, M. (2011). Using YouTube to enhance student engagement. Available from: http://www.ponce.inter.edu/html/retencion/using%20youtube.pdf. Retrieved February, 3, 2012.

230

Sumarie Roodt Helsper, E. J., & Eynon, R. (2010). Digital natives: Where is the evidence? British Educational Research Journal, 36(3), 503‐ 520. Jarrett, K. (2010). YouTube: Online video and participatory culture. Continuum, 24(2), 327 ‐330. Jones, C., Ramanau, R., Cross, S., & Healing, G. (2010). Net generation or digital natives: Is there a distinct new generation entering university? Computers & Education, 54(3), 722‐732. Jones, T., & Cuthrell, K. (2011). YouTube: Educational potentials and pitfalls. Computers in the Schools, 28(1), 75‐85. Kennedy, G., Judd, T., Dalgarno, B., & Waycott, J. (2010). Beyond natives and immigrants: Exploring types of net generation students. Journal of Computer Assisted Learning, 26(5), 332‐343. Oblinger, D. (2003). Boomers, Gen‐Xers and Millenials: Understanding the new students (pp. 37–47). Educause Review July/August 2003. Oblinger, D., & Oblinger, J. (2005). Educating the Net Generation. Boulder: EDUCASE. Page, R. D. M. (2010). Wikipedia as an encyclopaedia of life. Organisms Diversity & Evolution, 10(4), 343‐349. Palfrey, J., Gasser, U. (2010), Born Digital: Understanding the First Generation of Digital Natives, New York, Basic Books. Reeve, J. (2012). A self‐determination theory perspective on student engagement. In S. L. Christenson, A. L. Reschly & C. Wylie (Eds.), The handbook of research on student engagement (pp. 149‐172). New York, NY: Springer Science. Roodt, S, Joubert, P, De Villiers C, Oelofse, N. (2009): “Collaborative learning in the Web 2.0 environment: An Undergraduate Case Study.” Proceedings of EDULEARN 2009, July 6 – 8, Barcelona, Spain, ISBN: 978‐84‐612‐9802‐0. Roodt, S., & De Villiers, C. (2011). Using YouTube as an Innovative Tool for Collaborative Learning at Undergraduate Level in Tertiary Education. Proceedings of the AIS SIG‐ED IAIM 2011 Conference. Tan, E., & Pearce, N. (2012). Open education videos in the classroom: Exploring the opportunities and barriers to the use of YouTube in teaching introductory sociology. Research in Learning Technology, 19(3), 125‐133. Tapscott, D. (2009). Grown up digital McGraw‐Hill New York. Trowler, V. and Trowler, P. (2010) Student engagement literature review. Available from: www.heacademy.ac.uk/resources/detail/ourwork/studentengagement/Research_and_evidence_base_for_student_ engagement. Retrieved February, 3, 2012 Wang, F.Y., Carley, K.M., Zeng, D.,Mao, W. (2007), “Social Computing: From Social Informatics to Social Intelligence”, IEEE Intelligent Systems, Vol. 22, No. 2, pp 79‐83. Wattenhofer, M., Wattenhofer, R., & Zhu, Z. (2012). The YouTube social network. Wolters, C. A., & Taylor, D. J. (2012). A self‐regulated learning perspective on student engagement. In S. J. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 635–651). New York: Springer.

231

Knowledge Creation in Students’ Software Development Teams Mzwandile Shongwe Department of Information Studies, University of Zululand, Kwadlangezwa, South Africa shongwem@unizulu.ac.za Abstract: Knowledge creation is a process whereby individuals, groups and organizations engage in activities that generate new ideas. Knowledge creation can be a process or an output/ outcome. It is the subject of study in many fields; in computing, these studies tend to focus more on knowledge creation as a process and less on knowledge creation as an output/outcome. These studies also focus mainly on knowledge creation in business organizations and ignore public institutions such as higher education institutions. They also focus on professional teams and ignore novice developers such as students. The study addressed these short‐comings by investigating knowledge creation in students’ software teams. An ethnographic study was conducted on six Management Information Systems (MIS) students’ software development teams who were conducting a systems development term project at a University. Data were collected over a period of three month through participant observation, interviews and open ended questionnaires. The study answered the following questions: what knowledge creation activities take place in students’ teams? What channels of communication facilitate knowledge creation in student’s teams? What information sources do students use to create knowledge? And what challenges teams face when they create knowledge? The results reveal knowledge creation activities such as problem definition, brainstorming, programming and documentation. Furthermore, students use the Internet, books, class notes, class presentations, senior students and professional software developers as sources of knowledge. Mobile devices such as mobile phones and Blackberry devices facilitate knowledge creation. Challenges or barriers to knowledge creation in students’ teams were also discovered. They are: lack of material and financial resources, lack of technical skills, lack of time, students staying off‐campus, and ambivalent team members who do not do work. The conclusion drawn from this study is that students’ teams are capable of creating knowledge (a working system) just like professional teams. Keywords: knowledge creation, software development, computing, ethnography, students’ teams

1. Introduction According to Mitchell and Boyle (2010), knowledge creation as a field of study has gained a lot of popularity in recent years. In the field of computing, software development is regarded as knowledge creation (Bailin 1997), and a number of studies have been conducted in computing to investigate the knowledge creation phenomenon. Most of these studies have focused on private and business organizations, largely ignoring public institutions such as higher education institutions (except in the case of open‐source software development). Such studies are also inclined to focus on professional teams in software development and ignore novice/students’ software development teams. Computing students in their final year of study are given real life software development projects to complete and present/deliver to real clients (Strode & Clark 2007). There are many unresolved problems noticed in such studies. For example, how do the student teams in computing create knowledge (develop software)? In other words, what knowledge creation activities take place at individual and team levels? What channels of communication facilitate knowledge creation in students’ teams? What information sources do students use to create knowledge? And what challenges do teams face when they create knowledge? This study answered these questions in the context of knowledge creation in software development. This paper is divided into a brief literature review on knowledge creation; the methodology; results and discussions of the study; and the conclusion and recommendations for further research.

2. Literature review This section presents a brief literature review on knowledge creation, distinguishing between data, information and knowledge, followed by a discussion of knowledge creation and knowledge creation in software development.

2.1 Data, information, knowledge The concepts of knowledge, data and information may seem similar, but their meaning is not uniformly understood. Data is unprocessed information (Hey 2004); bits and numbers with no meaning (Rumizen 2002). Information is data that has been processed into a useful form and that can be used for decision making. Chaffey and Wood (2005) describe information as data that adds value or contributes to the understanding of a subject, while knowledge is “information possessed in the mind of individuals: it is personal information

232

Mzwandile Shongwe related to facts, procedures, concepts, interpretations, ideas, observation and judgment” (Alavi & Leidner, 2001, p.109). Knowledge can be viewed from six perspectives: as a state of mind, an object, a process, a condition of having access to information, or a capability (Alavi & Leidner 2001). In software development, knowledge can be an object, derived from a process, where the object is the output or end product (the software).The actions involved in software development are knowledge creation processes. Knowledge can either be tacit or explicit (Polanyi 1962; Nonaka & Takeuchi 1995). Tacit knowledge is derived from experience and embodies beliefs and values‐‐what the “knower knows“(Nonaka & Takeuchi 1995). Tacit knowledge is personal and difficult to formalize, making it difficult to communicate and share with others (Gladstone 2000; Li & Gao 2003; Elfving & Funk 2006). Explicit knowledge, on the other hand, is knowledge that has been articulated, more often than not in the form of text, tables, diagrams, and product specifications (Nickols 2000). In software development, tacit knowledge is the hidden knowledge in the experiences and mind of the developer, while explicit knowledge is the end product and the system’s documentation.

2.2 Knowledge creation Knowledge creation refers to the series of processes and activities that add value to an outcome, such as a product, service or process (Mitchell & Boyle 2010). It involves the definition of a problem and makes use of complex and discontinuous events in order to be able to deal with the situation accordingly (Parent et al. 2000; Styhre, Roth & Ingelgard 2002). As stated earlier, knowledge creation can be a process or an output/outcome. When defined as an output/outcome, knowledge creation refers to the generation of new ideas that reflect a significant enrichment of existing knowledge. Here, knowledge creation is defined as a process in which activities and initiatives are undertaken to generate new ideas or objects. It could also mean that new knowledge is diffused, adopted, and embedded as new products, services and systems (Mitchell & Boyle 2010).Wan et al. (2010, p. 488) state that knowledge is created in five steps: Sharing tacit knowledge, Creating concepts, Justifying concepts, Building a prototype, and Cross leveling knowledge.

2.3 Software development as knowledge creation As stated earlier, software development is knowledge creation (Bailin 1997). Software development is a knowledge intensive task that involves the processes of discovery and invention, accumulation, analysis, cognition, and integration (Bailin 1997; Dingsoyr 2002; Mitchell & Nicholas 2006). This means that software development processes (systems analysis, coding, documentation, testing, etc.) are knowledge creation processes. During these activities, individuals and teams acquire knowledge from different sources; apply their previously acquired knowledge; store the knowledge acquired in databases, routines, procedures, etc.; and share their knowledge with others to produce an outcome or output in the form of a software product. A number of studies have been conducted to investigate the knowledge creation phenomenon in computing. A few examples here will suffice, beginning with Arent and Norbjerg’s (2000) claim that Software Process Improvement (SPI) processes are knowledge creation activities; Kess and Haapasalo’s (2002) investigation into knowledge creation through a software project review process, where they came up with a tool to improve the software process; Klint and Verhoef’s (2002) study on how the principles of knowledge management could be applied in knowledge creation; Linden and Cybulski’s (n.d.) study on knowledge creation in pattern mining which led to the enrichment of the knowledge creation framework proposed by Wickramasinghe and Lichtenstein; Morner and von Krogh’s (2009) exploration of the conditions under which successful knowledge creation may take place; and Wan et al.’s (2010) recent investigation into the Requirement Elicitation Process (REP), where they concluded that REP is a knowledge creation process. While a lot of studies (Host, Regnell, & Wohlin 2000; Ow, 2008 ; He 2009, to name but a few) have been conducted in computing on students’ software development teams, these studies focus on many aspect of software engineering, but very few of these have focused on knowledge creation. Therefore the aim of the study was to investigate a students’ teams perspective of knowledge creation.

3. Theoretical foundations A number of theories are used to study knowledge creation in organizations. Widely used theories include the learning organization (Senge 1994), communities of practice (Wenger 1998), the theory of organizational knowledge creation (Nonaka and Takeuchi 1995) and the organizational learning theory (Angyris and Schon 1978). These theories explain how individuals learn, which influences group learning and finally organizational

233

Mzwandile Shongwe learning. Some of the above mentioned theories have been used in Computing. Other theorists have come up with knowledge creation theories specific to Computing. One example is Lee and Cole’s (2003) community‐ based model of knowledge creation in which they explain how knowledge is created by a software community of practice operating outside formal organizational boundaries. Among these theories, the organizational knowledge creation theory is the most suitable for the study thus it was selected to inform the study. This is because it explains how knowledge (an object/ end product such as software, car, etc.) could be created through the conversion of tacit (experiences) and explicit (books, notes, etc.) knowledge through socialization, externalization, combination, and internalization (SECI) processes. The theory also explains how knowledge is created by individuals who transfer it to teams, how transferred among team members, and the different information sources used to create knowledge.

4. Methodology The study adopted an ethnographic research approach (Myers 1999; Oates 2006; Saunders et al. 2009). 24 MIS students who were engaged in a software development project at a university were sampled. The students were divided into six teams; convenience sampling was used to select the teams. The teams were instructed to develop an e‐commerce system. Data was collected over three months through participant observation, interviews, and open‐ended questionnaires. Observation was conducted during group meetings, presentations, and lectures to see how teams interacted, how and what ideas they shared, and their impact on the project. Group interviews were conducted to determine what information sources and communication channels the teams relied on and what challenges the teams were facing in the course of the project. At the end of the project, open‐ended questionnaires were administered to assess the success of the project and to confirm the information obtained during interviews and observation. The researcher attended every lecture and every presentation that the teams attended. Some team meetings (during development) were also attended. The aim was for the researcher to be part of the group as if he were one of them. During this time, detailed notes were taken on what was learnt and its importance and the impact it had on the completion of the project. This was usually in the students’ computer laboratory and in the library. Qualitative content analysis was then used to analyze the responses.

5. Results and discussion 5.1 Knowledge creation activities This section reports on the knowledge creation activities that MIS students undertake at individual and group/team levels. 5.1.1 Individual knowledge creation activities Individual team members were found to perform a number of activities during the knowledge creation process. Key among them were coding, documentation, preparing power points for presentations, designing the user interface, research and information gathering, sourcing finance, and other relevant activities assigned to the individual. One respondent explained their role as follows: “I was mainly the website designer, but I also did documentation such as project plan, feasibility study and other activities [sic]”. Another reported, “As an individual, I created forms in the prototype [sic]”. Others listed other minor activities such as minute writing and arranging future meetings. The importance of individual knowledge activities was emphasized by all group members. One member stated that, “The contribution of each member was great. For every task we had to break it down to individual activities. In so doing we shared ideas which enabled us to finish the project quicker.” Knowledge creation literature highlights the importance of the individual in knowledge creation (Hedlund 1994; Nonaka & Takeuchi 1995). In students’ software development teams, individual team members have unique qualities that contribute immensely to knowledge creation. This is confirmed by the contributions that individual members made towards the accomplishment/completion of the project. 5.1.2 Team knowledge creation activities A number of activities were conducted as a team. Most of these activities occurred during team meetings, and key among these was problem definition. Students initially lamented that they were unsure as how to proceed with the project; “I had no idea what I was supposed to do”, and “We did not have a clue on what to do.” The

234

Mzwandile Shongwe teams then converged and discussed the problem, and sought assistance from the lecturer and senior students. They reported that, “It was after we met as groups that we shared ideas and came to understand what we had to do.” Morne and von Krogh (2009) state that in knowledge creation, the first step for the programmer lies in overcoming the software problem that he/she is confronted with. Nonaka and Takeuchi (1995) refer to such a state as ‘intentional creative chaos’. According to Nonaka and Takeuchi (1995), ‘intentional creative chaos’ describes a situation where management creates a sense of crisis in the organization by proposing challenging goals. Organizational members respond by defining the problem and finding solutions to the problem. In students’ teams, the lecturer evokes this sense of crisis by confronting students with a major real life problem that they must solve in a short space of time. They must then deliver this solution or product to a real customer. It is only upon meeting as a team that the students are properly able to define and understand the problem and plan how to solve it. Other activities that the students performed as a group include coding, documentation and presentation preparations. Team knowledge creation is emphasized by a number of knowledge creation theories. These theories (Hedlund 1994; Senge et al. 1994; Nonaka & Takeuchi 1995; Wenger 1998) confirm that teams play a crucial role in knowledge creation. It is at team level that ‘rough’ ideas are refined and developed into knowledge.

5.2 Information sources for knowledge creation A number of information and knowledge sources were used by the student teams during the knowledge creation process. The sources used include class presentations, lecturer and lecture notes, books, the Internet, and external sources (e.g. senior students and professional software developers). Class presentations were considered to be a good source of information. Upon achieving a major milestone such as the completion of system analysis, budgeting, prototyping, documentation, etc., the group would present the outcome to the whole class and the lecturer. Other teams and the lecturer would then interrogate the presenting group by asking questions and commenting on the outcome. Presenters were criticized, praised and/or advised on how to improve the project. One student stated that: “Presentations helped us to know what we did right, where to correct and how to improve the project.” Another concurred: “Presentations helped us to get clarity as to what is really required from us”. All the groups agreed that in one way or another, the presentations helped them to gain new knowledge about the project. Teams also stated that they gained knowledge from the presentations of other teams. They learnt communication and presentation skills and how to work under pressure, and the presentations boosted their morale when they realized that they were making progress. On a personal level, team members indicated that class presentations helped them to learn from others and improved their communication skills. The lecturer also played a crucial role in the knowledge creation process. Other than giving them the problem to solve, the lecturer was also involved in helping the teams solve the problem. In addition to teaching, the lecture played the role of advisor and mentor. At times, classes would be conducted to teach students how to carry out certain activities of the project. Before the start of each major milestone (e.g. systems analysis), the lecturer would conduct a lecture on the next activity. This helped students understand what was required of them and how to tackle the problem at hand. The lecturer also played a crucial role during the presentations. He would criticize the students’ work, offer advice on how the work should be done, and provide encouragement/credit to teams that did well. Students would also consult the lecturer whenever they needed advice or when they encountered problems in their projects. This was confirmed in many of the comments: “The lecturer gave us motivation to finish the project”; “The lecturer helped by explaining what is expected”; and “the lecture made us to attend classes, [sic] and provided us with technical skills”. However, not all students were happy with the lecturer’s role. These students indicated that the lecturer did not play any role in their knowledge creation process: “The lecturer did not play any role in the successful completion of the project. We struggled while he was present”; and “The lecturer had little impact on the project” were some of the comments students gave. Students indicated that they relied a lot on the Internet for information. Most teams indicated that they used the Internet to conduct research on the project’s activities (e.g. how to budget, how to perform systems analysis, etc.) and also to find technical information (e.g. coding information). They also surfed the Internet to

235

Mzwandile Shongwe find solutions to specific technical problems. One team indicated that they used the Internet to find information on how to set‐up the Apache, MySQL, PHP for Windows (WAMP) interface. The w3schools website was said to have provided much support to the teams. They also relied on books, senior students and professionals to complete their projects. Because this was an MIS group, most teams relied on Computer Science senior students for technical support, for example when they needed to link a database to a website or during other coding activities. Surprisingly, they also sought help from competing groups: “We sought help from other groups to do some of the activities”, they reported. Students’ teams therefore appear to use readily available sources of information for knowledge creation. Professional teams use slightly different (more formal) sources of information, such as in‐house databases (experience repositories) containing knowledge from previous projects (Schneider 2009; Basili, Caldiera, Rombach n.d.). But it can be argued that the sources that students use are good enough because they enabled the teams to create knowledge.

5.3 Channels of communication that facilitate knowledge creation The results revealed that the teams used different communication channels in the process of knowledge creation. The most used channels for information transfer were mobile devices such as mobile phones (e.g. Blackberrys) and the Internet. Teams indicated that they relied on emails, chat services such as Whats Application (WhatsApp), Blackberry Messenger (BBM), Mixit, and social networking sites (Facebook, Twitter) to communicate their ideas. They stated that they were able to use these channels at a time and place that was convenient, allowing them to share information faster than with other devices.

5.4 Challenges of knowledge creation The teams cited a number of challenges in the knowledge creation process. The most cited challenge was financial. The students explained that before the start of a project, the teams first had to identify the client that they had to develop the system for. This client had to be rural based, meaning that the teams had to travel to rural areas to find the client and collect information about their business before developing the system. All the teams indicated that they did not have enough money for this exercise. The average distance that they had to travel was 50 kilometers, and they did not have enough money for travelling and subsistence. One team stated it thus: “We did not have money to go and interview the client the way we could have liked. We also did not have money for printing our documents.” They also lacked financial resources to perform basic activities such as photocopying and printing documents. One team stated that, “As a group, we did not have the funds to help us successfully achieve our goal.” Another challenge cited by the teams was the lack of technical skills. As they were amateur teams, this challenge did not come as a surprise. For example one team stated that: “We had no skills and experience of web design.” The main technical problem that most teams cited was how to connect a database to the website (the WAMP platform). Lack of material resources was another challenge that faced the teams: “We encountered the challenge of limited resources that would have helped with the organizing, planning, and research about the project.” The material resources that they cited included computers, software, and a computer laboratory specially assigned for their project. Some teams only had one laptop computer to work with. The university computer laboratory did not have PHP software installed, which made it difficult for teams to complete the project. The teams also complained about the over‐crowded school laboratories. One group indicated that: “Time pressure was another big challenge that students’ teams faced while creating knowledge.” Teams stated that there was not enough time to complete the project. They complained that each outcome was to be submitted sooner than they would have liked. They also complained that they also had other courses to complete: “I had to juggle between activities of the project and other courses” and “The challenge was to balance between the amount of work the project had and studying for other courses.” Not all of the students resided on campus. Some teams were made‐up of on‐campus and off‐campus students. Some of the off‐campus students had to travel frequently to the campus for group meetings. At times they had to leave meetings early in order to travel off‐campus; “Since I am staying off‐campus, my difficulty was always

236

Mzwandile Shongwe travelling to the university and going back in the late hours of the day when it is not safe out there,” one student said. This caused challenges because it delayed the completion of the project. Some group members could not turn‐up for meetings which also affected the quality of the ideas contributed to the team. Other teams complained about lazy team members who did not complete the activities assigned to them. Strode and Clark (2007) report similar challenges in their study on students’ projects. They found that students’ projects were constrained by factors such as time and commitment, experience level, scope and complexity, technology, and the need to meet assessment criteria. Time, commitment, and financial and technology issues appear to be the more serious challenges facing students. The faculty should look into how these could be addressed, although these challenges are also faced by professional teams (Rus & Lindvall 2002).

5.5 Knowledge gained from the project All the teams indicated that they learnt a lot from the project. They indicated that the project improved their knowledge and experience in software development (“I have learned how to create a complete website”). Some respondents indicated that they had learnt to work in a team, a skill that is essential in the job market: “I believe I gained both knowledge and experience when it comes to programing and working in a team.” Others indicated that they had learnt to work under pressure and work in a group to achieve a singular goal.

6. Conclusion and further research The study contributes to literature by showing how knowledge is created in students’ software development teams. Most knowledge creation literature covers knowledge creation in professional teams in business organizations. This paper has presented a different perspective by focusing on knowledge creation in students’ software development teams in an educational institution. Several conclusions can be drawn from the study: Students’ software development teams can create knowledge (a working system) just like professional software development teams; Knowledge is created by the whole team, but the knowledge creation process begins with individual team members; Students rely on class presentations, lectures, the Internet and senior students and professional developers as major sources of information in the knowledge creation process; and Lack of technical skills, limited time, and lack of resources are major hindrances to knowledge creation. Further research is recommended to investigate the reliability and quality of the knowledge that students create and also to investigate the impact of the challenges to knowledge creation on the completion and quality of the knowledge produced.

References Alavi, M., and Leaidner, D.E. (2001) Knowledge Management Systems: issues, challenges and benefits. In S. Barnes, Knowledge Management Systems Theory and Practice, Oxford: Thompson Learning, pp 15‐32. Arent, J., and Norbjerg, J. (2000) Software Improvement as Organizational Knowledge Creation: A Multiple Case Analysis. Proceedings of the 33rd International Conference on System Sciences, 4‐7 January 2000, Hawaii, USA. Argyris, C, & Schon, D. A. (1978) Organizational learning, Addison‐Wesley, Reading. Bailin, S. (1997)“Software Development as Knowledge Creation”,International Journal of Applied Software Technology, Vol3, No.1, March 1997. Basili, VR., Caldiera, G., and Rombach, H.D. (n.d.) “The Experience Factory”, [online] available at: <http://agde.cs.uni‐ kl.de/teaching/qmss/ws2009/material/excercise/experienceFactory.pdf>[Accessed 23 November 2012] Chaffey, D., and Wood, S. (2005) Business Information Management: Improving Performance Using Information Systems, Pearson Education Limited, London. Dingsoyr, T. (2002)“Knowledge Management in Medium‐Sized Software Consulting Companies”,Empirical Computing, Vol7, No 4, December, 383‐386. Elfving, S., and Funk, P. (2006, January 25‐26)Enabling Knowledge Transfer in Product Development and Production – Methods and Techniques from Artificial Intelligence. Paper presented at the 1st Nordic Conference on Product Lifecycle Management, Goteborg, Sweden. Gladstone, B. (2000)From Know‐How to Knowledge, J.W. Arrow Smith, USA. Hedlund, G. (1994) “A Model of Knowledge Management and the N‐Form Corporation”, Strategic Management Journal, Summer, Vol 15, Special Issue, pp. 73‐90. He, J. (2009) “Free‐Riding in Students Software Development teams: An Exploratory Study”. AMCIS 2009 proceedings, [online]Available at: <http://aisle.aisnet.org/amcis2009/575>[Accessed 11 December, 2012]. Hey J. (2004)“The Data, Information, Knowledge, Wisdom Chain”, [online]Available at: <http://scholar.google.co.za/scholar?q=The+Data,+Information,+Knowledge,+Wisdom+Chain+%2Bhey+2004&hl=en &um=1&ie=UTF‐8&oi=scholart>[Accessed 16 July, 2008].

237

Mzwandile Shongwe Host, M., Regnell, B. and Wohlin, C. (2000) “Using Students as Subjects‐A Comperative Study of Students and Professionals in Lead‐time Impact Assessment”, Empirical Software Engineering, Vol5, pp 201‐204. Kess, P., and Haapasalo, H. (2002)“Knoweldge creation through a project review process in software production”.Int. J. Production Economics, Vol80, No. 1, November, pp 49‐55. Klint, P. and Verhoef, C. (2002)“Enabling the creation of knowledge about software assets”,Data & Knowledge Engineering, Vol41, No. 2‐3, June, pp 141‐158. Lee, G.K., and Cole, R.E., (2003)”From a Firm‐Based to a Community‐Based Model of Knowledge Creation: The Case of the Linux Kernel Development”, Organization Science, Vol 14, No 6, November‐December, pp 633‐694. Li, M., and Gao, F. (2003)“Why Nonaka highlights Tacit Knowledge: a Critical Revie”,.Journal of Knowledge Management,Vol7, No.4, pp6‐14. Linden, Tanya and Cybulski, Jacob (2009)“Knowledge creation in an application domain : a hermeneutic study”, in ICKM 2009, Proceedings of the 6th International Conference on Knowledge Management : Managing knowledge for global and collaborative innovations, ICKM, Hong Kong, China, pp. 1‐13. Mitchell, R., and Boyle, B. (2010)“Knowledge Creation Measurement methods”Journal of Knowledge Management, Vol 14, No.1,pp 67‐82. Mitchell, R., and Nicholas, S. (2006)“Knowledge Creation in Groups: The Value of Cognitive Diversity, Transactive Memory and Open‐mindedness Norms”,Electronic Journal of Knowledge Management, Vol4, No.1, pp67‐74. Morner, M., and von Krogh, G. (2009)“A Note on Knowledge Creation in Open‐Source Software Projects: What Can We Learn from Luhmann’s Theory of Social Systems”,Systems Practice and Action Research, Vol22, No. 6, December, pp 431‐443. Myers, M.D. (1999)“Investigating Information Systems with Ethnographic Research”, Communications of the AIS, Vol2, No.23, December, pp 1‐20. Nickols, F. W. (2000) The knowledge in knowledge management. In J.W. Cortada& J.A. Woods eds., The knowledge management yearbook 2000‐2001. Boston, MA: Butterworth‐Heinemann, pp. 12‐21. Nonaka, I., and Takeuchi, H. (1995)The Knowledge‐Creating Company, Oxford University Press, New York. Oates, B.J. (2006) Researching Information Systems and Computing.SAGE, Los Angeles. Ow, S.H. (2009). “Improving Soft skills of University Students through Software Development Team Projects”, Asian Social Sciences, Vol4, No.4, April, pp135‐148. Parent, Michael,Gallupe, R. Brent, Salisbury, W.M., David, and Handelman, Jay M. (2000)“Knowledge creation in focus groups: can group technologies help”? Information & Management, Vol38, No.1, October, pp 47‐58. Polanyi, M. (1962)“Tacit Knowing: Its Bearing on Some Problems of Philosophy”,Reviews of Modern Physics, Vol34, No.4, October, pp 601‐616. Rumizen, M.C. (2002) The Complete Idiot’s Guide to Knowledge Management, CWL Publishing Enterprise, USA. Rus, I. and Lindvall, M. (2002)“Knowledge Management in Computing”,IEEE Software, Vol 19, No.3, May/June, 2002, pp 26‐ 38. th Saunders, M., Lewis, P., and Thornhill, A. (2009)Research Methods for Business students (4 ed.). Prentice Hall, London. Senge, P.M., et al. (1994) The fifth Discipline Fieldwork: Strategies and tools for building a Learning organization, Doubleday Inc. Publishers, New York. Strode, D.E. and Clark, J. (2007)“Methodology in Software Development Capstone Projects”, Proceedings of the 20th Annual NACCQ, Nelson, New Zealand, 8‐11 July 2007. [online] available at:<http://hyperdisc.unitec.ac.nz/naccq07/proceedings/index.html>[Accessed 27 June 2012] Styhre, Alexander, Roth, Jonas, and Ingelgard, Anders (2002)“Care of the other: knowledge creation through care in professional teams”,Scandinavian Journal of Management, Vol18, No.4, December, pp 503‐520. Wan, J., Zhang, H., Wan, D., and Huang, D. (2010)“Research on Knowledge Creation in Software Requirement Development”,Journal of Computing& Applications, Vol3, May, pp 487‐494. Wenger, E. (1998) Communities of Practice: learning, Meaning, and Identity, Cambridge University Press, Cambridge.

238

Defining Social Technologies Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė Mykolas Romeris University, Lithuania aelita@mruni.eu tamosiunaite.ruta@mruni.eu izaleniene@mruni.eu Abstract: The popularity of social technologies continues to grow in the society. The term ‘social technology’ is often referred to digital social networks such as Facebook, Twitter, LinkedIn, etc. In order to this a redefinition of this concept based on the original definition is needed. Nowadays the concept of social technologies has several aspects which destabilize the dominant status of technology. It emphasizes social sciences and the humanities as society shapers, reconsiders the strength of social dimension in technological sciences. The aim of this paper is to provide insight into the concept of social technologies, to develop its meaning in information and knowledge society by analyzing new needs and application forms. Design/methodology/approach – The research results have contributed to the knowledge of the concept of social technologies. Based on the analysis of scientific literature and results of empirical research in the Focus group as well as Content analysis, theoretical framework for defining the concept of social technologies was developed. Research limitations/implications – The research is limited in a few aspects. To understand the concept of social technologies more deeply and to develop technological perspectives in social sciences a broader theoretical and empirical research is necessary. In order to generalise the research findings, it is recommended that further research includes different dimensions from the perspective of other fields of science. Keywords: social technologies, social engineering, social media, millennial generation

1. Introduction In today's world, when the time cost is substantial and competition is fierce, the importance of new technologies is increasing. Surveys conducted by analysts such as Forrester Research demonstrate that popularity of social technologies continues to grow in the society from politics to personal communication, from production of building materials to state management. The term ‘technology’ could no longer be understood in its narrow sense as manufacturing processes and equipment necessary for production. Technical definition of social space is a broader concept and can be modified to the next level of technology that can be defined as a social problem in search for and implementation of the decision theory approach (Derksen et al, 2012). The concept of social technologies became established in different fields of science in recent decades. It includes a new research area for interdisciplinary research practitioners as well as major practical application and the potential to make a real impact on social reality. Social technologies – “the computer code and the services that enable online social interaction – are, essentially, the product of 40 years of technology evolution and the fulfilment of a long‐held vision of what computers and digital technology could do” (Chui et all, 2012). In their research authors also state “that the Web’s growth in reach and capability, and as a medium for interaction, set the stage for the explosive growth of social technologies”. Social technologies unleash creative forces among users and enable new relationships and group dynamics. In the hyperactive world people can feel immediate benefits in connecting with the right peers, getting answers to questions and finding information. The Millennials, also known as the Millennial Generation (Generation Y), are people born between 1980 and 2000 (Alberghini, et al., 2010). They are now entering the workplace and have different habits and communication forms than the older generation. According McKinsy Global Survey “Americans spend approximately 11 hours a day communicating or consuming messages in various ways, including in‐person, watching TV, reading, and using e‐mail” (Chui et all, 2012). Just as email and instant messaging replaced the phone call, social technologies could have a similar effect in changing communication dynamics (De Gennaro, 2010). “Freed from the limitations of the physical world, people are able to use social technologies to connect across geographies and time zones and multiply their influence beyond the number of people they could otherwise reach” (Bughin et al, 2011). Koo et al (2011) emphasized, that “even though the term ‘social communication technologies’ is most commonly used to refer to new social media such as Twitter and Facebook, a redefinition of this concept based on the original definition is needed”. Nowadays the concept of social technology has several aspects which destabilize the dominant image of technology. It emphasizes social sciences and the humanities as society shapers, reconsiders the strength of ‘soft technologies’, and restores focus to human actors. It means

239

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė that social technology is increasingly salient as an object of study for social sciences: sociality is more and more something that people create technically (Derksen et al, 2012). “The instrumental, techno‐scientific approach to social life is not the exclusive province of social scientists any more, it demands all the more attention as an object of study” (Mayer, 2009). The purpose of this research paper is is to provide insight to the concept of ‘social technologies’, and to develop the role of social technologies in information and knowledge society by analyzing new needs and application forms of social technologies. The theoretical framework for defining the concept of ‘social technologies’ was developed based on the analysis of the scientific literature and the results of empirical study in the Focus group and content analysis.

2. Theoretical insights to the concept of social technologies The term ‘social technology’ was first mentioned at the University of Chicago by A. W. Small and Ch. R. Henderson around the end of the 19th century (Wikipedia, 2012). Henderson (1895) had used the term ‘social art’ for methods by which improvements to society are and may be introduced; “social scientists are the ones who make predictions and social art is what gives directions”. The term ’social technology’ has dual meaning (Li and Bernoff, 2011): as a term from its introduction was related to ‘social engineering’ (Schotter, 1981; Sugden, 1989; North and Wallis, 1994; Nelson and Sampat, 2001; Nelson, 2002; Pelikan, 2003, Leichteris, 2011), and since 21st century it gained another meaning as a ‘social software’ (Johannessen et al., 2001; Andersen, 2011; Duarte, 2011; Leibetseder, 2011; Bugin et al, 2011; Derksen et al, 2012). In modern understanding of social technology, it could be applied for various purposes, such as decision making, knowledge sharing, etc. Social technologies can be defined as any technologies used for goals of socium or with any social basis, including social hardware (traditional communication media), social software (computer mediated media), and social media (social networking tools) (Alberghini, et al., 2010). Chui et al (2012) defines social technologies “as digital technologies used by people to interact socially and together to create, enhance, and exchange content”. Social technologies distinguish themselves through the following three characteristics (Bugin et al, 2011):

they “are enabled by information technology”;

they “provide distributed rights to create, add, and/or modify content and communications”;

they “enable distributed access to consume content and communications”.

Social technologies include a wide range of various technological instruments that can be used by people, private or public sector organizations, or as an interaction tool between them. They include many of the technologies that are classified as ‘’social media’’, ‘’Web 3.0’’, and ‘’collaboration tools’’ (see Figure 1). All these types of social technologies can be described in terms of three dimensions (Johannessen et al., 2001):

Richness: “the ability to convey verbal and nonverbal cues, and facilitate shared meaning in a timely manner”;

Interactivity: “the extent to which rapid feedback is allowed”;

Social presence: “the degree to which virtual team members feel close to one another”.

Different technologies may be better applicable for conveying data‐information‐knowledge, while others are better suited for convergence‐related tasks such as making decisions. For example, “e‐mail facilitates well the fine‐tuning and re‐examination of messages, but richer synchronous technologies (such as videoconferencing) are needed to resolve differing viewpoints among team members and to develop a consensus for decision making” (Montoya‐Weiss et al, 2001). The definition of social technology is characterized by multiplicity and the concept is not defined unambiguously. The term ‘social technology’ is defined as a set of potentially arbitrary effective social challenges refillable solution, ways to achieve the intended results, doing social impact of human, social groups, different social structures' behaviour (Alberghini, et al., 2010; (Bugin et al, 2011; Chui et al , 2012). The empirical research for supporting theoretical insights and developing concept of social technologies were planned.

240

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė

Source: developed by authors Figure 1: Social collaboration tools and technologies

3. Empirical interdisciplinary approach to analysis of the concept of ‘social technology’ A Focus group method and content analysis were chosen for an empirical research. The research group established the workshop “Defining Social Technologies” during the International conference on Social Technologies at Mykolas Romeris University, Lithuania, in October, 2012. During the workshop the selection procedure for Focus group representatives was made. The results received in the Focus group were supported with the results of virtual brainstorming using social technologies developed by MRU Faculty of Social Informatics. In the context of interdisciplinary understanding of Social technologies concept mix of Field (natural) and Nominal (also known as Delphi technique) types of Focus groups were used (expert choosing is reasoned in table 1 below). In total, 13 experts from 9 research fields and 10 countries participated in discussions via three sessions. All sessions were organized in the period of 8 month during the year of 2012. Focus group method was chosen for its following features according Kimel (2003): “(a) it provides rich depth of understanding of the phenomenon of interest; (b) it is valid used in isolation from other research methods (might be used to support quantitative research as well); (c) it is as useful and as strong as its link to the underlying research question and the rigor with which it is applied; (d) it provides concentrated amounts of rich data, in participants’ own words, on precisely the topic of interest; (e) it provides critical information in development of hypotheses or interpretation of quantitative data”. As Social technologies is a phenomenon missing clear definition the Focus group method was chosen as appropriate according to the goals and the object of the research. According to Grudens‐Schuck et al. (2004) composing Focus group with highly different characteristics (such as status, income, education, personal features, etc.) will decrease the quality of the data. Interdisciplinary research requires different characteristics. In order the data not to decrease in quality Focus group members were unfamiliar with each other and kept away from direct contact in sessions 1 and 3. Session 2 was conducted in natural environment with careful monitoring of the process. Anonymity among experts does not lower validity of collected data and is appropriate characteristic for this type of empirical research (Tidikis, 2003). Focus group method has some limitations (challenges) as according to Kimel (2003) they are as follows: (a) small number of participants; (b) limited generalizability, (c) group dynamics can be a challenge, (d) time consuming interpretation, (e) requires experienced analysts. Challenges of the method were overcome with experience of and thorough analysis executed by researchers.

241

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė Table 1: Analysis of experts’ relevant characteristics

Expert 1 Expert 2 Expert 3 Expert 4 Expert 5 Expert 6 Expert 7 Expert 8 Expert 9 Expert 10 Expert 11 Expert 12 Expert 13

Main research field Communication Economics Economics Education IT 2 IT IT Law Management Management Mathematics Politics Sociology

Research work in ST 1 field experience (in years) 6 3 5 2 7 6 6 2 3 6 5 2 2

Research work experience (total in years) 17 14 5 9 21 11 8 5 13 8 19 17 8

Country of affiliated research institution The Netherlands Denmark Spain Norway Portugal Finland Germany Lithuania Poland United Kingdom Lithuania Finland Lithuania

Outcomes of the content analysis revealed that essence of Social technology is seen in 7 different sub‐fields (grouped in to major fields) and is close in characterization to the content of concept descriptions found in scientific literature (see table 2 below, keywords are in bald shrift). Table 2: Essence of social technologies Field

Sub‐Field Public policy creation means

Social engineering

Group behaviour pattern creation methods/means Individual behaviour shaping means

Collaboration tools

Information aggregation tools

Social software

Knowledge aggregation tools Social networking tools

Quotes “<...> ways in decision making process for community needs <...>’’ “<...> social structures are defined’’ “<...> with determination of social order <...> “,<...> society aims are in the heart of the process of effective public governance<...> those aims are set with intention of social shaping’’ “<...> only when societal effect is made’’ “<...> determines societal structures <...>’’ “<...> a certain class/group/(or whatever we call it) is allowed<...> and in this way in a long‐term behaviour code is formed.’’ “<...> people act on codes (civil, moral, and other)<...> the manner of design of those codes’’ “<...> citizen is obliged to obey the official rules of society’’ “When environment forces on to act in a particular direction <...>’’ “<...> social structure sets you into a pattern <...>’’ “<...> after accepting the new societal structure (Soviet Union example) critical mass force new structure on individual members of society <...>’’ “<...> wiki type online platforms enable users <...> to work with each other with no restrictions of time or space. Socially friendly cooperation’’ “One of them is Google Docs work space <...>’’ “<...> in work places which are not geographically concentrated <...>’’ “<...> as business information systems or IT in electronic public services <...>’’ “<...> distant ICT based learning systems<...>’’ “<...> wiki type online platforms enable users to compile their knowledge on one topic <...>’’ “Web 3.0 basically covers the topic.’’ “It is all about social networking sites as Twitter, Facebook, Pinterest and others, which basically means IT for social interactions.’’ “<...> one of the elements would be social sites.’’ “<...> social sites currently reveal the core of social technology <...> networks as societal structure and ICT as main interaction tool’’ “<...> in contemporary practice it is modern social media’’ “IT, mobile or other communication technologies <...> (e.g. various apps) of those technologies are eligible for social effect if properly used’’

Social Technologies Information technologies

242

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė Another outcome relevant to scientific literature analysis became evident from the main segregation of historically developed definition of social technologies as social engineering (first) and social software (later). Focus group data analyses showed that both conceptions of social technologies are still vivid though gradually compiling to one. As social technology with the aspect of social engineering is to be understood as (a) public policy creation means; (b) group behaviour pattern creation methods/means; (c) individual behaviour shaping means; in the social software field of social technology the subfield of social networking tools those two seemingly different aspects are starting to combine. Other sub‐fields (collaboration tools, information aggregation tools, knowledge aggregation tools) in social software field are more of a separate sphere more oriented not to changing current structures or forming new ones, but more related to the collective of the community/society. Social technologies is a possible solutions to a problem when any event a combination of tools and techniques (technology) support is replaced by the more socially desirable. It is practically any activity which is replaced by the object. As a counterbalance to this general concept of social technologies, in research lays another concept, simplifying previously defined from the theoretical approach on the study of innovative technical device. At its most narrow sense social technologies can be understood as information and communication tools that have a range of economic, social, cultural or other public life processes available to each person: computers, smart phones, social networks, etc. Thus, the analysed object is called an instrument through which members of the public relevant information just become available. Both general and as most narrow concept of social technologies gives out the same keywords ‐ social technology is what is innovative, efficient, and changes us past the usual social processes. Additional notices, relevant to a deeper understanding of the concept of social technologies, were abstracted from the ideas presented in Focus group discussion (see table 3 below, keywords are in bald shrift). Table 3: Evaluation of Social technologies’ effects Effect direction

Positive

Sphere Innovation encouragement Collective intelligence Time management

User generated content quality Safety Negative Satisfaction in interpersonal communication

Quotes “<...> they might bring society to the next level’’ “<...> new types of social interaction tools are being created, new trends and lifestyle’’ “<...> synergy in wisdom of crowds <...>’’ “<...> crowd sourcing economic effect<...>’’ “<...> exact time losses meaning <...>’’ “<...> activities may be bonded with interconnection sequence with final deadlines and not with office hours <...>’’ “<...> occupational activities stop being determined by specific timeframe <...>’’ “<...> work possible at anytime<...>’’ “<...> everyone can post everything on open source content platforms <...>’’ “<...> mass copy pasting <...>’’ “<...> high level of personal date exposure <...>’’ “<...> hacking <...>’’ “<...> online communication overtakes eye‐to‐eye contact <...>’’ “<...> ability to choose way of communication is decreasing <...>’’ “<...> children social communication skills are not being developed <...>’’

Experts in line with the discussion of the content of social technology concept suggested possible negative or positive effects of this phenomenon. Positive effects were mostly related to social technology as a collaboration or information/knowledge aggregation tool, as in this case it was taken as tool helping to assure efficiency of the processes in collaboration, while negatives rose from social technology as social networking tool with the agenda of personal data privacy or face‐to‐face communication skills. No strong opinions were given on positive or negative effects of social technology as social engineering.

4. Conclusions and discussion: The potential and risks of social technologies Summarizing theoretical insights, results from the Focus group research and content analysis the following conclusion and questions for further research on the potential and risks of social technologies can be formulated:

243

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė It can be concluded that the current function of social technology is for social purposes via digital means. The real power of social technologies is only started to be understood. Its power steams from the innate appeal of interacting socially, pleasure and intellectual stimulation that people drive from sharing what they know, expressing opinions, and learning what other know and think (Bugin et al, 2011). Despite rapid application of social technologies, much more lies ahead. Today, more than 80 percent of the world`s online population is interacting via social networks on a regular basis, but 65 percent of the world population‐4,6 billion people‐still lacks internet access (McKinsey research, 2012). As has been seen in early use of social technologies, when these ways of interacting are applied to commercial and professional activities (e.g. developing and selling products, working together to solve a business problem), the resulting value creation is impressive (Chui at al, 2012). Social technologies have the potential to affect positive change in communities and governments. Social technologies can be disruptive to existing power structures (corporate and governmental) as happened, for instance, during the Arab Spring 2011. Social technologies allow people to connect at a different scale and create a unified, powerful voice – as consumer groups or entire societies – that can have significant impact on the ways in which dialogues are shaped and policy is made (Bugin et al, 2011). According to Norvaišas et al (2011) ,,<...> a multitude of business, administration, communication and other processes are digitalized thus placing them in a huge network, organizations need a system which would enable analysing people’s opinion and finding the best solution regarding the development of new products and services”. Social interaction via technologies is a powerful way to efficiently organizing knowledge. The same effect is valid to culture, economics, and political power. Social technologies are becoming the preferred method of communication of new generations and communication styles are evolving into a more collaborative approach (Alberghini, et al., 2010). According with Forrester analysts as De Gennaro and Fenwick (2010), there are key trends that will make the inclusion of social technology in society life a necessity. These trends are the physical distance between teams and the entrance of Millennials into the workforce (De Gennaro, 2010). These new employees bring very different needs, experiences, and expectations to the job and often meet a seasoned workforce that has very different work styles (Schooley, 2009). New technologies allow people to raise questions, share knowledge and ideas, and discover people skills regardless of hierarchy. According to Koplowitz and Owens (2010) such tools help “to break down organizational and cultural barriers such as time differences’’. The application of social technologies in organisational management has become crucial for success in network society. Nevertheless, over the next few years the emerging “social technologies” of Web 2.0 and Web 0.3 are likely to transform the management. Social technologies enable organisational interactions to take place online with the scale, speed, and economics of the Internet. Virtual networked teams have made management more efficient, because they are reducing the costs of communication, collaboration and coordination. McKinsey’s fifth annual survey on social tools and technologies shows that when integrated into the daily work of employees and adopted on a large scale throughout a new kind of business—the networked enterprise— they can improve operations, financial performance, and market share (Chui at al, 2012). Social networking capabilities are providing vital information in a way that is adaptive and user‐driven. However, all these technologies have limitations that can easily lead to misinterpretation, as with the lack of non verbal communication, they are not capable of providing the same quality of communication as eye‐to‐ eye interaction. “Because of delays in transmission and the lack of social and nonverbal cues, communication technologies can interfere with open communication, knowledge sharing, and the ability of teams to identify and resolve misunderstandings” (Cohen, Gibson, 2012). Old generations tend to be sceptical about social technologies. Therefore, it is important to implement something useful, to monitor the user engagement and to educate the community for using social technologies (Allberghini et al, 2010). Online collaboration, in its current state, is not a very good substitute for the sort of unscripted, face‐to‐face interactions that are critical to producing genuine breakthroughs. And complex coordination tasks, like those involved in the design of a new aircraft, still require a dense matrix of “strong ties” among critical contributors, rather than the “weak ties” that are typical of web‐based communities (McKinsy Global Survey, 2012). The use of social technologies can also carry risks. Chui et al (2012) and other also researchers from McKinsy Global Institute, working in the field of social technologies identified several risk group: “employee time spent ‘’chatting’’ about not work‐related topics on internal or external social networks or using social media to attack

244

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė fellow employees or management”, different risks related to consumer privacy, information security and data security. These risks could limit the ways in which social technologies can be applied. Also “censorship and restrictions on Internet use stand in the way of value creation by companies that hope to enable consumer to interact with them and that wish to harvest deep insights from social data” (Chui et al, 2012). Social technologies enable more and more users to become a part of global conversation, creating their own content rather than just consuming it. However, the quality of user‐generated content varies dramatically – from excellent works of journalism to spam and even abuse. Bauerlein (2008) critics have argued that the very disintermediating power of social technologies has reduced the overall quality of discourse. Carr (2010) have argued that the short‐form content made me available through social technologies is making people less able to digest large and complex amounts of information. The opposing view is that even our existing means of content selection didn’t ever assure quality, that the diversity of opinions is healthy, and that if people can learn social media literacy, access to a broader set of opinions can actually promote critical thinking. Discussion about social technologies potential ought to be a possibility to address the following question for future research, through fundamental conceptual reflections and empirically‐oriented contributions: which social technologies are most important in current social environment? How can we study them? What is the future of social technologies and network society?

References Alberghini E., Cricelli L., Grimaldi M. (2010). Implementing knowledge management through IT opportunities: definition of a theoretical model based on tools and processes classification, The Proceedings of the 2nd European Conference on Intellectual Capital, Lisbon, Portugal, 29‐30 March, 2010, pp. 22‐33. Andersen, K. N. (2011). Social Technologies and Health Care: Public Sector Receding, Patients at the Steering Wheel?, Conference proceedings ,,Social Technologies '11: ICT for Social transformations‘‘, 17‐18 November, 2011, Vilnius‐ Net. Bauerlein, M. (2008). The Dumbest Generation, New York: Penguin Group. Bughin, J., Byers, A. H., Chui, M. (2011). How business uses social technologies, McKinsy Quarterly, November 2011, access online 16 Sep. 2012, url: http://www.mckinseyquarterly.com/newsletters/chartfocus/2012_05.html Carr, N. (2010). The shallows: What the Internet is doing to our brains, New York: W.W.Norton. Chui, M., Manyika, J., Bughin, J., Dobbs, R., Roxburgh, Ch., Sarrazin, H., Sands, G., Westergren, M. (2012). The Social Economy: Unlocking Value and Profuctivity Through Social Technologies, McKinsey Global Institute Report. Cohen, S.G., Gibson, Ch.B. (2012) Virtual teams effective, General Management review, reached at: http://www.etgmr.com/GMRjan‐mar04/art1.html De Gennaro T. (2010). Social Technologies Will Penetrate IT Management Tools, Forrester Research. Derksen, M., Vikkelsø, S., Beaulieu, A. (2012). Social technologies: Cross‐disciplinary reflections on technologies in and from the social sciences, Theory Psychology, vol. 22, no. 2, p. 139‐147. Duarte, A. T. (2011). Privacy and Health System Solution Case, Conference proceedings ,,Social Technologies '11: ICT for Social transformations‘‘, 17‐18 November, 2011, Vilnius‐Net. Forrester Research. (2009). Global IT Market Outlook: 2009. Cambridge: Forrester Research, Inc. Grudens‐Schuck, N., Lundy Allen, B., Larson, K. (2004). Focus Group Fundamentals. Iowa State University Methodology Brief. Access online 7 Sep. 2012, url: http://wwwstatic.kern.org/gems/region4/FocusGroupFundamentalsRobert.pdf Henderson, C. R. (1895). Review. Journal of Political Economy, 3(2), p. 236‐238. Johannessen, J. A., Olsen, B., Olaisen, J. (2001). Mismanagement of tacit knowledge: the importance of tacit knowledge, the danger of information technology, and what to do about it, International Journal of Information Management, Vol. 21, Iss. 1, p. 3‐20. Kimel, Bar‐Din M. (2003). Focus group methodology. The FDA Drug Safety & Risk Management Advisory Committee Meeting, Gaithersburg, Maryland, December 4, 2003. Koo, Ch., Wati, Y., Jung, J.J. (2011). Examination of how social aspects moderate the relationship between task characteristics and usage of social communication technologies (SCTs) in organizations, International Journal of Information Management, Vol 31, Iss.5, p. 445‐459. Koplowitz R., Owens L. (2010). Disciplined Social Innovation, Forrester Research. Leibetseder, B. (2011). A Critical Review on the Concept of Social Technology. Social Technologies, 1(1), p 7–24. Leichteris, R. (2011). Mokslo ir technologijų parkai socialinių technologijų kontekste. Social Technologies, 1(1), p 139–150. Li, Ch. and Bernoff, J. (2012). Groundswell, Expanded and Revised Edition: Winning in a World Transformed by Social Technologies. Harvard Business School Press Books, 352 p. Mayer, K. (2009). Who produces social technologies? access online 16 Sep. 2012, url: http://socialtechnology.wordpress.com/2009/10/23/who‐produces‐social‐technologies/ McKinsy Global Survey. (2012). McKinsey Global Survey results, access online 16 Sep. 2012, url: http://www.mckinseyquarterly.com/Business_Technology/BT_Strategy/Minding_your_ digital_business_McKinsey_Global_Survey_results_2975

245

Aelita Skaržauskienė, Rūta Tamošiūnaitė and Inga Žalėnienė Montoya‐Weiss, M., Massey, A.P., Song, M. (2001). Getting It Together: Temporal Coordination and Conflict Management in Global Virtual Teams, Academy of Management Journal, 44(6), 1251–1262. Nelson, R. R. (2002). Bringing institutions into evolutionary growth theory. Journal of Evolutionary Economics,Vol. 12, p. 17–28. Nelson, R. R. and Sampat, B. N. (2001). Making Sense of Institutions as a Factor Shaping Economic Performance, Journal of Economic Behavior and Organization, iss. 44, p. 31–54. North, D. and Wallis, J. (1994). Integrating institutional change and technological change in economic history: a transaction cost approach, Journal of Institutional and Theoretical Economics, Vol. 150, p. 609–624. Norvaišas, S., Mažeika, A., Paražinskaitė, G., Skaržauskienė, A., Šiugždaitė, R., Tamošiūnaitė, R. (2011). Įtinklintos vadybos studijos. Vilnius: Mykolo Romerio universiteto Leidybos centras. Pelikan, P. (2003). Bringing institutions into evolutionary economics: another view with links to changes in physical and social technologies, Journal of Evolutionary Economy, Vol. 13, p. 237–258. Schooley C. (2009). The Millenials are here! Are you prepared?, Forrester Research. Schotter, A. (1981). The economic theory of social institutions. Cambridge: Cambridge University Press. Sugden, R. (1989). Spontaneous order. Journal of Economic Perspectives, Vol. 3: p. 85–97. Tidikis, R. (2003). Socialinių tyrimų mokslo metodologija, Vilnius: Mykolo Romerio universitetas, p. 474−518. Wikipedia (Wikipedia, the free Encyclopedia). (2012). Social technology, access online 16 Sep. 2012, url: en.wikipedia.org/wiki/Social_technology#cite_note‐1

246

Community ICT Hubs Sustainability: Result From a Field Study of ICT4D Project Haslinda Sutan Ahmad Nawi1, 2, Nur Syufiza Ahmad Shukor1, 2, Suzana Basaruddin1, Siti Fatimah Omar1 and Azizah Abdul Rahman2 1 Information Systems Department, Faculty of Computer Science and Information Technology, Universiti Selangor, Bestari Jaya, Malaysia 2 Information Systems Department, Faculty of Computing, Universiti Teknologi Malaysia, Johor Bahru, Malaysia haslindasan@unisel.edu.my nur_syufiza@unisel.edu.my suzana_b@unisel.edu.my ctpatimah@unisel.edu.my azizahar@utm.my Abstract: Technically, a project ends when the project closure and product handover are undertaken. However, the results of the project, specifically the product that the project has borne, will take on a new life in the user environment. The processes undertaken to ensure the usability and effectiveness of the product in that context will determine the product’s future life or its sustainability. The purpose of this paper is to identify and understand the sustainability criteria of community ICT hubs implemented at 9 districts in one of the most developed state in Malaysia. This study uses case study as a strategy to collect its qualitative data using document review, observation and interview of 92 respondents. There are 8 sustainability criteria discovered, grouped within 3 sustainability dimensions: social / cultural; economical; and institutional. Keywords: community ICT hub, ICT sustainability, ICT4D, sustainability dimension, sustainability criteria

1. Introduction As information and communications technology (ICT) is affordable, growth rapidly, and generate an environment where everyone has to rush to catch and remain updated in modernity, most government around the world proposed rural environment ICTs, to make sure the benefits of new technology are obtainable and accessible to all. However, ICT initiatives in developing countries have shown a low success ratio in terms of sustainability (Heeks, 2002). (Musa, 2006) cites a number of authors who indicated that there is huge evidence of failure and wastage of resources linked to sudden massive implementations of ICT projects in developing countries, with the hope of promoting development and alleviating poverty. In Malaysia, a state's public ICT hub has been introduced to support community development and bridging the digital divide through the use of communications and computing technology by one of its most progressive states. However, after 10 years in operation, only 19% were retained, while the other 81% have failed to sustain, either because it were underutilized, abandoned or closed. This study derived from (Basaruddin et al., 2010) where the aim is to identify and describe the sustainability criteria of public ICT hub in developing countries with reference to Malaysia.

1.1 ICT for development Now days, ICT plays an important role in development, especially in ensuring data and communication is precise and recent. It helps to achieve development goals and spur progress in developing countries. There are also signs that the pervasive use of ICTs are globally leading to the transformation of some societies to a new kind of society: the information or knowledge society (Unwin, 2009). ICT for Development (ICT4D) initiatives have been launched concurrent with the growth in prominence of ICT4D on worldwide policy agendas during the late 1990s (Leye, 2009). (Heeks, 2008) described the evolution of ICT4D in three phases: ICT4D 0.0, 1.0 and 2.0. In the first phase, until about 1990, computers were used in government administration and by multinationals to foster economic growth. From the mid‐1990s onwards, ICT4D 1.0 started as development actors such as the World Bank called for the adoption of ICTs as a tool for development – a call which was in response to the growth of the Internet and the adoption of the Millennium Development Goals (MDGs). Due to the need for a rapid response to the plight of poor, rural communities, a popular choice was the deployment of telecentres or community ICT hubs to deliver information, communication and various services.

247

Haslinda Sutan Ahmad Nawi et al. However, the sustainability of ICT4D initiatives remains an enduring concern, and projects typically have a high rate of failure. A narrow focus on the project itself does not suffice to surface the reasons for failure, or to identify the route towards sustainability. The reasons for failure lie inside the scope of the project, within the community itself, and outside the community in the larger socio‐economic system which includes the economy.

1.2 The concept of sustainability The broad definition of sustainability has been reinterpreted in the domain of information systems to address challenges in the design and implementation of sustainable IT solutions (Reynolds and Stinson, 1993, Korpela et al., 1998, Misund and Hoiberg, 2003, Oyomno, 1996). According to (Misund and Hoiberg, 2003), sustainable IT is a technology that is capable of being maintained over a long span of time independent of shifts in both hardware and software. While (Scheirer, 1993) defined a sustained programme or project as a set of durable activities and resources aimed at program‐related objectives. (Unwin, 2009) (p. 365) states that sustainability is primarily a problem with “externally situated ICT4D programmes, and in part reflect a desire by those who create them to guarantee their continued success after the initial period of investment is over”. His opinion is that too little attention is paid on how initiatives can become self‐supporting, and recommends that all ICT4D programmes that are introduced by external players have a framework for ensuring “continued viability beyond the initial period of funding” in order to not saddle the beneficiaries with the burden (ibid). This research works around the definition of sustainability by (Unwin, 2009, Misund and Hoiberg, 2003, Scheirer, 1993).

1.3 Sustainability dimensions Sustainability is not only about being better environmental stewards, but it should also include giving a comprehensive response to both the internal and external impacts of social, cultural, and resource trends (Werbach, 2009). As managers internalize a strategic approaches and responses that encapsulates these four sustainable factors, only then will that organization, whether public or private, be able to adequately sustain its existence in the future. In other discussion, most research considers sustainability to be closely linked to the ability of a project to be financially sustainable, in that a project must be capable of cost recovery, in order to be continuously operative and dynamic in the services they provide (Pade et al., 2006). However, sustainability encompasses more than just the financial or economic aspect of the project, but it is also considers other significant facets such as rootedness in local communities, cultural and political acceptance, and value to rural individuals (Keniston, 2005). ICT projects need to take this into account, as (McNamara, 2003) proposed that most projects proven to be unsustainable in the long‐run because they have not been accompanied by, or failed to generate the broader economic and social changes that would lead to sustainable demand for ICT resources in rural development, especially relative to other demands on scarce resources. While (Ali and Bailur, 2007) have identified five main dimensions of sustainability in the ICT4D literature: financial; social; institutional, technological and environmental (Ali and Bailur, 2007). There is a need to integrate social, cultural, institutional, economic, political, and technological sustainability as vital elements in the planning and operation of ICT projects (Stoll, 2003). Many studies refer to sustainability as being a key to long term development outcomes for ICT projects. (Batchelor and Norrish, 2003) distinguished between economic sustainability (achieved when a given level of expenditure can be maintained over time), social sustainability (achieved when social exclusion is minimized and social equity maximized and institutional sustainability (achieved when prevailing structures and processes have the capacity to perform their functions over the long term). Evidently any long term development goals for ICT project could be achieved with 4 main capital assets: human capital; financial capital (cover cost and maintenance); physical capital (technology is one of the keys as will be the infrastructure which enable the technologies work); and social capital (social and institutional arrangements that will keep the ICT being used for its intended benefits). However, this study adopts the three main dimensions of sustainability proposed by (Batchelor and Norrish, 2003) in 2003 as these dimensions are able to differentiate between different kinds of sustainability dimensions which are useful in relation to ICTs. Using these dimensions will help this study to achieve its goal; which is to understand the sustainability criteria in depth. The different dimensions include: Social and Cultural Sustainability: This dimension considers the social and cultural context in which a project operates, and the response of the ICT project to this context. As the ICT project takes into account the social

248

Haslinda Sutan Ahmad Nawi et al. and cultural aspects of the community, people in the community feel empowered by the project and hence become active in seeking ways in which to keep the project running, as it is in their own vital self‐interest (Stoll, 2003). According to (Batchelor and Norrish, 2003) social exclusion is therefore minimized, and social equity is continuously built on and not undermined. Economic Sustainability: This dimension could be associated with the level of expenditure that can be sustained in the long term (Batchelor and Norrish, 2003). ICT projects in rural areas are initially funded by development organizations; however, in the long term the ICT services provided will need to develop cost recovery mechanisms to generate enough income to keep the project sustainable. In 2003 (The World Bank, 2003) indicated that the ability for ICT services to be financially self‐sustainable is a key concern; hence there is a need to promote a spirit of entrepreneurship to market ICT services rendered and secure grant contributions. Institutional Sustainability: Institutional sustainability is achieved when prevailing structures and processes, where they have the capacity to perform their functions over the long term (Batchelor and Norrish, 2003). Aspects of institutional sustainability that need to be put in place include well‐defined ICT laws, participatory policy‐making processes, and effective public and private sector organizations that develop a framework in which the livelihoods of the community can be continuously improved.

2. Case Study: State public ICT hubs The focus of current research is in one of the state in Malaysia that has nine districts with the largest population. The state public ICT hubs were established in 1999 to serve the needs of ICT usage for the local community with the vision to increase ICT literacy among the community. The initiative is intended to provide equal opportunity for the community, especially those in the rural areas, to get into the information technology wave. Being the nation’s only developed state, community development play a pivotal role in the state’s economic growth. Therefore the project is also intended as an approach in meeting the state's objective of making 40% of the population IT savvy. There are 39 hubs at nine districts under the responsibilities of State Public Library and State Federal Office. Each hub is equipped with minimum of six computers for users, one computer for administration, one scanner, one printer and a dial‐up internet connection for all computers. For the past ten years of its operation, the facilities have benefited the users, especially those in rural and sub‐urban areas of the state. Nevertheless, these hubs have many management and operational issues, which have lead to discontinued services of a few hub facilities. The State Federal Office, through their ICT Department has noticed the hub operational issues and is looking forward to minimizing the issues in sustaining the hubs. Table 1 shows the number of hubs by district. Table 1: Public ICT hubs by district and the responsible agencies Location

Number of Hubs

District 1

Responsibility of State Federal Office

State Public Library

District 2

District 3

District 4

District 5

District 6

District 7

District 8

District 9

Total

Source: State’s Economic Planning Unit (UPEN), 2010

249

Haslinda Sutan Ahmad Nawi et al. This research was conducted at 9 districts of Selangor state and involved 26 hubs. Data from semi‐structured interview, observation, and document reviews were collected from the Federal Office’s officers and the users at each hub.

3. Methodology The overall research investigation examined the implementation of the Selangor state ICT hubs. The study aims to draw general lesson where an exploration of the criteria of sustainability formed a part of the overall research investigation. A survey was conducted, where a case study qualitative research methodology has been adopted to assess the ICT hubs project in a real‐life environment (Yin, 2009). Techniques employed in data collection at 26 public ICT hubs were semi‐structured interviews, non‐participant observations, and document reviews. However, the main instrument used was semi‐structured interview questionnaire.

3.1 Semi‐structured Interviews Semi‐structured interviews were done with two different groups of respondents. The first group represents people who are responsible in managing and running the ICT hubs. A total of 28 officers were contacted and interviewed. The respondents were asked on their routine operation activities of providing services to the customers; challenges and problems in administrating the hubs; and their perception on how the hubs could be sustained, experience in operating the hub, the history of the hub – from the first day it was opened to date. The second group interviewed was 64 respondents from the community who have been using the facilities in the ICT hubs. The intention of this interview was to explore the users’ perspectives on the uses and challenges of ICT hubs, and the project’s approach to promote sustainability. This interviewed conducted separately from the first group of the respondent.

3.2 Non‐participant observation Apart from the semi‐structured interview, researchers were also exposed directly to the operations associated with the hub through observation of the administrators’ and users’ activities at 5 hubs for two weeks. The observation focused on how the operators handle their daily operations of the hub where the users involved particularly on how the service is served to the users at the hub. The observation includes the process of registering the users’ attendance, recording the users’ activities (i.e. printing, browsing the Internet, scanning, etc.) and recording the maintenance of the software and hardware supplied to the hubs.

3.3 Document review To help the researchers to better understand the processes observed, documentation of the hubs operation activities were examined. These included the attendance log, the usage log, and the maintenance log. Data from the documents were reviewed and analyzed, conforming what was previously observed at the hubs. Attendance for the first 2 years of its implementation was high with an average of 30 users /day, however after 2 years, there was a decline in the number of visitors as recorded in the log book. In some cases, the hubs were closed from operation.

3.4 Establishing the research validity and reliability Reliability and validity are conceptualised as trustworthiness, rigour and quality in the qualitative paradigm (Golafshani, 2003) and without these criteria research is worthless, becomes fictional, and loses its utility (Morse et al., 2002). Table 2 describe the strategies taken to ensure validity and reliability of the study. Validity: The methods of data collection discussed above had enabled the researchers to have rich, in‐depth information generated through multiple sources of evidence. The use of multiple sources of evidence, within a single study is known as data triangulation (Yin, 2009). In addition, data collected from different sources provided more credibility to the data (Lillis, 1999). Furthermore, several participants were asked to read through the findings to ensure that they depict a ‘faithful’ experience (Miles and Huberman, 1994). Since all four participants make no amendment to the finding, other participants were assuming the same. Other than that, during the time the study is in progress, the researchers presented the findings to the panel of Centre of Research and Innovations. The discussion and feedback from these platforms support for the trustworthiness,

250

Haslinda Sutan Ahmad Nawi et al. rigorousness and quality of the findings. (Lincoln and Guba, 1985) define the role of peer reviewers as a ‘devil’s advocate’ – an individual who keeps the researcher honest by asking the hard questions about method, meanings and interpretations. While a purposive sampling of two different groups of respondents mentioned above support the transferability of the study. Then the validity of the data gathered was maintained by transcribing the interview verbatim. Table 2: Strategies taken to ensure research validity and reliability Quantitative Term

Qualitative Term

Description

Internal Validity

Credibility

External Validity

Transferability

Reliability

Dependability

Truth value: How can one establish confidence in the ‘truth’ of the findings of a particular inquiry – for the subjects (participants) and the context with which the inquiry was carried out? Applicability: How can one determine the degree to which the findings of a particular inquiry may have applicability in other contexts or with other subjects (participants)? Consistency: How can one determine whether the findings of an inquiry would be consistently repeated if the inquiry were replicated with the same (or similar) subjects (participants) in the same (similar) context?

Strategies Taken in the Study Triangulation Member checks Peer debriefing

Purposive sampling Collect and develop thick descriptions

Overlap methods Audit trail

Source: (Guba, 1981) Reliability: Overlap methods used in this study are one kind of triangulation process, whereby different methods of inquiry used in tandem support dependability (Guba, 1981). The researcher documented the research process on transcript and by memo report. This provides an audit trail for assessing the reliability of the research and analytical process.

4. Analysis result and discussion The results from the data gathering showed that there are eight significant criteria that were directly influence the sustainability of the ICT hub. These criteria then were categorized into three dimensions mentioned in the earlier section, accordingly. Table 3 depicts the eight criteria identified and have been categorized within the three sustainability dimensions. Table 3: Sustainability criteria categorized within 3 sustainability dimensions

Sustainability Criteria

Frequency mentioned in the Interview (%)

Social & Cultural Dimension

Economic Dimension

Institutional Dimension

Community development

12.4

√

Ethics

5.6

√

Social Network

4.7

√

Financial

18.7

√

People

18.7

√

Infrastructure

18.7

√

Policy and strategy

18.7

√

Political Influences

2.7

√

251

Haslinda Sutan Ahmad Nawi et al.

4.1 Social and cultural dimension As discussed earlier, in order for the ICT hub to sustain, the social and cultural dimension must be taken into account. This dimension handles the social and cultural aspects of the community by making the people in the community feel empowered by the project. Thus, it creates self‐belonging that drives them to become active in seeking ways in which to keep the project running, as it is in their own vital self‐interest. There are three criteria that fit into this dimension: community development; ethics; and social network. Community Development: Community development is where the community learn and develop from what they have learned. In this case, the ICT hub helps the community to learn via the learning environment created from the services provided by the hubs. The Internet connectivity provided is used to create the learning environment and enable linkages between local community and outside world. It also helps to encourage local ownership, establish the community of practice (CoP), facilitate local content development and create technically literate user. Apart from that, the government to citizen (G2C) linkages is also established through the e‐government application. The G2C is especially significant for the citizen to communicate directly with the government without discrimination and bureaucracy. Ethics: When users that range from all ages are using the facilities provided, a proper guideline or code of conduct must be put in place. This will ensure the ICT hubs’ sustainability as it addresses the issues that alarming the society and parents at large by ensuring a systematic guide or code of conduct is adhered to when users are using the facilities. It is important for the users to be able to differentiate between what is right and what is wrong.Therefore ICT hub must be administered with some guidelines or code of conduct to ensure its sustainability. Social Network: For decades, people will look for innovative way to communicate with one another. The Internet has helped people with geographical distance to communicate easily. The connectivity is now easier than before via the Internet through applications like emails, social network and blogs. ICT hubs with access to these applications will ensure their sustainability. It is now a mandatory for the hubs to have reliable Internet facility for social networking.

4.2 Economic dimension The second sustainability dimension that will be discussed is the economic dimension. Three criteria are categorised into this dimension namely strong financial planning, alongside with spirit of entrepreneurship to market ICT services; people; and infrastructure are factors that are the factors to be taken into account. Financial: The hubs must be equipped with a thorough financial plan that could fund the hubs for a long enough period. A solid financial model or strategy that empowers the hubs operators to manage their own financial affairs (including generating and distributing income) is also seen as important by the respondents. This is the investment that produces champions – a working life‐long ICT hub. People: This criteria address on staff competency and qualifications, staff incentive, and dedicated and permanent staff appointment. The hubs are desperately in need of qualified permanent staff that could provide and deliver better service needed by the community for example in conducting the computer classes or to perform simple maintenance on the computers when necessary. Infrastructure: To provide a conducive ICT centre that adds values to its users, it requires updated technology in use, reliable Internet access and ICT services, and continuous and scheduled maintenance of ICT devices. In other word, a design of context‐ appropriate technology is critical to ensure its sustainability. Based from the observation, there are numbers of hubs still in operation but do not have many users. The hubs are in a good condition but the hardware and software are obsolete. There are abandon hubs with hardware broken and not fixed. Evidently accessible, reliable, and good infrastructure is very important and this will ensure users’ satisfaction and continuous ICT centre services.

4.3 Institutional dimension Institutional ensures well‐defined ICT laws and policy are in place by authoritative party(s) and it also promotes for the public and private organizations to work hand‐in‐hand in developing, owning and maintaining the hubs through their active participations. Based on the transcribed interviews the following two criteria were identified for institutional sustainability: policy and strategy; and political influences. Policy and strategy: In

252

Haslinda Sutan Ahmad Nawi et al. previous study (Basaruddin et al., 2010) mentioned about policy and strategy as an influencing factors in sustaining community ICT hubs. Policy and strategy is derived from the attributes of policy maker and project champion. A policy maker is the responsible party that produce standard operating procedures (SOP).This leads to a proper organizational structure that helps to identify the roles and responsibilities of each parties involved that allows continuous monitoring, control and evaluation to be easily executed. Thus it is mostly important for any ICT4D projects to have project ownership so that the project champion would have full authority to delegate work and provide continuous support and commitment. The project champion is the responsible party(s) that focuses on strategizing the existing public facilities for each ICT services centre at strategic location; making sure systematic documentation on facilities and operations, and hand them over when there is change in management; implementing affordable membership fees; and providing value‐added services to users with minimum charge, for example computer classes, scanning and printing services. Political influences: A well defined project ownership regardless of political changes in management is relevant in any projects. The project champion that is political influences‐free will enable a clear workflow to be exercised whenever there is a need of handover. A clear defined roles and responsibilities of all parties involved in the handover will then make a smooth transition process that will be almost unnoticed by the users. This is highly applicable especially when there is a transition of the state government reign.

Figure 1: Frequency criteria mentioned in the interviews As depicted in Figure 1, the presented case allows us to draw some important lessons and to provide recommendations for policymakers in other similar ICT4D future projects. There are 5 major criteria to consider. Before the project could even start, the project champion(s) must be equipped with a comprehensive policy and strategy to ensure smooth administration. Continuous financial support to run the ICT hubs with adequate resources and good infrastructure for the community development has been identified to be crucial. ICT hubs also help the community to reach out to the Government, or to their social acquaintances and friends through social network. In order for the whole model to work, the people factor is utterly important. Proper guidelines for the use of the facilities and their applications are also important to ensure a life‐long usage. A good strategy on making sure the community development activities equipped within its implementation is another criterion that will support its sustainability. Good people resources will ensure all the other mentioned criteria are synergized and exploited to their optimum potential in delivering a sustainable ICT4D projects.

5. Conclusions This study found an array of eight sustainability criteria of public ICT hubs project according to the three sustainability dimensions adopted from (Batchelor and Norrish, 2003). These include: community development, ethics, social network, financial, people, infrastructure, policy and strategy, and political influences. As a conclusion, we believe that the role of technology is limited in sustainable ICT4D interventions, the human dimension is paramount. This research points out that; 5 major criteria (a viable financial model, a qualified permanent people resource, proper infrastructure with a design of context‐appropriate technology, a

253

Haslinda Sutan Ahmad Nawi et al. well defined policy and strategy, and a good community development programme) are the accountability of human dimension that will be the main role of a public ICT hubs sustainability. We believe that the sustainability criteria emerged from this research has created further research in investigating the implementation of community ICT hubs in developing countries. Study on the measurement of the most significant criteria towards community ICT hubs sustainability is relevant for future research.

Acknowledgements The research team would like to express their sincerely thank to Centre for Research & Innovation, Universiti Selangor (Project No.: G‐SUK/ST‐001) for the financial support of the research work.

References Ali, M. & Bailur, S. 2007. The Challenge Of “Sustainability In Ict4d – Is Bricolage The Answer? 9th International Conference On Social Implications Of Computers In Developing Countries. Sao Paulo, Brazil. Basaruddin, S., Nawi, H. S. A., Shukor, N. S. A., Jusoff, K., Jahari, N. A. & Salleh, H. 2010. Influencing Factors For Effective Community Ict Hubs. World Applied Sciences Journal, 11, 114‐117. Batchelor, S. & Norrish, P. 2003. Sustainable Information Communication Technologies (Ict). Available: Http://Www.Sustainableicts.Org/Sustainable.Htm [Accessed September 2010]. Golafshani, N. 2003. Understanding Reliability And Validity In Qualitative Research. The Qualitative Report, 8, 597‐607. Guba, E. G. 1981. Criteria For Assessing The Trustworthiness Of Naturalistic Inquiries. Educational Communication And Technology Journal, 29, 75‐91. Heeks, R. 2002. Information Systems And Developing Countries: Failure, Success, And Local Improvisations. The Information Society, 18, 101‐112. Heeks, R. 2008. Ict4d 2.0: The Next Phase Of Applying Ict For International Development. Computer, 41, 26‐33. Keniston, K. 2005. Notes On Sustainability. Available: Http://Web.Mit.Edu/~Kken/Public/Papers/On_Sustainability.Html [Accessed January 2011]. Korpela, M., Soriyan, H., Olufokumbi, K. & Mursu, A. Year. Blueprint For An African Systems Development Methodology: An Action Research Project In The Health Sector. In: Avgerou, C. (Ed.), Implementation And Evaluation Of Information Systems In Developing Countries, International Federation For Information Processing, 1998 Vienna 173‐286. Leye, V. 2009. Information And Communication Technologies For Development: A Critical Perspective. Global Governance, 15, 29‐35. Lillis, A. M. 1999. A Framework For The Analysis Of Interview Data From Multiple Field Research Sites. Accounting And Finance, 39, 79‐105. Lincoln, Y. S. & Guba, E. G. 1985. Naturalistic Inquiry, Newbury Park, Ca, Sage Publications. Mcnamara, K. S. 2003. Information And Communication Technologies, Poverty And Development: Learning From Experience. Washington Dc: The World Bank. Miles, M. B. & Huberman, A. M. 1994. Qualitative Data Analysis, Thousand Oaks, Sage Publications. Misund, G. & Hoiberg, J. 2003. Sustainable Information Technology For Global Sustainability.Digital Earth. Information Resources For Global Sustainability Symposium [Online]. Available: Http://Www.Ia.Hiof.No/ Gunnarmi/Omd/Dig_Earth_03/ [Accessed February, 2012]. Morse, J. M., Barrett, M., Mayan, M., Olson, K. & Spiers, J. 2002. Verification Strategies For Establishing Reliability And Validity In Qualitative Research. International Journal Of Qualitative Methods, 1, 13‐22. Musa, P. F. 2006. Making A Case For Modifying The Technology Acceptance Model To Account For Limited Accessibility In Developing Countries. Information Technology For Development, 12, 213‐224. Oyomno, G. Z. Year. Sustainability Of Governmental Use Of Microcomputer‐Based Information Technology In Kenya. In: Mayuri Odedra‐Straub (Ed.), Global It & Socio‐Economic Development, 1996 Marietta, Ga. Ivy League. Pade, C., Mallinson, B. & Sewry, D. Year. An Exploration Of The Categories Associated With Ict Project Sustainability In Rural Areas Of Developing Countries: A Case Study Of The Dwesa Project. In: Proceedings Of Saicsit 2006, 2006. 100‐106. Reynolds, J. & Stinson, W. 1993. Sustainability Analysis. Primary Healthcare Management Advancement Programme. Bangkok: Aga Khan Foundation. Scheirer, M. 1993. Are The Levels Of Institutionalization Scales Ready For Prime Time? A Commentary On Development Of Level Of Institutionalization Scales For Health Promotion Programs. Health Education Quarterly, 20, 179‐183. Stoll, K. 2003. Telecentres Sustainability: What Does It Mean? Available: Http://Topics.Developmentgateway.Org/Ict/Sdm/Previewdocument.Do~Activedocumentid==442773 [Accessed December 2011]. The World Bank 2003. Ict For Development Contributing To The Millennium Development Goals: Lessons Learnt From Seventeen Infodev Projects. Washington Dc: The World Bank. Unwin, T. 2009. Ict4d, Information And Communication Technology For Development. Cambridge, Uk: Cambridge University Press. Werbach, A. 2009. Strategy For Sustainability: A Business Manifesto, Harvard Business Press. Yin, R. K. 2009. Case Study Research: Design And Methods, Thousand Oaks, California, Sage.

254

Sources of Disturbances Experienced During a Scrum Sprint Maureen Tanner and Angela Mackinnon University of Cape Town, South Africa mc.tanner@uct.ac.za MCKANG008@myuct.ac.za Abstract: Scrum, as a form of agile project management methodology, comes with many benefits derived from the iterative form of development processes. Various organisations within South Africa have implemented Scrum within their development teams and are fast receiving the positive benefits from it. While Scrum, in many aspects of the methodology, is highly effective and successful within different organisations, there are always going to be pitfalls and negative attributes associated with adapting a new methodology. The purpose of this research is to determine the different sources of disturbances in the middle of a sprint while using Scrum. Case studies were conducted which included 3 companies in Johannesburg and Cape Town that have implemented Scrum: 12 face to face, one‐on‐one interviews with participating Scrum team members; 1 group discussion with 8 participating Scrum team members; and 8 questionnaires After analysis, 6 primary themes emerged as the leading sources of disturbance in the middle of a sprint: the Client, Management, External Influences, Processes, Communication, and Experience. This study opens up various doors of possible research including research on possible ways to prevent the sources from having negative effects on the Scrum team and the project; the effect that these disturbances have on the relevant stakeholders involved; and ways in which these sources could be incorporated into the Scrum processes in order to decrease the negative impact. The results of the study thus provide managers with the opportunity to take a deeper look into the sources of their Scrum problems and provide them with an understanding as to how they may prevent these disturbances from causing long term, negative effects on the project and the team.

Keywords: scrum, sprint, sources of disturbances, agile software development

1. Introduction Scrum, as a form of agile project management methodology, comes with many benefits derived from iterative software development. Many organisations in South Africa have implemented Scrum within their software development teams and are reaping benefits from it. In Scrum, the software development cycle is organised in sprints (or iterations) of three to four weeks, during which a potentially shippable set of completed requirements is meant to be produced (Deemer et al, 2010). Changes to requirements in the middle of a sprint are not welcomed by the Scrum master and the Scrum team as a sprint is not supposed to have any external influences while in progress (Cocco et al, 2011; Rising, & Janoff, 2000). Such forms of disturbances should therefore be avoided at all costs as they leave the team members in a state of restlessness and uncertainty as to what they are actually supposed to accomplish (Wysocki, 2009). However, many sprints fail because of such forms of disturbances induced by a lack of planning, a lack of team cohesion, and a lack of understanding as to what the team members are actually required to accomplish and what they are responsible for (Wysocki, 2009). Therefore, while Scrum is in many aspects highly effective and successful, practitioners can also experience challenges while engaging with it. The purpose of this research is to determine the different sources of disturbances which could occur in the middle of a sprint while using Scrum. This research also explores the processes that are in place within organisations to manage the sprint disturbances. The research questions posed for this study are:

What are sources of disturbances in the middle of a sprint?

What are the processes in place to manage sprint disturbances?

This study followed an interpretive research philosophy along with an exploratory research purpose. Three case studies were conducted in companies located in Johannesburg and Cape Town (South Africa) that have implemented Scrum. In particular, 12 face to face, one‐on‐one interviews with participating Scrum team members; 1 group discussion with 8 participating Scrum team members; and 8 questionnaires were conducted. The qualitative analysis focusing on the sources of disturbances in the middle of a sprint used an inductive coding approach. Six themes emerged as the leading sources of disturbance in the middle of a sprint: the Client, Management, External Influences, Processes, Communication, and Experience.

255

Maureen Tanner and Angela Mackinnon This study is important as it provides Scrum practitioners with useful information on how to identify the sources of disturbances during their sprint and proposes ways of managing these disturbances. This study also opens up various possible research avenues including research on ways to prevent the sources from negatively affecting the Scrum team, the project, and company as a whole; the effect that these disturbances have on the relevant stake holders involved; and ways in which these sources could be incorporated into the Scrum processes in order to decrease the negative impact.

2. Literature review Literature, particularly experience reports written by practitioners have identified various forms of disturbances during a sprint. The disturbances can be categorised into five categories namely: The Client, Management, External Influences, Processes, Communication, and Experience. These are further detailed below. Clients, in this context, refer to the internal or external clients of the Scrum team. The main cause of this problem is the fact that the clients are unsure of what they actually want (Wysocki, 2009). “Projects may have a relatively clear mission, but the specific requirements can be volatile and evolving as customers and development teams alike, explore the unknown” (Highsmith, 2011, p4). One of the greatest complaints by developers is that the constantly changing requirements are difficult to keep up with and are a waste of time (Rising, & Janoff, 2000). When the stories are not properly understood or are vague, the team may find themselves having to change the story in the middle of the sprint due the fact that they are not on the correct path (Moe, Dingsoyr, & Dyba, 2009). Scrum incorporates self‐managed teams which, if not controlled correctly can lead to a lack of proper effective and efficient management (Blankenship, Bussa, & Millet, 2011). A change in management or a lack of proper management in Scrum teams can lead to disruptions in the middle of a sprint due to the fact that the team is not effectively managed and is not working efficiently (Schwaber, 1995). External sources of disturbances, in this context, refer to sources of disturbance outside the Scrum team. External influences need to be managed by the Scrum master to ensure that there are no interferences and interruptions to the Scrum team in the middle of a sprint. Blocking out external influences, however, is not always possible as the interferences often come from higher management (Keil et al, 2003; Stettina, & Heijstek, 2011). Some of the external influences are out of the control of the Scrum team and include environment influences (Wysocki, 2009), psychological, social, and organizational influences (Keil et al, 2003). Processes, in this context, refer to the processes involved within Scrum teams and within the company as a whole. A loss of resources, such as staff, could be a source of disturbance in the middle of a sprint (Schwaber, 1995; Stettina, & Heijstek, 2011). Losing Scrum team members in the middle of a sprint and having to replace them, often negatively impacts the sprint as it is difficult to accommodate the changes (Stettina, & Heijstek, 2011). Time pressure is a source of sprint disturbance because if the sprints and iterations are not planned and timed properly, the Scrum team could run into trouble leading to the modification of, and possible scrapping of various stories in the middle of a sprint (Schwaber, 1995). Communication is a big problem within Scrum teams and management need to ensure that their Scrum teams have all the relevant and necessary tools they need in order to communicate effectively (Verner, Brereton, Kitchenham, Turner, & Niazi, 2012). The main problem is the lack of effective communication mechanisms and tools (Sutherland, Viktorov, Blount, & Puntikov, 2007). Cross‐cultural communication is also a source of disturbance in the middle of a sprint as the problems are compounded by language barriers, cross‐continent time zones, and different work styles that are adapted (Sutherland, Viktorov, Blount, & Puntikov, 2007). The current literature on this topic provides a rather high level view on the sources of disturbances in the middle of a sprint. The purpose of this thesis is to break down these sources of disturbances found in the literature and pin point the root sources in more detail. This thesis aims to provide Scrum masters and product owners involved in Scrum with a deeper look into the sources of disturbances in the middle of a sprint.

256

Maureen Tanner and Angela Mackinnon

3. Methodology This study was interpretive and qualitative in nature. An exploratory research approach was also used to understand the sources of the disturbances experienced by Scrum teams in the case organsiations. Given that no prior hypotheses or propositions were put forward, the study was inductive in nature. Three case studies were conducted at three separate companies involved with Scrum and included a detailed investigation of the Scrum processes and problems within each of these organisations. The case study was conducted in a series of 4 steps as shown in Figure 1. Design refers the structuring of the case study and the development of the interview questions. Design also refers to the organising of interviewees at different companies and the arrangement of the interviews and meetings. Conduct refers to the actual interview and meeting process that took place. Transcribe refers to the transcription of all the interviews into one spread sheet. Analyse refers to the in depth analysis that was conducted on the findings and Report refers to the conclusions that were drawn at the end of the research process.

Figure 1: Case study design flow These case studies included three companies practicing Scrum in Johannesburg and Cape Town. The sample was made up of 20 participants who will, along with the company names, remain anonymous throughout this thesis. Table 5 shows the breakdown of the participants in terms of their company, A, B, or C, their participant number, and their position in the Scrum team. A1 and A2 refer to separate branches within a company. Table 1: Case study respondents

Conducting twenty interviews and one group discussion allowed for a wide range of information, from very different perspectives, to be brought up. Participants included people from each of the different roles within a Scrum group and this too added to the credibility of the research. All the interviews were recorded and

257

Maureen Tanner and Angela Mackinnon correctly transcribed ensuring that anyone who has access to it can fully understand it. Together, these aspects of the analysis ensure that it is credible, transferable, dependable and confirmable (Thomas, 2006). The interviews at company A and C were face to face, semi‐structured interviews and were conducted on a one on one basis. Each of these interviews lasted approximately twenty minutes. The interviews at company B took the form of a face to face, semi‐structured group discussion as well as a questionnaire that was filled out by each of the participants. The Scrum team at company B had limited time available and a group discussion was the most feasible option. The group discussion lasted approximately an hour and the participants were given a few minutes at the end in order to fill out the questionnaires. The data was then analysed using the general inductive approach to derive themes and concepts through interpretations of the raw data (Thomas, 2006).

4. Findings The framework shown in Figure 2 shows the relationship between the primary themes identified during this study. Themes will, from now on, be referred to as sources of disturbances as they are the answer to the primary question and are further discussed below.

Figure 2: Sources of disturbances identified in the study The Scrum team is at the centre of the diagram as it is the main party affected by the disturbances in the middle of a sprint. The source ‘Processes’ was the most common source of disturbance in the middle of a sprint. The Scrum team and the ‘Processes’ are related as the Scrum teams follow certain processes depending on the team and depending on their company. ‘Management’ and ‘Client’ appear above ‘Processes’ as they were found to have the final say on what is done. ‘Management’ and ‘Client’ are linked because management is often the primary source of communication between the Scrum team and the client. ‘Communication’ lies on the same level as ‘Processes’ as the main source of communication disturbances occurred within the Scrum teams. ‘External’ source sits on the same level as ‘Processes’ due to the fact that external disturbances are usually caused by other work that the Scrum team is responsible for. ‘External’ is linked to ‘Client’ as they are both external sources to the Scrum team. ‘Experience’ sits below ‘Processes’ as it refers to the skills of the Scrum team members and has no relation to any of the other sources of disturbances. Black arrows are double sided and represent linkages in both directions. An example if where the use of a black arrow is necessary is between ‘Management’ and ‘Client’ as there are lots of interactions between the two sources which often results in a disturbance to the Scrum team in the middle of a sprint. Blue arrows are one sided and represent disturbances that are one sided. Due to the fact that management have the final say, in most situations, the Scrum team has to deal with this disturbance.

258

Maureen Tanner and Angela Mackinnon

4.1 The client Fourteen respondents suggested that the client is a source of disturbance in the middle of their sprint. Client, in terms of this research, is the internal client that provides the Scrum teams with the development projects. All three companies had internal clients with some minor work being done for clients external to the business. The two sub‐themes identified for the Client theme were: Unclear objectives and Story Changes. Unclear objectives refer to poorly defined objectives for the project at the beginning. Story changes refer to the modifications in the middle of the project. 4.1.1 Unclear objectives In line with Wysocki (2007), it was identified in the study that clients are usually unsure of what they actually want when it comes to developing a new system for their business: “Upfront planning is the biggest problem as no one knows what they want”. Scrum uses the iterative approach which requires an extensive amount of communication and cooperation from all stakeholders and project team members (Cocco et al, 2011). This iterative approach allows for the full set of requirements to be determined iteratively. However, some participants admitted to continuing the use of some aspects of the Waterfall methodology, which are not in line with the above‐mentioned basic principle of Scrum: “We used to be waterfall based and still have a lot of principles that are still waterfall. We haven’t changed over completely yet”. This can thus be an explanation for some participants’ uneasiness about clients not knowing exactly what they want from the beginning. 4.1.2 Story changes Rising and Janoff, (2000) stated that clients are constantly changing their requirements. Cervone (2010) and Wysocki (2009) suggested a similar problem that clients often try to introduce modifications into the middle of a sprint. The research is consistent with the literature in terms of story changes in the middle of a sprint. Various participants mentioned that: “As people get closer to seeing what the final or end product is going to look like, then that’s when they start thinking that they actually want it another way […] we face a lot of problems with people changing what they want as they go along”. It is clear from the research that story changes in the middle of a sprint due to the clients changing their minds, and clients not knowing what they want from the beginning has a negative impact on the Scrum team making the client one of the primary sources of disturbances in the middle of a sprint.

4.2 Management Seven participants suggested that management is a possible source of disturbance in the middle of a sprint. The study revealed that the Management theme was composed of two sub themes: Lack of understanding and Interference. A lack of understanding refers to management not fully understanding the processes of the Scrum team. Interference refers to management interrupting the Scrum team and its processes. 4.2.1 Lack of understanding The findings of the research differ from the literature with regards to the level of understanding that is held by management. The literature obtained expressed problems pertaining to the issue of self‐managed teams which, according to Blankenship et al (2011), if not controlled correctly can lead to a lack of proper effective and efficient management. There were also no problems in this research with a change in management and poor management in the Scrum teams as suggested by Blankenship et al (2011) and Schwaber (1995). The findings from this research revealed that the primary issue when it comes to management is the lack of understanding. There were no mentions of the problems of self‐managed teams. One participant mentioned that: “Management do not push out the deadline. We just have to work harder. We do lots of overtime and sit here until the early hours of the morning and weekends just to try get it all done and cover the workload”. Another participant, when speaking about the relationship between top management and the managers involved with Scrum said that: “It is very important that the scrum master and the product owner have a strong relationship because otherwise things don’t get done and you are both left standing in front of the boss

259

Maureen Tanner and Angela Mackinnon explaining why nothing got done”. It is thus important for management to understand the processes of Scrum so that they can understand the causes of the problem when something goes wrong. 4.2.2 Interference The literature states that interferences in a project often come from higher management and can disrupt a Scrum team (Keil et al, 2003; Stettina, & Heijstek, 2011). The findings of this research are consistent with the literature on the subject of interference from management as various participants made mention of such disturbances: "Agile is supposed to be flexible on both side but a lot of places it’s only flexible in one direction, management direction, management just chop and change what they want, when they want to". It is noticeably clear that interferences from higher management are disrupting to the Scrum teams but this is generally due to a lack of understanding from higher management. This leads to the conclusion that management are a definite source of disturbance in the middle of a sprint.

4.3 External influences Seven participants suggested that the external influences are possible sources of disturbance in the middle of a sprint. External influences, in terms of this research, refer to the influences external to the Scrum team. The two sub‐themes were External Work and External Teams. External Work refers to work such as bugs fixing tasks that team members are responsible for that fall outside the scope of their Scrum work. External Teams refers to external influences from other teams such as other departments within the company. 4.3.1 External work One of the major sources of disturbance in terms of external influences is relative to bug fixing. Participants from all three companies mentioned that bugs of system portions that they had already completed take up a lot of time and this time is often not accounted for. Participants were quoted saying: “Bugs take priority over everything, some things need to be done now now now such as bugs and they take priority even though the sprint is supposed to take priority”. In one of the companies that took part in the interview, there was proof of processes in place to cater for the hours lost due to fixing bugs. One of the participating product owners stated: “We have a tab on our spread sheet called Interferences and basically our guys are keeping track there”. Another problem in terms of work is the expertise of some of the team members. There was a case where only one of the developers had a certain set of skills that was required to fix a problem. The problem took two full days to fix and this developer did not manage to complete any Scrum work hours in these two days. Because of this problem, some developers were forced to put their work on hold until the problem was resolved. This problem caused a great deal of tension within the teams and led the team to not complete the sprint backlog on time. 4.3.2 External teams External teams of people are sources of disturbance in the middle of a sprint but do not have as much of an effect as the external work does. The main problem determined in the research was that all the teams operate independently and demand certain things. It is impossible for every team within a company to have what they need when they need it when they are not communicating these needs to the relevant teams. The main problem is between departments within a company. Participants have mentioned that: “Every department has urgent things that they need done and you're in the middle of development and everyone has deadlines. You feel like you don’t have time to do it all and you can’t please everybody”

4.4 Processes Thirteen participants suggested disturbances can emerge because of the specific Scrum processes within an organization. The literature did not contain a lot of useful information pertaining to processes as a source of disturbance in the middle of a sprint. Processes are quite specific to different companies and it is therefore not an aspect that is broadly researched. The main processes that were evident as being sources of disturbances were the loss of resources, determined by Schwaber (1995) and Stettina and Heijstek (2011) and the time

260

Maureen Tanner and Angela Mackinnon pressures of Scrum as determined by Schwaber (1995). This study identified disturbances inherent to two aspects of Scrum processes within a company as discussed below. 4.4.1 Work load The work load of Scrum team members is the biggest and most disturbing source of disturbance in the middle of a sprint and was highlighted by all participants. Work‐load issues are related to wrong estimation of task duration as well as external work such as bug fixing. Another challenge identified relate to the running concurrent sprints. There was one instance where a developer was working on multiple sprints at the same time. This led to many unfinished tasks and a chaotic work ethic. This concurrency has been stopped but had, in the past, had been a source of disturbance in the middle of a sprint: “We have had a bad time with individuals in concurrent sprints which we have stopped”… “We tried having them work on concurrent sprints but it didn’t work and now we make sure that they are only working on one sprint at a time”. 4.4.2 Prioritising Prioritising is linked to work load as well as external work. Prioritising have negative effects on the sprint if done incorrectly. Priority is given to bugs and other important aspects that do not relate to the Scrum work from the product backlog. In particular, participants mentioned that: "…it was just a matter of prioritising time because some programmers would see one project as more important than the other” … “Priority shifted and other things such as bugs would take priority and it was a matter of not putting in concrete the actual processes we should have been following".

5. Communication Eleven participants suggested that communication challenges experienced during Scrum are possible sources of disturbance in the middle of a sprint. In the literature, there is more of a focus on the tools and mechanisms that need to be in place in order for effective communication to be possible (Sutherland, Viktorov, Blount, & Puntikov, 2007). This suggestion is supported by Verner et al (2012) who said that necessary tools are needed in order to communicate effectively. Although the findings of this research do support the need for effective communication tools and mechanisms, there was more of a focus on the particular problems caused by a lack of communication and the source of the problems. Interdepartmental communication, as a source of disturbance in the middle of a sprint, is closely linked to that of external teams. Problems occur when teams require something from another team but do not get it due to a breakdown in communication between the two teams. It was found, in the research, that often teams are unaware that another team requires information from them and therefore they do not produce the relevant information on time. Communication is the primary cause of the problems between departments and this was evident in various statements made by participants when asked about the problems. "Big problems with interdepartmental communication" “There are always problems between departments when you think that department isn’t getting things done but everyone also has lots of things that they are busy with” “On a high level, our teams interact. One team will carry on doing work but they don’t take into account that there is something that the other team needs from them in order to carry on. It causes delays”. It is evident from the findings that there are attempts at methods and processes to remove this problem but it is currently still an on‐going issue and therefore a source of disturbance in the middle of a sprint.

5.1 Lack of experience with scrum Five participants suggested that the lack of experience with Scrum and software development, are both possible sources of disturbance in the middle of a sprint. For example, one developer who had little knowledge of Scrum and felt that he had been thrown into the deep end due to the fact that Scrum was newly implemented by the time he arrived. He was not brought up to speed on what Scrum involved and how to go about his daily tasks. He felt that this lead to major disturbances in the sprint because he was unsure of what

261

Maureen Tanner and Angela Mackinnon he was actually supposed to be doing. Other problems include change management as there were a lot of changes implemented when Scrum was introduced and some team members found it difficult: "I think it is just the introduction of it in the first place because generally the developers are introverts which is not the case with our guys but a developer can be set in their ways and not open to change", “The system is not well documented at all and the procedures are tough to understand. I didn’t know who to email and when not to do things and that sort of thing. I got thrown in the deep end”.

6. Conclusion The purpose of this research was to investigate the primary sources of disturbances in the middle of a sprint. After conducting twenty interviews with participants from three different companies, an in depth analysis was undertaken in order to develop major themes and regular occurrences in the findings. Six themes were identified as being the sources of disturbances in the middle of the sprint: the Client, Management, External Influences, Processes, Communication, and Lack of Experience with Scrum. Each of these themes was detailed with reference to quotes from the interviews, linkages to the current literature, explanations of ideas and phrases, and detailed descriptions of the sub‐themes that became present throughout the process. In conclusion, there are various sources of disturbances in the middle of a sprint but there are companies that are aware of these disturbances and are in the process of developing procedures to mitigate the effect of these disturbances.

References Blankenship, J., Bussa, M., and Millet, S. (2011) “Managing Agile Projects with Scrum”, Project Agile .Net Development with Scrum, Professional and Applied Computing, pp 13‐27 Cervone, H.F. (2010) “Understanding agile project management methods using Scrum”. OCLC Systems & Services International Digital Library Perspectives, Vol 27, No. 1, pp18‐2 Cocco, L., Mannaro, K., Concas, G., and Marchesi, M. (2011) “Simulating Kanban and Scrum vs. Waterfall with System Dynamics”, 12th International Conference Proceedings, XP 2011, Madrid, Spain, pp117‐131 Deemer, P., Benefield, G., Larman, C., and Vodde, B. (2010) “The Scrum Primer”. Retrieved April 04, 2012, from http://goodagile.com/scrumprimer/scrumprimer.pdf Highsmith, J. (2011) “What is Agile Software Development?”, Cross Talk, The Journal of Defense Software Engineering, Vol 15, No.10, pp 4‐9. Keil, M., Rai, A., Mann, J.E.C., and Zhang, P. (2003) “Why Software Projects Escalate: The Importance of Project Management Constructs”, IEEE Transactions on Engineering Management. Vol 50, No. 3, pp 251‐262 Moe, N.B., Dingsoyr, T., and Dyba, T. (2009) “Overcoming Barriers to Self‐Management in Software Teams”, IEEE Software, Vol 26, No.6, pp 20‐26 Rising, L., and Janoff, N.S.,(2000) “The Scrum Software Development Process for Small Teams”, IEEE Software, Vol 17, No. 1, pp 26‐32 Schwaber, K., (1995) “Scrum Development Process: Advanced Development Methods”, Proceedings of OOPSLA ’95 Workshop on Business Object Design and Implementation, London, UK Stettina, C.J., and Heijstek, W. (2011) “Five Agile Factors: Helping Self‐management to Self‐reflect”, Proceedings of European Software Process Improvement Conference 2011, Roskilde, Denmark pp 84‐96. Sutherland, J., Viktorov, A., Blount, J., and Puntikov, N. (2007) “Distributed Scrum: Agile Project Management with Outsources Development Teams”, Proceedings of the 40th Hawaii International Conference on System Sciences 2007, Hawaii, USA Thomas, D.R. (2006) “A General Inductive Approach for Analysing Qualitative Evaluation Data”. American Journal of Evaluation, Vol 27, No 2, pp237‐246 Verner, J., Brereton, O.P., Kitchenham, M., Turner, M., and Niazi, M.K., (2012) “Risk Mitigation Advice for Global Software Development from Systematic Literature Reviews”, School of Computing and Mathematics, Keele University, Keele, Staffordshire, UK. Wysocki, R.K. (2007) Effective Project Management – Traditional, Agile, Extreme, Fifth Edition, Wiley publishing inc. Indianapolis

262

How Reflective Professionals Design and Evaluate Financial Information Management Systems Courses Hien Minh Thi Tran1 and Farshid Anvari2 1 Office of Financial Services, Macquarie University, Australia 2 Department of Computing, Macquarie University, Australia hien.tran@mq.edu.au Abstract: Financial Information Management Systems (FIMS) or Accounting Information Systems (AIS) is a cross‐discipline subject, often taught by Computing and Accounting disciplines. In recent years, demand for this subject has grown. However, educators have lamented high failure rates among accounting information systems students; professional bodies have reported that graduates lack sufficient meta‐cognitive knowledge of information systems to perform their tasks. Quality teaching of FIMS or AIS requires instructors to actively update their knowledge of accounting systems and information technology as well as to reflect on teaching techniques. Reflection and reflective practices are taught within the education discipline, and have grown in popularity among many other disciplines. Yet little has been written about how accounting and IT professionals reflect on their practice and how they apply their reflections to their teaching. This paper explores the research question: how can reflective professionals assist computing and accounting academics in the design and delivery of the FIMS or AIS courses? Through our case study at an Australian university, we provide insights into the application of constructivist theory and reflective practice strategies in teaching FIMS courses. We discuss (1) the rationale for the importance of constructivist theory, cognitive load theory, reflective and action‐research in teaching and learning, (2) Bloom’s Revised Taxonomy, (3) the applications of Bloom and the reflective concept to design and deliver FIMS courses, (4) reflection on our teaching strategies in applying these concepts and, (5) conclusions on how reflective professionals can assist computing and accounting academics in the design and delivery of FIMS or AIS courses. Our study supports the view that reflection is a strategy; the Bloom’s Revised Taxonomy and the PEER Model are tools to assist instructors in designing and delivering courses that enhance participant’s learning abilities. We propose the five dimensional reflective cycle to facilitate reflective practice among academic and professional instructors for designing and delivering FIMS and AIS courses. Keywords: active learning, reflective practice strategies, action‐research, evaluation, financial information management systems (FIMS), accounting information systems (AIS)

1. Introduction FIMS or AIS is a cross‐discipline subject, often taught by Computing and Accounting disciplines. The standard of the courses must meet the requirements of professional bodies. The American Institute of Certified Public Accountants and the International Accounting Education Standards Board have specified that new accountants should be familiar enough with technology to assess risk and automated business processes. (IFAC, 2007; Harrast, Strong and Bromley, 2010). Either FIMS or AIS is one of the core units required to be admitted as an associate member of the Certified Practising Accountant (CPA) Australia and is within the Commerce degree programs in many universities in Australia and the US (Lenard, Wessels and Khanlarian, 2010; Badua, Sharifi and Watkins, 2011; Essayyad and Ortiz, 2011). FIMS or AIS is a relatively new area in the Accounting discipline; in recent years, demand for it has grown (Ragan et al. 2006; Harrast et al. 2010; Lenard et al. 2010; Badua et al. 2011; Vatanasakdakul and Aoun, 2011). However, it has been reported that the integration of technology into the curriculum has been slow (Roberts, Kelly, and Medlin 2007; Harrast et al. 2010). Students have self‐reported lack of database knowledge, Enterprise Resource Planning and relevant technology topics including business accounting systems, spreadsheets and encryption (Harrast et al. 2010). Educators have lamented high failure rates among accounting information systems students (Yong and Aoun, 2008; Vatanasakdakul and Aoun, 2009, 2011). Professional bodies, employers and academics have voiced concern that graduates lack sufficient meta‐ cognitive knowledge of information systems to perform their tasks or are ill‐equipped to meet the challenges of working in modern organisations (Velayutham and Perera, 2008; Harrast et al. 2010; Rai, Vatanasakdakul and Aoun, 2010; Tan and Sedera, 2010; Badua et al. 2011). Quality teaching of FIMS requires instructors to actively update their knowledge of accounting systems and information technology as well as to reflect on teaching techniques. Reflection and reflective practices are taught within the education discipline (Birenbaum and Amdur, 1999; Carlo, Hinkhouse and Isbell, 2010), and have grown in popularity in other disciplines e.g. health education (Thorpe 2004; Plack and Greenberg 2005;

263

Hien Minh Thi Tran and Farshid Anvari Mann, Gordon and MacLeod 2009). Reflective studies have been introduced in engineering (Kelly, 2008), computing (Hazzan, 2002; Hazzan and Tomayko, 2003) and accounting disciplines (Samkin and Francis, 2008; Hancock et al. 2009, McGuigan and Kern, 2009, 2010). However, accounting students have found the concept of reflection difficult to comprehend and educators are warned about staff additional workload (Samkin et al. 2008). Schön (1983), an influential thinker in developing the theory and practice of reflective professional learning, recognised the significant contribution of critical reflection in the development of professional knowledge. Schön provided examples of how other professions reflect however he did not include accounting and IT professionals in his books. Little has been written about how accounting and IT professionals reflect on their practice and how they apply their reflections to the teaching of FIMS or AIS. This paper explores the research question: how can reflective professionals assist computing and accounting academics in the design and delivery of FIMS or AIS courses? We discuss (1) the rationale for the importance of constructivist learning theory, cognitive load theory, reflective and action‐research in teaching and learning, (2) Bloom’s Revised Taxonomy, (3) the applications of Bloom and the reflective concept to design and deliver FIMS courses, (4) reflection on our teaching strategies in applying these concepts and, (5) conclusions on how reflective professionals can assist computing and accounting academics in the design and delivery of FIMS or AIS courses.

2. Literature review Constructivist learning theory states that learning is an active process of creating meaning from experiences. Learners interpret concepts and principles in terms of the ‘schemata’ that they have already developed (Biggs and Tang 2011, p.22). Cognitive Load Theory (CLT) states that human memory consists of sensory memory, working memory where during learning schemas are generated and, long‐term memory where knowledge is stored in the form of schemas; a well designed instructional material would present the new information such that the schema generation is within the working memory of a learner (Sweller, Merrienboer and Paas 1998). Instructional strategies that encourage deeper thought and further consideration of course topics lead to robust discussions and heighten student interest and motivation levels. Deep learning occurs when students are able to connect with course topics, find value in them and see how to apply them to real‐world situations (Delotell, Millam and Reinhardt, 2010). Schön (1983) introduced the terms of reflection‐in‐action and reflection‐on‐action. He describes reflection‐in‐ action as ‘thinking on our feet’, the thinking and reflecting that happens in the midst of activity and, reflection‐ on‐action as the thinking and reflecting that occurs after an event. Killion and Todnem (1991) extended Schön’s concepts to include reflection‐for‐action which is to review what has been accomplished and identify constructive guidelines to follow to succeed in the given task in the future. The importance of reflection is noted in the literature and reflective capacity is regarded by many as an essential characteristic for professional competence. (Mann et al. 2009) There are four key dimensions of reflection: describe, analyse, transform meaning and action (McGuigan et al. 2009). Biggs et al. (2011, p.45‐46) introduced the term transformative reflective and defined it as a multi‐stage process of: reflect‐plan‐apply‐evaluate (did it work?). Reflective practice requires action‐research. Action‐research is when teachers perform research on themselves, their students, classrooms etc for the purpose to improve teaching and learning (Carlo et al. 2010).Through action‐research, the teacher is self‐monitoring the improvement of his or her decisions and actions. Action‐research involves changing the teaching practice using whatever on‐the‐ground evidences that the teacher can obtain and can be put into the right action (Biggs et al. 2011, p.51). The one‐minute paper is a technique whereby students write brief answers to such questions as: What do I most want to find out in this class? What was the main point left unanswered in today’s session? One‐minute papers have been used in varied courses e.g. chemistry, art history, multicultural seminars, accounting, economics (Stowe, 2010).

3. Bloom’s revised taxonomy Bloom initiated taxonomy of education objectives. Bloom’s Revised Taxonomy incorporates the Knowledge Dimension and the Cognitive Process Dimension (Anderson and Krathwohl, 2001). The Knowledge Dimension is grouped into four categories as follows:

Factual knowledge is knowledge of discrete elements of content, including knowledge of terminology or of specific details or elements;

264

Hien Minh Thi Tran and Farshid Anvari

Conceptual knowledge is knowledge of more complex, organised forms such as classifications, categories, principles or generalisations, theories models and structures;

Procedural knowledge is the knowledge of how to do something, including the use of skills and algorithms, techniques and methods. This category also includes knowledge of ‘when to do what’;

Meta‐cognitive knowledge is a meta understanding; that is, the learner knows what they know and don’t know and knows what to do to come to know. It is both self‐knowledge and strategic knowledge (Anderson et al. 2001).

The Cognitive Process Dimension, which refers to learning processes, is grouped into six categories (Krathwohl, 2002):

Remember is to retrieve relevant knowledge from long‐term memory;

Understand is to construct meaning from instructional messages: oral, written or graphic;

Apply is to carry out or use a procedure in a given situation;

Analyse is to break material into parts and determine how they relate to each other and to the overall structure or purpose;

Evaluate is to make judgments based on criteria and standards;

Create is to pull elements together into a new pattern or structure.

These categories differ in complexity, with remember being the least complex and create being the highest rung of the Cognitive Process (Krathwohl, 2002). In teaching methods that use a linear presentation of materials (e.g. textbooks, lectures) students gain knowledge at the most basic level and memorise scientific facts without understanding the underlying concepts (Cepni, Tas and Kose, 2006). Table 1 lists examples of the verbs that can be used for each Cognitive Process to develop course materials and to assist in devising methods to teach at any level of the Knowledge Dimension; which verbs to be used and for what category of the Knowledge Dimension depend on the topics of the course. This list has been adapted from the diagram of the “task‐oriented question construction wheel” from St Edward’s University, (2004), based on the Bloom’s original taxonomy. Table 1: Verbs used to describe each category of the cognitive process dimension KNOWLEDGE DIMENSION Remember FACTUAL KNOWLEDGE CONCEPTUAL KNOWLEDGE

PROCEDURAL KNOWLEDGE

META‐COGNITIVE KNOWLEDGE

Memorise Name Recite Define Describe Identify State Locate Recognise Select

Draw Label Write

COGNITIVE PROCESS DIMENSION Understand Apply Analyse Match Confirm Restate Change Compare Paraphrase Express Explain Extend Relate Distinguish Illustrate Defend Generalise Infer Predict Summarise Transform

Choose Compare Collect Contrast Discover Distinguish Classify Analyse Draw Categorise Interpret Classify Construct Differentiate Apply Analyse Change Examine Draw Infer Make Select Modify Investigate Report Point out Show Dramatise Research Draw Separate Model Subdivide Paint Survey Prepare Take apart Produce

265

Evaluate

Create

Compare

Add to

Apprise Assess Consider Relate Summarise Judge Recommend Summarise Solve

Combine Construct Organise Create Develop Formulate Role‐Play Plan Produce What if? Create Design Invent Hypothesis e Originate

Criticise Critique Judge Recommend Weigh

Hien Minh Thi Tran and Farshid Anvari

4. The applications of Bloom and reflective concept in designing and delivering FIMS courses At a research‐intensive metropolitan Australian University, FIMS courses have been offered to professional staff since 2007. FIMS consisted of four three‐hour face‐to‐face interactive learning workshops, spanning over a nine‐week period which covered General Ledger, Purchase Requisitions, Financial Information Management Reports and Accounts Receivable modules. Participants were administration and finance staff, department and faculty managers, post‐graduate students and academics.

4.1 Reflective strategies To reflect‐on‐action (Schön, 1983) and reflect‐for‐action (Killion and Todnem, 1991), the author Tran kept a reflective journal in which she described her teaching and learning environments, analysed theories and techniques (such as constructive and cognitive load theory), summarised action‐research findings, noted her reflection and self‐evaluated her teaching methods. Reflection was her teaching strategy and the reflective journal was a tool for her reflection. To address the needs of a diverse group, participants provided expectation forms prior to the first hands‐on workshop. To engage participants in the course materials, Tran reflected upon their expectations and prior knowledge and incorporated these into the training program. A few days prior to each training workshop, participants emailed Tran one‐minute papers (Stowe, 2010), answering two questions: ‘what did you find useful in the last training workshop?’ and ‘what questions remain on your mind?’ The one‐minute papers encouraged participants to raise queries and provided Tran a means to follow up. From participants’ expectation forms and one‐minute papers, topics that needed to be addressed were identified. Issues were reflected upon, action‐research was conducted and appropriate strategies were carried out. For example, various methods of delivery were adapted to suit participants’ special needs (e.g. more theory or more practice; more financial reports or more procedures). Training documents were regularly updated to enhance participants’ learning abilities. If the questions raised in one‐minute papers were private Tran answered them via email. Questions that were general, were answered in the workshops. Tran demonstrated how to solve them using real‐world scenarios.

4.2 Teaching and learning resources and activities Bloom’s Revised Taxonomy Knowledge Dimension was used to identify the Factual, Conceptual, Procedural and Meta‐cognitive knowledge parts of FIMS curriculum. Table 2 shows the application of Bloom’s Revised Taxonomy Knowledge Dimension to identify teaching and learning processes. Table 2 contains samples of two Knowledge Dimension categories (Conceptual and Procedure) listed in order of the Cognitive Process Dimension from the lowest to the highest rung. The cognitive load on the memory and the learn‐ability of learners depend on the training materials (Sweller et al. 1998). Cognitive load on participants were considered e.g. participants’ expectations and prior knowledge; worked‐examples and scenarios were used as the modelling technique; training documents were illustrated with pictures and descriptions of real‐world scenarios. All participants had access to the university financial system to practice. Participants actively learnt during the three‐hour workshops then independently practised worked‐examples documented in the course materials within the training environment.

4.3 Evaluation and assessments To develop questionnaires for evaluation, Tran used the Peer Review of Learning and Teaching model – also known as the PEER model (Macquarie University, 2012). This model consists templates and check list forms with questions Why; What; Who; How; Reporting; and Follow up. Questionnaires suggested in the PEER model were adapted. Twenty‐five questions were developed under five headings:

Planning and organisation;

Documentations and contents;

266

Hien Minh Thi Tran and Farshid Anvari

Training strategies and resources;

Presentation and management;

Feedback and follow up of participants.

Table 2: Samples of the application of Bloom’s Revised Taxonomy Knowledge Dimension into the learning and teaching of FIMS TEACHING AIMS: KNOWLEDGE DIMENSION Conceptual Knowledge of FIMS reports.

LEARNING OUTCOMES

Module 1: Identify the various types of reports that are used by managers and business analysts and run some simple reports. Conceptual Module 1: Knowledge of the Compare and University’s contrast the Accounting various account Ledgers. components and selection codes of various ledger accounts. Procedural Knowledge of Purchase Requisitions Processing. Procedural Knowledge of Tax Invoices.

Module 2: Generate Purchasing Requisitions, reports and interpret the data. Module 3: Monitor the status of processing of Tax Invoices.

Procedural Knowledge of AR Tax Invoices.

Module 4: Create an AR scheduled Tax Invoice and, interpret the features and components of a Recurring Documents Tab.

LEARNING PROCESSES

TEACHING PROCESSES & LEARNING ACTIVITIES

COGNITIVE PROCESS DIMENSION

Memorising key terms. Understanding concepts.

Explanation; Practice with feedback.

Remember This is an example of the lowest level of the Cognitive Process Dimension.

Recognising, understanding rules and concepts; Interpreting ledgers’ account codes; Understanding how the account codes work within FIMS context. Raising Purchase Requisitions; Understanding Purchase Requisitions Processes; Interpreting Purchase Requisition reports. Understanding cause and effect in relation to certain functions; Distinguishing transactions in various scenarios.

Explanation; Reading; Practice with feedback.

Understand This is an example of the second lowest level of the Cognitive Process Dimension.

Case Studies; Exposure to Real‐life worked‐ examples and processes; Modelling by instructors; Practices with feedback. Case Studies; Exposure to Real‐life worked‐ examples and processes; Modelling by instructors; Scaffolded exercises; Practices with feedback. Explanations; Case Studies; Exposure to Real‐life worked‐ examples and processes; Modelling by instructors; Scaffolded exercises; Repeated practices with feedback.

Apply and Analyse This is an example of the mid‐level of the Cognitive Process Dimension.

Recognising rules, analytical reasoning to be able to discriminate between different processes in various scenarios; Developing meta‐ cognitive, self diagnosed strategies.

Evaluate This is an example of the second highest level of the Cognitive Process Dimension.

Create This is an example of the highest level of the Cognitive Process Dimension.

Tran also employed self‐assessment techniques. Participants’ feedback and surveys confirm the benefits listed by Falchikov and Thompson (2008) in involving students in assessment. Empowering learners: Participants actively engaged in learning during the three hour workshop. Active learning with technology had positive effects on student learning; real‐world application of the materials heightened students’ interest and enhance their learning abilities (Ragan et al. 2006).

267

Hien Minh Thi Tran and Farshid Anvari Encouraging attention: Through one‐minute papers we encouraged participants to raise queries and provided constructive and practical feedback. By involving participants, we were able to empower them, encourage dialogue and attract attention. Creating partnerships: Through expectation forms and one‐minute papers we encouraged participants to raise questions that would not otherwise have been raised in the class. Fostering proactive strategies in teachers and students: One‐minute papers and surveys helped to identify topics to be address and strategies to deliver them. Reflection: By asking participants to submit one‐minute papers and to reply to the survey questions, we encouraged them to reflect‐on‐action. Enhancement of learning: By recognising participants’ prior knowledge from their self‐assessments, we were able to develop course materials that challenged them. Performance: Participants were more engaged in class discussions when they were challenged to perform at the higher rung of cognitive processes. This is consistent with the results of the research conducted by Arum and Roksa (2011) that when high‐order thinking is included in the coursework students perform better on tests measuring critical thinking. Personal development/autonomy: Our case supports prior studies that a hands‐on approach motivates students which subsequently increase their understanding of the business processes (Draijer and Schenk, 2004).

5. The reflection of our teaching strategies in applying these concepts Based on our case study and literature review, we propose the reflective cycle that can be used to facilitate reflective practice among academic and professional instructors for designing and delivering high‐quality FIMS and AIS courses. The reflective cycle consists of five dimensions: Describe, Analyse, Transform, Act, and Evaluate. For every process or issue identified by the instructor these five dimensions should be considered in a cycle as illustrated in figure 1.

Figure 1: The five dimensional reflective cycle

Describe: This dimension is to describe the process or the issue requiring reflection. The instructors use reflective journals or diaries to describe it. For example, when the instructors design course materials, they first describe who the learners are and what they want to learn; what contents the instructors need or wish to include; what survey or research questions the instructors need to formulate. Another example is that the instructors reflect on their teaching methods after each class then describe an event or an incident which they wish to reflect‐on‐action in their journals. A reflective journal enables a teacher to

268

Hien Minh Thi Tran and Farshid Anvari take regular notes of events and observations, record a tentative hypothesis and develop new understandings similar to a scientist keeping laboratory notes (UTS 2007).

Analyse: This dimension is to analyse the process or the issue which the instructors have described in dimension one. The instructors analyse the methods which they plan to act or how they go about to obtain the necessary information and gain the required knowledge. Moon (cited in McGuigan et al. 2009) defines the analysis dimension as ‘a mental process with purpose and/or outcome in which manipulation of meaning is applied to relatively complicated or unstructured ideas in learning or to problems for which there is no obvious solution.’ For example, the instructors use the Bloom’s revised Taxonomy to analyse how to design course materials that encourage critical thinking from the learners or to analyse the intended learning outcomes in comparison with the participants’ learning outcomes to identify issues. Another example is that the instructors reflect on their actions or on the questions raised by learners in the one‐minute papers to determine what they could have done better or to incorporate the issues raised by participants in their training materials where applicable.

Transform: This dimension often requires action‐research or reflect‐on‐action to transform meaning of the above analysis. Action‐research is about creating awareness and understanding, in order to improve a certain practice (Mahani and Molki, 2012). For example, the instructors conduct action‐research to find possible solutions to an issue and, develop examples that they can use to illustrate the issue. Another example is that the instructors introduce innovative teaching methods to effectively transmit knowledge or new techniques to inspire learners personally engage with learning materials and design training materials that keep the cognitive load on the working memory within the limits of a person.

Act: This dimension is to act on issues that the instructors have analysed in dimension two based on the decisions which they have decided in dimension three above. For example, the instructor conducts literature review and learns that for deep learning to occur students need to connect with course topics and apply them (Delotell et al. 2010). The instructor then decides to use real‐world applications and hands‐on approach to address issues raised in the one‐minute papers and, involve learners in making decisions on the learning activities that are applicable to them.

Evaluate: This dimension is to evaluate whether the instructors’ actions in dimension four indeed enhances learning and teaching experiences of the instructors and the participants and, to monitor their actions to gauge their success. Expert teachers evaluate their teaching methods to more effectively motivate students to solve their own problems (Biggs et al. 2011, pp 45‐55). For example the instructors employ the PEER model to evaluate the effectiveness of their teaching strategies and methods. Another example is that the instructors practice reflection to self‐evaluate and commit to continuous improvement that enhances teaching and learning experiences.

The above five dimensional processes are repeated continually. The processes help to enhance the learning and teaching qualities. Biggs et al. (2011, p.45) emphasises that expert teachers continually reflect on their teaching.

6. Conclusions Quality teaching of FIMS or AIS requires instructors to actively update their knowledge of accounting systems and information technology as well as to learn how to teach and to reflect on teaching techniques. Constructivist Learning theory and Cognitive Load theory help instructors to design course materials that assist learners to absorb new information. Bloom’s Revised Taxonomy, the Peer model, one‐minute papers and expectation forms are tools to assist instructors in designing and delivering courses that enhances participant’s learning abilities. Reflection is a strategy. Being a reflective instructor enables teachers to learn about themselves. We can enhance students learning abilities by improving the quality of our teaching through our own reflections, self monitoring the improvement of our decisions and actions and, change our teaching practice using the evidences that are available and can be put into action. Through our case study we provide insights into the application of constructivist theory and reflective practice strategies in teaching FIMS. The constructivist framework adopted in our case was that of a blend between participants’ personal and developmental position and the hands‐on activity workshop environment. The implication of our study is that three‐hour intensive hands‐on FIMS workshops on real‐world scenarios benefit

269

Hien Minh Thi Tran and Farshid Anvari the learners and maximises their learning abilities. We provide the five dimensional reflective cycle and demonstrate how it can be used to facilitate reflective practice among academic and professional instructors for designing and delivering FIMS courses for professional staff or AIS courses for undergraduate students.

Acknowledgements We thank Dr Agnes Bosanquet and the anomyous reviewer for their helpful comments.

References Anderson, L. W. and Krathwol, D. R. (2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Arum, R. and Roksa, J. (2011) Academically Adrift: Limited learning on college campuses. Chicago: The University of Chicago Press. Badua, F., Sharifi, M. and Watkins, A. (2011) “The Topics, They Are A‐Changing: The State of the Accounting Information Systems Curriculum and the Case for a Second Course”, The Accounting Educators’ Journal, Vol. 21, pp 89‐106. Biggs, J. and Tang, C. (2011) Teaching for quality learning at University, McGraw Hill Society for Research into Higher Education and Open University Press, 4th Edition. Birenbaum, M. and Amdur, L. (1999) “Reflective Active Learning in a Graduate Course on Assessment”, Higher Education Research and Development, Vol. 18, No. 2, pp 201‐218 Carlo D. D., Hinkhouse H. and Isbell L. (2010) “Developing a Reflective Practitioner Through the Connection Between Educational Research and Reflective Practices” J Sci Educ Technol, Vol. 19, pp 58‐68. Cepni, S. Tas, E. and Kose, S. (2006) “The effects of computer‐assisted material on students’ cognitive levels, misconceptions and attitudes towards science”, Computers and Education, Vol. 46, pp 192‐205. Delotell, P. J., Millam L. A. and Reinhardt, M. M. (2010) “The Use Of Deep Learning Strategies In Online Business Courses To Impact Student Retention”, American Journal of Business Education, Vol. 3, No. 12. Draijer, C. and Schenk, D. (2004) "Best Practices of Business Simulation with SAP R/3,"Journal of Information Systems Education, Vol. 15, No. 3, pp 261‐265 Essayyad, M. and Ortiz, D. (2011) “Evaluation of an accounting programme for the purpose of improving the passing rates of CPA examination and meeting CPA supply shortage”, in the Texas Lower Rio Grande Valley. Int. J. Trade and Global Markets, Vol. 4, No. 1. Falchikov, N. and Thompson, K. (2008), “Assessment: What Drives Innovation?” Journal of University Teaching & Learning Practice, Vol. 5, No. 1. Hancock, P., Howieson, B., Kavanagh M., Kent J., Tempone I., Segal N. (2009) Accounting for the future: more than numbers, Vol. 2. Harrast, S., Strong, J. and Bromley, R. (2010) “More Accounting Theory or More Information Technology?” The Accounting Educators’ Journal, Vol. 20, pp 1‐19 Hazzan, O. (2002) “The reflective practitioner perspective in software engineering education”, The Journal of Systems and Software, Vol. 63, pp 161–171. Hazzan, O. and Tomayko, J. (2003) “The Reflective Practitioner Perspective in eXtreme Programming” in Maurer, F. and Wells, D. (Eds.): XP/Agile Universe, LNCS 2753, pp 51–61. IFAC (2007) “Information Technology for Professional Accountants”, in I.E.P.S. 2. New York: International Federation of Accountants. Kelly, P. (2008) Towards Globo Sapiens: Transforming Learners in Higher Education, Sense Publishers, Rotterdam. Killion, J. and Todnem, G. (1991) “A process for personal theory building”. Educational Leadership, Vol. 48, No. 7, pp 14‐16. Krathwohl, D. (2002) “A revision of Bloom’s Taxonomy: An Overview”, Theory Into Practice, Vol. 41, No. 4. Lenard, M. J., Wessels, S., Khanlarian, C. (2010) “Gender Differences in Attitudes Toward Computers and Performance in the Accounting Information Systems Class”, American Journal of Business Education, Vol. 3, No. 2. Macquarie University (2012) Peer Review of Learning and Teaching, viewed 26 Jan 2013, <http://staff.mq.edu.au/teaching/teaching_development/peer/> Mahani S. and Molki A. (2012), “Enhancing The Quality Of Teaching And Learning Through Action Research”, Journal of College Teaching & Learning, Vol. 9, No. 3. Mann, K., Gordon, J. and MacLeod, A. (2009) “Reflection and reflective practice in health professions education: a systematic review”. Adv in Health Sci Educ, Vol. 14, pp 595–621. McGuigan, N. and Kern, T. (2009) “The Reflective Accountant: Changing Student Perceptions of Traditional Accounting through Reflective Educational Practice”. The International Journal of Learning, Vol. 16, No. 9, pp 49‐68. McGuigan, N. and Kern, T. (2010) “Leaping over the perceptual threshold in accounting courses: embracing sustainable assessment and learning design”. ATN Assessment Conference: Assessment: Sustainability, Diversity and Innovation Plack, M. M. and Greenberg, L. (2005) “The Reflective Practitioner: Reaching for Excellence in Practice”, Pediatrics, Vol. 116, pp 1546‐1552 Rai, P., Vatanasakdakul, S. and Aoun, C. (2010) "Exploring perception of IT skills among Australian accountants: An alignment between importance and knowledge", AMCIS Proceedings. Ragan, J., Gantner, M., Caffrey, P. and Schultz, M. (2006) “A Teaching Case Using Cost/Benefit Relationships Within An ERP System”, Journal of Business Case Studies, Vol. 2, No. 3, pp 17‐24.

270

Hien Minh Thi Tran and Farshid Anvari Roberts, F. D., Kelley C. L. and Medlin B. D. (2007) “Factors Influencing Accounting Faculty Members' Decision to Adopt Technology in the Classroom”, College Student Journal, Vol. 41, No. 2, pp 423‐35. Samkin, G. and Francis, G. (2008) “Introducing a Learning Portfolio in an Undergraduate Financial Accounting Course”, Accounting Education: An International Journal, Vol. 17, No. 3, pp 233‐271. Schön, D. A. (1983) The reflective practitioner: how professional think in action, Basic Books, New York. St Edward’s University (2004), “Task‐oriented question construction wheel, based on Bloom’s taxonomy”, viewed 17 November 2012, http://think.stedwards.edu/cte/sites/webdev1.stedwards.edu.cte/files/docs/BloomPolygon.pdf Stowe K. (2010), “A Quick Argument for Active Learning: the Effectiveness of One‐Minute Papers”, Journal for Economic Educators, Vol. 10, No. 1, pp 33‐39. Sweller, J. Merrienboer, J. J. and Paas, F.G. (1998) “Cognitive Architecture and Instructional Design”, Educational Psy‐ chology Review, Vol. 10, No. 3, pp 251–296. Tan, F. T. C. and Sedera, D. (2010) "Introducing a Small and Medium Enterprises Syllabi Design: An Enterprise Systems Hands‐On Approach", ACIS Proceedings. Thorpe K., (2004) “Reflective learning journals: From concept to practice”, Reflective Practice: International and Multidisciplinary Perspectives, Vol. 5, No. 3, pp 327‐343 UTS (2007) “Keeping a Reflective Journal”, University of Technology Sydney, viewed 6 July 2007, http://www.clt.uts.edu.au/Scholarship/Reflective.journal Vatanasakdakul, S. and Aoun, C. (2009) "I Don't Like IT: Exploring Challenges in Accounting Information Systems Education", AMCIS Proceedings. Vatanasakdakul, S. and Aoun, C. (2011) "Why don't accounting students like AIS?" International Journal of Educational Management, Vol. 25, No. 4, pp 328–342 Velayutham, S., Perera, H. (2008) “The role of professional accounting programmes: towards a reflective practicum”, International Journal of Management Education, Vol. 7, No. 1 Yong, R. and Aoun, C. (2008) “Generic Skills to Reduce Failure Rates in an Undergraduate Accounting Information System Course”, Asian Social Science, Vol. 4, No. 10, pp 60‐70.

271

Implementation of a Corporate Performance Management System in a Large City Mark Tudor and Mike Hart University of Cape Town, Cape Town, South Africa Mark.Tudor@hcl‐axon.com mike.hart@uct.ac.za Abstract: This study examines the implementation of a Corporate Performance Management (CPM) Information System in the context of a large South African city. The role of Information and Communication Technology (ICT) in supporting and facilitating CPM (also known as Business Performance Management) is also examined. CPM has evolved greatly since the early static Executive Information Systems of the late 1980’s, now usually building on the Balanced Scorecard and Strategy Map concepts of Kaplan and Norton. To be effective, CPM systems require sound Business Intelligence capabilities and integration of many technological aspects, apart from many organizational considerations such as top management support, organizational culture and change management. Starting at a high strategic level, CPM systems ripple down to lower levels of the organization, incorporating key performance indicators (KPIs) and performance metrics, business processes, dashboards and scorecards. Given recent events in the financial sector, many CPM systems now incorporate additional elements concerning Corporate Governance and related legislation. For this research, key role players from the city and consultants involved in the implementation of enterprise resource planning (ERP) systems were interviewed, and documentation and the CPM system examined. Analysis used an interpretive Actor‐Network Theory (ANT) framework. Findings show that: 1) ICT can play a strong role in enabling an effective CPM system, but when it does not deliver as expected, this may have negative consequences. 2) Adoption of the business processes associated with CPM is closely linked to adoption of the CPM ICT, but the performance culture of an organisation can block the adoption of both. 3) ANT proved to be a useful way of analysing the overall implementation process. While findings were for a specific city, it is argued that knowledge of many of the issues that arose in this implementation could be of value to others embarking on introduction of CPM systems in either the public or private sector. Keywords: performance management, implementation, public sector, CPM system, KPI

1. Introduction This paper examines the introduction of a corporate performance management (CPM) system to a large city, and highlights certain issues that emerged from interviews with some of the key players. While also referred to as business performance management (BPM), CPM will be used to avoid confusion with business process management. Leading consultants (Rayner 2009) note that CPM system implementation and adoption is typically very challenging, and that most organisations have not progressed very far in this regard. Literature usually focusses on commercial organisations, as opposed to public institutions, tending to be dominated by positivistic research. It is argued that this has somewhat curtailed the description of the complexities associated with organisational installation of such systems. This research therefore used interpretivism, hermeneutics and Actor‐Network Theory (ANT) with the objective of providing a rich description of an example of the phenomenon of implementation of CPM in a large city. A literature review summarises key aspects of CPM and of ANT. Details of the research methodology and the case to be analysed follow. The analysis then exposes the emergent actors and pertinent issues surrounding the CPM implementation, followed by a brief discussion and conclusion.

2. Corporate performance management (CPM) The concept and implementation of CPM has evolved over a long period. Pratt (1991) suggests that organisations have been formulating strategies and measuring their performance against these plans for centuries. Executive information systems (EIS) were a 1980’s attempt to create management information systems at a strategic level, but failed to gain traction (Eckerson 2011). The Balanced Scorecard concept of Kaplan and Norton (1992) expanded the areas of performance to be monitored from being largely financial to also include internal business processes, clients and stakeholders, and learning and innovation. The data warehouse created improved structure and capabilities for data storage and extraction, and business intelligence (BI) became recognised as a new umbrella concept. During the 2000’s CPM became one of the major applications of BI, and various vendors created CPM modules, and integrated these with existing BI infrastructure (Wade & Recardo 2001). Neely, Marr, Roos, Pike and Gupta (2003) introduce the concept of

272

Mark Tudor and Mike Hart third generation performance measurement systems, while Rayner (2009) defines Gartner’s four stages of performance management evolution. Following the financial debacles of the early and late 2000’s, financial compliance and governance became increasingly important, and Van Decker, Chandler, and Iervolino (2012) note that aspects of these are now being included in expanded CPM suites, with components such as: Budgeting, planning and forecasting (BP&F); Profitability modeling and optimization (PM&O); Strategy management; Financial consolidation and close; and Financial and management reporting and disclosure. This paper will concentrate largely on the strategy management aspects, which cover all parts of the organisation, and “provide a packaged approach to support strategic planning, modelling and monitoring to improve corporate performance, accelerate management decision making and facilitate collaboration” (Van Decker, et al. 2012, p2). These typically incorporate scorecards, and dashboards which set goals for and monitor Key Performance Indicators (KPI’s) (Eckerson 2011; Viaene & Willems 2007). Neely, Richards, Mills, Platts, and Bourne (1997) recommend 22 different criteria for selecting performance measures. Ariyachandra and Frolick (2008) define a CPM framework of four core processes: (1) strategize, (2) plan, (3) monitor and analyse, and (4) take corrective action; and suggest ten critical success factors for a CPM initiative.

3. Actor‐network theory (ANT) Many different theories, frameworks and lenses have been used in attempts to shed light on implementation, adoption and diffusion of information systems in organisations, and space does not permit discussion here. Actor‐Network Theory is “a complex and continually developing theory” (Ranerup 2012), which builds on the seminal work of Callon, Latour and Law (e.g. Callon 1986; Latour 1987; Law 1992). Tatnall and Gilding (1999) state that it is particularly useful for examining implementation of information systems. Actors build networks combining social and technical elements, and principles of heterogeneity, symmetry and agnosticism imply that all actors, human or technological, receive equal attention (Hardy & Williams 2008; Ranerup 2012). The focal actor in the network attempts to Translate the interests of other actors to their own interest. The process of translation has four “moments” (Stanforth 2006): The first is Problematization, where the focal actor(s) try to make themselves indispensable by defining the nature of the problem, their role in a proposed solution, and the roles of other actors. They also identify an obligatory passage point (OPP) for solving the identified problem (Callon 1986). The second is Intéressement, the process of locking in place the identities and roles defined in the problematization, or convincing the other actors to accept the definition of the focal actor. Enrolment deals with defining and coordinating the roles, with negotiation leading to a stable network of alliances. Finally, Mobilization occurs when the proposed solution gains wider acceptance, and those enrolled also act as spokespersons for the network. The concept of Inscription refers to “the type of transformation through which any entity becomes materialised into a sign, an archive, a document...” (Latour 1999 in Hardy & Williams 2008), and the strength of the inscription depends on the Irreversibility of the actor‐network (its durability and institutionalisation). Black boxing enables actors in an Actor‐Network to be viewed as either a larger macro entity (Black Box), or broken down into its own Actor‐Network.

4. Research approach The research aimed to explore in some depth the issues and roles of actors in the phenomenon of implementation (not adoption or diffusion) of a CPM system, and used interpretivism, hermeneutics (Klein & Myers 1999) and ANT to obtain a rich description of an example of the implementation of CPM in a large city administration. In the public sector space, for example, ANT has been used to examine e‐procurement (Hardy & Williams 2008), IT governance (Hsbollah et al. 2012), DSS and pension reform (Ranerup 2008), e‐governance (Ranerup 2012) and e‐government in developing countries (Stanforth 2006). Interviews (some repeated) were held with the SAP Programme Manager (SAPPM), the CPM Project Manager (CPMPM), the Technical Architect (TA), the Lead Business Analyst (BA), and the BI Lead (BIL), and relevant documentation was obtained where possible. Interviews were staggered over several weeks, shortly after the

273

Mark Tudor and Mike Hart system had theoretically gone live, allowing for critical reflection and evaluation between interviews, which were transcribed and analysed through the lens of ANT. Authorisation was given at a senior level for a study in Cape Town. The researcher also spent time at municipal offices gaining insight into the social context of the implementation, and the organisational culture through informal discussions with staff, and obtained understanding of the technical solution through online analysis of the CPM tool.

5. Case study: Cape Town 5.1 Cape Town and South Africa in context South Africa is a country of great contrasts, illustrated by some ratings from the latest Global Competitiveness Report (World Economic Forum 2012). It ranks as number one out of 144 countries for each of its “strength of auditing and reporting standards”, efficacy of corporate boards”, “regulation of securities exchanges”, and financial “legal rights index”, and second for its “soundness of banks”, “availability of financial services” and “protection of minority shareholders’ interests”. While rated 35th for its “transparency of government regulation”, it is 88th for its “public trust in politicians”, 123rd for “burden of government regulation” and 106th for “government provision of services for improved business performance”. It is 144th (last) in labour‐ employee cooperation (seven places below France). It comes in 134th (immediately after Greece and Spain) for its low linkage between worker pay and productivity. It is 15th in terms of quality of management schools, but 143rd for maths and science education, 62nd in “technological readiness” and 38th in “business sophistication”. Some ratings are still affected by the country’s apartheid legacy, but the ruling African National Congress (ANC) government took over in 1994, with much new legislation since then and a new “progressive” 1996 constitution. Municipal (local) government experienced fundamental changes as part of the post‐1994 reforms. Municipalities are elected every four years, and run local affairs in line with principles of co‐operative governance of National and Provincial government, subject to legislation passed down by both these legislatures. The seven largest local municipalities in South Africa are known as Unicities. Cape Town hosts the National Parliament and is capital of the Western Cape province. The 2011 census found that this province had grown by 29% since 2001 to 5.9 million, making it the fastest growing province (Johns, Nkomo, and Jooste 2012). Sixty four percent, or 3.7 million, of the Western Cape population live in the Cape Town area (Coetzee 2012).

5.2 City governance The Cape Town Unicity is governed by an Executive Mayor with a Mayoral Committee. Working with this elected and political body is The City Manager, whose office is contractual, and reporting to him/her is the Executive Management Team of eleven, all permanent employees of the Municipality, with no political affiliation. The legislative environment for performance management within local government is onerous, based on two central legislative documents from the national treasury. The City develops a five year Integrated Development Plan (IDP), and the Service Delivery and Budget Implementation Plan (SDBIP) gives effect to the IDP and the budget of the City. This expresses the objectives of the City, in quantifiable outcomes to be implemented for the City’s financial year. It includes service delivery targets and performance indicators for each quarter which should be linked to the performance agreements of senior management. It therefore facilitates oversight over financial and non‐financial performance of the Municipality and allows the City Manager to monitor the performance of the Executive Directors, the Mayor/Council to monitor the performance of the City Manager, and the Community to monitor performance of the City Government. Performance to the IDP and SDBIP is legislated in terms of the Balanced Scorecard concept. The legislation provides a solid foundation from a performance management perspective in line with key expectations of a CPM business process. The planning process is legislated, the key planning deliverables are explained and written into legislation, and the performance management methodology is in place and agreed. It should however be noted that rather than coming from the organisation itself, these CPM processes are essentially legislated and are more an act of compliance by the City.

274

Mark Tudor and Mike Hart In 2000 the Cape Town Unicity was formed by merging seven smaller municipalities. This was a challenge from three perspectives: political, business processes, and information technology, as illustrated by the Chief Information Officer (CIO) for the City at that time (SAP 2006, pg.1): “We were faced with a very difficult decision. In my mind, unification meant delivering a single view of the city and a single view of the citizen. The only way to achieve this was through a single ERP system.” The city used Gartner to assist them in selection of an ERP solution and implementation provider. The result was a Strategic Roadmap for SAP ERP implementation, with Accenture as the lead implementation partner in a consortium including four other smaller vendors. The project was at that time the largest local‐government Enterprise Resource Planning system in the world (SAP 2006). The implementation began in early 2002 with initial releases going live in December 2002 and mid 2003 respectively, and later growth into other areas like CRM and CPM as part of a ten year plan. Initially the Unicity was governed by an ANC‐led coalition, but in March 2006 this changed to a coalition led by the Democratic Alliance (nationally the official opposition) with Helen Zille elected as the executive mayor, the only non‐ANC Unicity head. (In 2008, she was rated the world’s best mayor (www.worldmayor.com), was later elected the only non‐ANC provincial leader and is head of the official opposition). The CPM implementation took place during her mayoral term.

6. Findings 6.1 Implementation process The SAPPM explained that although CPM was part of the ten year plan, the major Unicity consolidation and ERP implementation were focussed on initially. The CPMPM pointed out that the CPM project was originally due to follow after CRM implementation, but this order was later reversed. The large data volumes were causing problems in operational and simple management reporting, and they realised that a data warehouse was needed. With this available they could also progress to more complex management reporting, dashboards, and ultimately a full CPM solution. The CPM project started in 2007, led by the CPMPM from Accenture, and overseen by the SAPPM as part of the overall SAP ERP implementation. The CPM team was responsible for requirements analysis and implementation of the SAP CPM module, and comprised business analysts and technical application consultants. They worked alongside the SAP Data Warehouse (DW) team, and a Technical Architect oversaw the solution architecture. The project required two key components:

The Balanced Scorecard reported and tracked the performance of the agreed key strategic indicators in the IBP. Targets had to be set and commentary recorded against KPI’s in accordance with legislation.

A Management Dashboard that would track additional KPI’s not in the Balanced Scorecard. These were not required by the legislation, and their emphasis was on tactical reporting.

The initial objective of the CPMPM was to automate twelve balanced scorecard KPI’s for calculations based on data within the ERP, using the SAP CPM system, essentially delivering a technical solution in support of the legislation. The BA noted however that on unpacking the KPI calculations, it was discovered that “about 10% [of measures required] exists in the current SAP system. The other 90% still needs to be captured”. Pressure from the mayor and the SAPPM also extended the initial scope to include further measures for a Management Dashboard. This required modification of the technical architecture, and use of multiple toolsets, one stable but inflexible, the other new and unproven. Reporting combined both in a single divided portal screen. This enabled access to both KPI components, but when implemented at go‐live time gave major performance problems. The BI Lead said “It is unworkable” and the TA commented “a dashboard used to open within 30 seconds to 50 seconds before the AIX upgrade or the conversion of hardware to 64 bit to now where it takes 10 minutes 15 minutes. We can’t train because it takes too long”. Upgrades to hardware and operating systems had taken place across much of the City’s infrastructure at virtually the same time as CPM go‐live. The CPMPM and some others left the team at official go‐live, having carried out what was in the original specifications, leaving a relatively inexperienced team to solve the performance issues.

275

Mark Tudor and Mike Hart

6.2 Emergent actors Using ANT, seventeen actors emerged during the analysis. These are shown in Table 1, and the role of each in the actor network is then discussed. It can be seen that some are “environmental” and others internal, some much more involved and influential in the CPM implementation than others. Table 1: Actors that emerged during the study Emergent Actors Abbreviation 1 National Government, the Treasury and Legislation 2 Strategic Advisors and the Strategic Roadmap for ERP 3 Political Parties, the Executive Mayor and the Mayoral Committee 4 Organisational Culture 5 SAP ERP Programme Steering Committee 6 SAP Programme Manager for the City SAPPM 7 CPM Implementation Team Project Manager CPMPM 8 Existing Information Infrastructure within the UniCity 9 KPI Owners in the Organisation (mainly Executive Directors) 10 Corporate Performance Management Department 11 KPI Responsible Persons 12 CPM Team Technical Architect TA 13 CPM Team Business Analysts BA 14 CPM Team Technical Consultants 15 SAP CPM Software Solutions 16 SAP Software & Service Provider 17 SAP ERP Programme Implementation Partners

National Government, the Treasury and Legislation These can be combined in a single “black box”. National Government inscribes desired behaviour of local government in the form of legislation, and translates the interest of local government through their definition of the OPP (their legislation), and expects enrolment and mobilization of local government leadership to deliver, based on legislated timelines. The SAPPM mentioned Legislative Compliance as a key driver for the implementation of CPM, and noted that new performance management departments were created in local governments to address this. Strategic Advisors and the Strategic Roadmap for ERP Consultancies (PWC, Accenture and Gartner) provided strategic advice which was translated into a strategic roadmap and budget for ERP. These can be simplified into a “black box” which defined an OPP by which the ERP Programme had to be enrolled. To some extent the SAPPM could translate this roadmap to his own interests – the move of CPM ahead of CRM showed little irreversibility. No interviewee commented on strategic advice affecting CPM (vs ERP) implementation itself. Political Parties, the Executive Mayor and the Mayoral Committee As the elected executive mayor and mayoral committee are political appointments, party politics has a direct impact on the measures within CPM. The BI Lead noted that many of the KPIs were relevant irrespective of the party in power, but the SAPPM pointed out that priorities would differ from one leadership to the next, with varying emphasis on social and service delivery responsibilities. All interviewees stated that the executive mayor positively supported the CPM implementation. She went beyond support to say “I want to have a daily scorecard, I want to have KPIs”. Her requirements went beyond the legislated KPIs to include tactical information of a shorter time horizon, and her management style was much more KPI‐focused than the prevailing organisational culture. Organisational Culture Many of the methodologies of CPM have come from the private sector, and applying them in the public sector without understanding its idiosyncrasies could make adoption difficult. Regarding data capture for the CPM system, the BA commented: “There was a lot of negative feedback like: ‘Why do we have to do this again?

276

Mark Tudor and Mike Hart There are places where this stuff has been done already’. At the end of the day they have to comply. If their KPI is on that scorecard then they have to submit.” There is mandatory adoption, but this cannot be inscribed within the technology. If management have not been through enrolment, will they enforce CPM usage? The SAPPM recalls the statement “If you build it they will come” from the film Field of Dreams, noting “[CPM here] is never going to be urgent, but it will be important”. SAP ERP Programme Steering Committee Although this body nominally controlled the implementation roadmap for ERP as a whole, it appeared as if they were largely guided by the SAPPM, and were not reported by any interviewees as having serious impact on the CPM implementation. SAP Programme Manager for the City (SAPPM) The SAPPM had overall responsibility for both the SAP strategy and implementation of the SAP ERP system. He had a long history within the Municipality outside of IT, and from all interviewees was very well respected by both municipal staff and the implementation partner. The BI Lead, TA and BA saw him as the visionary and driving force behind the CPM system, although he also credited the mayor. The CPMPM noted his business understanding and strong engagement with senior management. He created the OPP in terms of the installation of the SAP CPM solution and translated those in the implementation team to work towards its delivery. In many ways the implementation can be viewed as a long process of enrolment of the business by the SAPPM. CPM Implementation Team Project Manager (CPMPM) Contracted from Accenture to project manage the CPM project, the CPMPM was clearly in a state of mobilisation, aligned to both the interests of her employers and the City. From interviews, her influence appears to have been restricted to technical delivery of the CPM system through her implementation team. Existing Information Infrastructure within the UniCity The CPM solution had to be delivered into the existing technical landscape, including the live SAP ERP applications and the SAP Business Warehouse (BW), which was technically compatible, but being upgraded. The TA noted “I have been through various different upgrades of BW, and this is the most unstable one I have ever seen”. Technical performance problems which later developed could not easily be isolated to the BW upgrade or to the CPM solution. The inscription found within the technical landscape led to an uncomfortable translation of the CPM implementation team, who had limited expertise. KPI Owners in the Organisation (mainly Executive Directors) The SAPPM explained “The ED’s have performance based bonuses. Their performance based contracts include all those metrics”. A system providing more accurate KPI’s might require alignment of performance based salaries. This could create negativity, but according to all interviewees there was no resistance on this aspect, nor any particular KPI requirements from the ED’s, only the strong “pull” from the executive mayor. The BA noted that “The owner of the KPI is supposed to assess the KPI’s. That is also mandatory….After our first phase that wasn’t really done”. It appeared that they hadn’t really bought into the system, and that Intéressement and Enrolment had not taken place with any of this community. Corporate Performance Management Department This relatively new department was scheduled to be the business owner of the implemented CPM solution, responsible for legislative compliance. They were wanting to progress from their current spreadsheet system to find something better, and had explored alternative solutions. However the CPMPM pointed out that the CPM Department resource on the project hadn’t really got “involved with the implementation”. It seemed instead that many key requirements came through the SAPPM from the mayor, who was generally reported as providing the real performance drive within the organisation.

277

Mark Tudor and Mike Hart KPI Responsible Persons They calculate and capture KPI’s into the CPM system that cannot be automated, essentially an administrative function. No interviewee suggested that any specific person had had a significant influence on the project, and they seemed to be at the Intéressement stage. CPM Team Technical Architect (TA) The TA was required to translate business requirements into a technical solution, working with the CPMPM, who project managed the technical team members. Interviews showed him to be mobilised, but having difficulty in resolving performance problems when requirements expanded. CPM Team Business Analysts The TA noted that “those business analysts that understand the business fed in a lot of the non‐scorecard measures”. From the perspective of the SAPPM they were in a state of mobilisation. CPM Team Technical Consultants After enrolment through the project management structures, they were mobilised, but interviewees did not mention them having any impact, positive or negative. SAP CPM Software Solutions SAP software offered various application toolsets and after analysis by the technical team a combination of two solutions was chosen, with the belief that this could be translated by the technical team to inscribe the required CPM solution. This was largely accomplished except for performance issues. SAP Software & Service Provider and 6.2.17 SAP ERP Programme Implementation Partners No significant influence was attributed in interviews to these actors.

7. Discussion This briefly covers some of the issues that emerged from the analysis. Two networks and focal actors All interviewees perceived the SAPPM to be the main focal actor, but all also referred to the strong involvement of the executive mayor. There appear to be two closely associated cooperative networks rather than one complex network covering both technology and business processes. One is aligned to adoption of the business processes, with the mayor as focal actor, and the other aligned to the ICT to support these processes, with the SAPPM as focal actor. The second is essentially a subset of the first. Public sector vs private sector Although implementation in both sectors have much in common, public sector institutions will often have different objectives and criteria from the profit‐oriented private sector, and it may generally be harder to enrol and mobilise their actors. In this case the organisational culture had not generally been performance‐oriented, and change management over time was needed to encourage this. Measures for KPI’s Until a detailed analysis of the correct sources of data and calculation of KPI’s has been done, one may have limited knowledge of how many of these can be easily automated. KPI’s for compliance to national requirements may differ in many respects from those used for shorter term tactical purposes, and fewer measures may reside in organisational systems.

278

Mark Tudor and Mike Hart Different perspectives of actors A limitation of the research was that most actors were from the ICT, not business side, and consequently their focus was mainly on the technology. Strangely the SAPPM and the CPMPM were the only interviewees who did not mention performance problems. This might imply that they saw these as small compared to challenges of organisational adoption. System performance problems The management dashboard request expanded the initial scope, and the focus of the development team had appeared to be on usability more than performance. The ERP and balanced scorecard technologies were established and stable, even with the changed operating system and hardware, but the dashboard was not. Given the limited enthusiasm and enrolment of business users, the slow response and curtailed training threatened to derail the adoption of the system.

8. Conclusion The research aimed to gained insight into some of the issues surrounding CPM system implementation in the public sector, and is acknowledged to have limited generalizability. It does show that:

ICT can play a strong role in supporting and facilitating an effective CPM system, but when it does not deliver as expected, this can have negative consequences for a range of actors.

The adoption of the business processes associated with CPM is closely linked to, if not an antecedent of, the adoption of the CPM ICT.

The performance culture of an organisation can block the adoption of both these.

ANT proved to be a useful way of analysing the overall implementation process.

References Ariyachandra, T.R. and Frolick, M. (2008) “Critical Success Factors in Business Performance Management—Striving for Success”, Information Systems Management, Vol 25, pp113–120. Callon, M. (1986) “Some elements of a sociology of translation: Domestication of the scallops and the fishermen of St Brieuc Bay”. In J. Law (Ed.), Power, action and belief, Routledge Kegan Paul, London. Coetzee, C. (2012) “Province’s changing face”, Cape Times, November 13, 2012. Eckerson, W.W. (2011) Performance Dashboards – Measuring, Monitoring and Managing Your Business, 2nd Ed., John Wiley & Sons, New Jersey. Hardy, C. A. and Williams, S.P. (2008) “E‐government policy and practice: A theoretical and empirical exploration of public e‐procurement”, Government Information Quarterly, Vol 25, pp155–180. Hsbollah, H.M., Simon, A. and Letch, N. (2012) “A Network Analysis of IT Governance Practices: A Case Study of an IT Centralisation Project”, 23rd Australasian Conference on Information Systems, Geelong, 3‐5 Dec 2012. Johns, L., Nkomo, S. and Jooste, B. (2012) “Cape’s residents better off – Census”, Cape Argus, October 31, 2012 Kaplan, R. and Norton, D. (1992) “The Balanced Scorecard – Measures that Drive Performance”, Harvard Business Review, January – February 1992, pp75–85. Klein, K.K., and Myers, M.D. (1999) “A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems”, MIS Quarterly, Vol. 23, No. 1, pp67‐86. Latour, B. (1987) Science in action. How to follow scientists and engineers through society, Harvard Univ. Press, Cambridge, MA. Law, J. (1992) “Notes on the theory of the actor–network: Ordering, strategy and heterogeneity”, Systems Practice, Vol 5, No 4, pp379−393. Neely, A., Richards, H., Mills, J., Platts, K. & Bourne, M. (1997) “Designing performance measures: a structured approach”, International Journal of Operations & Production Management, Vol 17, No 11, pp1131‐1152. Neely, A., Marr, B., Roos, G., Pike, S., and Gupta, O. (2003) “Towards the Third Generation of Performance Measurement”, Controlling, Vol 15, No 3/4, pp129‐135. Pratt, H.J. (1991) “Principles of Effective Performance Management”, MIS Quarterly, Vol. 25, No. 1, pp47‐ 58. Ranerup, A. (2008) “Decision Support Systems for Public Policy Implementation: The Case of Pension Reform”, Social Science Computer Review, Vol 26, No 4, pp428‐445. Ranerup, A. (2012) “The socio‐material pragmatics of e‐governance mobilization”, Government Information Quarterly, Vol 29, pp413‐423. Rayner, N. (2009) “Understanding the Four Stages of Performance Management Evolution”, Gartner Research Publication ID Number G00172786, 25 November 2009, pp1‐7

279

Mark Tudor and Mike Hart SAP (2006) “SAP Customer Success Story, Public Sector – Local Government, City of Cape Town”, Retrieved Nov 23, 2012, from: http://www.sapinvites.co.za/ps/CoCT%20SAP%20reference.pdf Stanforth, C. (2006) “Using Actor‐Network Theory to Analyze E‐Government Implementation in Developing Countries”, Information Technologies and International Development, Vol 3, No 3, pp35–60. Tatnall, A. and Gilding, A. (1999) “Actor‐Network Theory and Information Systems Research”, Proceedings of the 10th Australasian Conference on Information Systems, Wellington, Victoria, pp955‐966. Van Decker, J.E., Chandler, N. and Iervolino, C. (2012) “Magic Quadrant for Corporate Performance Management Suites”, Gartner Research Publication ID Number G00219353, 19 March 2012. Viaene, S. and Willems, J. (2007) “Corporate Performance Management: Beyond Dashboards and Scorecards”, Journal of Performance Management, Vol 20, No 1, pp13‐32. Wade, D. and Recardo, R. (2001) Corporate Performance Management: How to Build a Better Organisation Through Measurement‐driven, Strategic Alignment, Butterworth‐Heinemann Ltd., US. World Economic Forum (2012) “Global Competitiveness Report 2012‐2013”, (http://reports.weforum.org/global‐ competitiveness‐report‐2012‐2013/).

280

Digital Archiving and e‐Discovery: Delivering Evidence in an age of Overload Geert‐Jan van Bussel and Hans Henseler HvA Amsterdam University of Applied Sciences, Amsterdam, The Netherlands g.j.van.bussel@hva.nl j.henseler@hva.nl Abstract: Within e‐Government, trust in electronic stored information (ESI) is a necessity, not only when communicating with citizens, but also for organizational transparency and accountability. In the last decades, most organizations under‐ went substantial reorganization. The integration of structured data in relational databases has improved documentation of business transactions and increased data quality. That integration has improved accountability as well. Almost 90% of the information that organizations manage is unstructured (e.g., e‐mail, documents, multimedia files, etc.). Those files cannot be integrated into a traditional database in an easy way. Like structured data, unstructured ESI in organizations can be denoted as records, when it is meant to be (and used as) evidence for organizational policies, decisions, products, actions and transactions. Stakeholders in e‐Government, like citizens, governments and courts, are making increasing demands for the trustworthiness of this ESI for privacy, evidential and transparency reasons. A theoretical analysis of literature of infor‐ mation, organization and archival science illustrates that for delivering evidence, reconstruction of the past is essential, even in this age of information overload. We want to analyse how Digital Archiving and e‐Discovery contribute to the reali‐ zation of trusted ESI, to the reconstruction of the past and to delivering evidence. Digital Archiving ensures (by implement‐ ing and managing the ‘information value chain’) that: [1] ESI can be trusted, that it meets the necessary three dimensions of information: quality, context and relevance, and that [2] trusted ESI meets the remaining fourth dimension of informa‐ tion: survival, so that it is preserved for as long as is necessary (even indefinitely) to comply to privacy, accountability and transparency regulations. E‐Discovery is any process (or series of processes) in which (trusted) ESI is sought, located, secur‐ ed and searched with the intent of using it as evidence in a civil or criminal legal case. A difference between the two mechanisms is that Digital Archiving is implemented ex ante and e‐Discovery ex post legal proceedings. The combination of both mechanisms ensures that organizations have a documented understanding of [1] the processing of policies, decisions, products, actions and transactions within (inter‐) organizational processes; [2] the way organizations account for those pol‐ icies, decisions, products, actions and transactions within their business processes; and [3] the reconstruction of policies, decisions, products, actions and transactions from business processes over time. This understanding is extremely impor‐ tant for the realization of e‐Government, for which reconstruction of the past is an essential functionality. Both mecha‐ nisms are illustrated with references to practical examples. Keywords: e‐government, accountability, evidence, digital archiving, e‐discovery

1. Neglected challenges of e‐Government In the 1980s and 1990s, in view of a transformation into an information society, in many organizations the in‐ crease of business process efficiency was a predominant item. In the process of environmental adaptation, organizations re‐engineered their business processes and exchanged their standalone applications for more standard, integrated solutions. The subsequent integration of structured data in relational database management systems (RDBMSs) improved the documentation of business processes, increased the quality of structured data and improved accountability. But almost 90% of the electronic stored information (ESI) that organizations manage is unstructured (Scholtes 2009), and cannot easily be integrated into traditional RDBMSs (Van Bussel 2011). Unstructured ESI is stored in text, spreadsheets, emails and/or multimedia files. IDC estimates a 40 % annual growth rate of ESI until 2020 (IDC 2011). From 1986 until 2007, Hilbert and López (2011) concluded, ESI had an average annual growth rate of 23 %. From 2000 until 2007, this growth rate was especially high: in 2000 25 % of all data were stored as ESI; by 2007, that percentage had risen to 94 %. The amount of unstructured ESI is not likely to diminish: within public and private organizations the use of collaborative Web 2.0. technologies to streamline business processes creates ‘big data’ (Jacobs 2009). Social media tools generate large amounts of unstructured ESI, created by knowledge workers who engage in peer‐ to‐peer knowledge sharing across organizational boundaries. Using all different sorts of (social) media, they participate in organization‐wide and inter‐organizational collaboration. The storage, dissemination and processing of unstructured ESI require complex ICT systems. In this dynamic environment, ICT systems present security and durability challenges that pose a major threat for information quality (Bearman 2006). These threats are, together with the exponentially rising amount of

281

Geert‐Jan van Bussel and Hans Henseler ESI, the most neglected challenges of e‐Government. Those challenges threaten the trustworthiness of organ‐ izational records, that ESI that is meant to be (and used as) evidence for policies, decisions, products, actions and transactions. In this paper, we will denote those organizational records as ESI. Stakeholders in e‐ Government, like citizens, governments and courts, are making increasing demands for the trustworthiness of this ESI, mostly for privacy, accountability and transparency reasons. In this context, the audit reports of the Dutch Algemene Rekenkamer (Court of Auditors) present an interesting case. Those reports are very critical of the design and deployment of ICT systems within the Dutch government. The Dutch Court of Auditors identi‐ fies several very serious consequences for information security, data quality, transparancy and privacy. Accord‐ ing to these audit reports, those consequences should be tackled forthwith (AR 2007‐2008, 2011).

2. Research question A theoretical analysis of literature of information, organization and archival science illustrates that for deliver‐ ing evidence, reconstruction of the past is essential (as cases presented in, for instance, Barata and Cain (2001), the Sedona Conference (2004), Allman (2007), Paul and Baron (2007), and Toebak (2010) illustrate). This literature suggests two mechanisms to reconstruct the past and to realize trusted ESI: Digital Archiving (DA) (or, as it is sometimes called, enterprise records management), and e‐Discovery. In this paper, we will analyse how those two mechanisms are contributing to the realization of trusted ESI, to the reconstruction of the past and to delivering evidence.

3. Accountability, evidence and the reconstruction of the past Accountability is the acknowledgement of responsibility for policies, decisions, products, actions and trans‐ actions, and the obligation to report and be answerable for consequences. It is a social relation between an actor and a forum. When the actor is an organization (as it is here), we talk about 'organizational accountabili‐ ty'. The forum is a designated forum (shareholders, citizens, courts, etc.) or a virtual entity ('society', 'the peo‐ ple'). A forum will ask an actor to provide insight in its process effectiveness and the lawfulness or unlaw‐ fulness of its actions and transactions. It will ask an actor to provide evidence for all assertions made in this audit. The forum passes judgement on the actor’s conduct and (if necessary) imposes sanctions (Bovens 2006). Evidence includes everything that is used to determine or demonstrate the truth of an assertion. Evidence must be: authentic, accurate, complete, reliable, and in conformity with legislative rules. It must be tied to a fact to prove something. The evidence collection and analysis procedures must not cast doubt on the evi‐ dence’s authenticity and veracity (Kozushko 2003). Barata and Cain (2001) prove conclusively (based on several practical cases) that accountability without trusted ESI as evidence of the organizational past is not possible. This means that organizations need an ac‐ countability function to safeguard that evidence. To improve accountability, new ICT systems, concepts and methods have to be implemented to identify, structure, organize, process and retain that ESI that is used with‐ in organizational processes, as well as all the ESI that is used to document how actions and transactions have been performed. Ensuring the quality of this ESI is difficult. Redman (2004) states that information quality is a ‘disaster’. ESI is inaccessible, incomplete, inconsistent, irrelevant, untimely, inaccurate, and/or not under‐ standable. Its provenance and contextual environment are unknown (Epler 2006). ESI is generated in a large variety of formats without adding something new (Paul and Baron 2007). It is stored in different forms and places and in various business processes. Organizations tend to facilitate this by allowing users to maintain this business related ESI within personal archives. In addition, ICT creates technological obsolescence. ESI has a longer lifespan than the configurations in which it is created or managed (Boudrez, Dekeyser and Dumortier 2005). Trusted ESI is indispensable as evidence and necessary for making reconstructions of the organizational past. For e‐Government to succeed, ensuring trusted ESI is a necessity, as an Singaporean empirical study suggests (Thompson, Srivastava and Jiang, 2008‐2009).

4. Digital archiving 4.1 ‘Information value chain’ ESI used for informational and productive purposes in business processes, and meant to be or to be used as evidence, can be text, multimedia files, database records, technical drawings, or combinations thereof. This ESI is critical for business process performance (Marchand, Kettinger and Rollins 2001, ch. 6). Besides that, ESI documents how policies, decisions, products, actions and transactions have been performed. ESI’s business im‐ portance means that it needs to be identified and controlled. According to Van Bussel (2012), DA organizes the

282

Geert‐Jan van Bussel and Hans Henseler 'information value chain', the chain that ensures that the informational and evidential 'value' of ESI is utilized in business processes to improve performance. This chain includes all information processes: generation or receipt, identify, capture, storage, processing, distribution, structuring, publication, (re‐)use, appraisal, selec‐ tion, disposal, retention, security, auditing and preservation. DA is oriented [1] on the effects of this chain on business processes, and [2] on the realization of the four dimensions of information. That way, it ensures the provision of trusted ESI for informational and evidential reasons (Van Bussel and Ector 2009).

4.2 The four dimensions of information In this age of organizational chains, inter‐organizational data warehouses, cloud computing, authentic regis‐ trations, and computer mediated exchange, the four dimensions of information allow for the reliable reconstruction of policies, decisions, products, actions and transactions. Those four dimensions are quality, context, relevance and survival (Van Bussel 2012) (Figure 1).

Figure 1: The four dimensions of information For Quality, DA is focused on the quality requirements of [1] (structured or unstructured) ESI and [2] the ‘information value chain’. For ESI, four quality requirements are recognized: integrity (ESI cannot be manipulated), authenticity (ESI presents the required (and original) content and lay‐out), controllability (ESI can be tested on reliability) and historicity (ESI can be reconstructed). Those requirements realize the fixity of ESI. This means that ESI is (or can be reconstructed as) an ‘immutable mobile’ (Latour 1990). Fixity is necessary because ESI is [1] recorded for later consultation and [2] used repeatedly for the reconstruction of past happenings. Fixity enables users to trust ESI and to use it as evidence. Levy (2001, ch. 2), gives several practical examples. The ‘information value chain’ ensures that ESI is correct and complete in spite of all necessary handling. The requirements for this value chain are identical to those for organizational business processes. They are well‐known, namely reliable time of delivery, effectiveness, efficiency, product quality, alignment of needs, product management, and compliance (Van Bussel and Ector 2009). The second dimension, Context, provides meaning (Duranti 1997). Knowledge of the (environment of the) poli‐ cies, decisions, products, actions or transactions for which ESI was generated is necessary for extracting mean‐ ing out of ESI. This knowledge applies to the existing juridical system, the organizational structure, the pro‐ cedures by which ESI is generated and the ESI collection to which it belongs. Groth (2007) suggests three char‐ acteristics of context: [1] it needs accurate documentation, [2] it is in the past, and [3] it is necessary for the tracking and the reconstruction of business processes. Pipek, Wulf and Johri (2012) describe the case of a German steel mill to elaborate on the importance of this dimension of information.

283

Geert‐Jan van Bussel and Hans Henseler Relevance is a fundamental concept in human communication. As Saracevic (2007) explained, ESI is only relevant for users if it fits the context in which it is used, managed and retrieved. For organizations, ESI is only relevant if it fits the organizational objectives of performance and accountability (Van Bussel 2012). A special kind of relevance is appraisal. It is developed within archival theory to determine the ‘value’, relevance, of information over time. It means the complex evaluation of ESI to determine its economic, organizational, financial, fiscal, juridical, legal, societal and historical relevance and to develop a retention schedule for ESI. In such a schedule the periods ot time that ESI should be kept or ‘retained’ (as, for instance, stated in (privacy) law and regulations) are documented, including indefinite retention for ESI of ‘enduring value’ (Cox and Samuels 1988). When the retention period has expired, ESI has no organizational (or legal) relevance any more and should be irreparably deleted (Van Bussel 2012). Disposing of irrelevant ESI saves high costs for retention and accessibility. Besides that, irrelevant ESI makes organizations vulnerable to legal proceedings, for instance in the context of privacy law, fraud or corruption. The much disputed ‘right to be forgotten’ is an essential part of the discussion on the relevance of ESI (Blanchette and Johnson 2002; Rosen 2012). Appraisal ensures that within the ‘information value chain’ (privacy) laws and regulations are respected. Pederson, Routledge and Thurston (1999) have collected 24 case studies on this kind of relevance. The fourth dimension, Survival, concerns the many security and durability challenges, which have to be overcome to realize access, retrieval and preservation over time (Bearman 2006). It stresses the importance of a reliable and durable ICT infrastructure (which is not self‐evident) to enable the continuous and secure storage and manipulation of ESI. This infrastructure assists organizations in seeking competitive advantages, developing the organization learning concept, increasing autonomy and being accountable (Croasdell 2001). The infrastructure needs to safeguard the first three dimensions of information over time, but its features are fragile and continuously influenced by the restructuring of organizations (Boudrez, Dekeyser and Dumortier 2005). First, hard‐ and software configurations are needed for accessing, retrieving and viewing ESI, which means that a solution for technological obsolescence should be available. ESI has a longer lifespan than the configurations in which it is created or managed. Secondly, the large influx of information requires automated archiving and retrieval functionalities. The ICT infrastructure needs to adapt, transform, renew and grow. Thirdly, ESI is of a diverse nature. There is a diversity of object types, operating systems and applications. The handling of this diversity is not self‐evident, while at the same time information can be continuously modified. This endangers the trust in reliable ESI. Fourthly, ESI can only be reliably used, when it can be interpreted by users in its original organizational context. Context and ESI need to be forever linked (Van Bussel 2011). A case‐ based review of this dimension has been offered, among others, by Hockx‐Ju (2006).

4.3 The benefits of digital archiving DA [1] safeguards the 'information value chain', [2] ensures that ESI meets the four dimensions of information, [3] realizes a reliable, effective and efficient use of trusted ESI in business, [4] provides an ICT infrastructure to (indefinitely) store (identified and trusted) ESI and keep it accessible, [5] ensures that (privacy) laws and regulations are respected within the ‘information value chain’, and [6] audits periodically the possibility to reliably reconstruct the past and to use ESI as evidence. In DA, it is emphasized that the failure to realize the dimensions of information is a threat to the possibilities to reliably reconstruct the past. Because of that, the organizational accountability function can not be successful. In managing the ‘information value chain’, DA en‐ sures that ESI can be used as evidence. Its purpose is to reduce the costs of transactions, to enlarge the speed of access to past experiences, to help in decision‐making, to share knowledge and to realize accountability. DA, therefore, is a necessity for realizing organizational accountability. DA is always implemented ex ante the beginning of possible legal proceedings.

5. e‐Discovery E‐Discovery can be seen as a process or a series of processes in which (trusted) ESI is sought, located, secured and searched with the intent of using it as evidence in a civil or criminal legal case (Conrad 2010). It tries to reconstruct the past to learn the informational context of ESI, because only then ESI can be used as evidence. Forensic accountants, lawyers, internal and external auditors, they all resemble archaeologists trying to recon‐ struct events in an almost forgotten past. Similar to archaeology, there may not be much left to investigate, making advanced techniques to reconstruct the past necessary. e‐Discovery is an essential tool for investiga‐ tors to reconstruct past actions and transactions and to support findings in order to prosecute organizations that are in violation with (inter‐)national regulations. For instance, the European Union aims to reduce all forms of corruption at every level. It also aims to prevent fraud by setting up anti‐fraud offices and actively

284

Geert‐Jan van Bussel and Hans Henseler investigates and prosecutes violations of competition regulations. E‐Discovery has to be attuned to (inter‐ )national laws and regulations. This is a challenge, especially as far as privacy law is concerned, for there are worldwide different interpretations of privacy and divergent privacy law traditions (Berman 2010). E‐Discovery itself does not contribute to the realization of trusted ESI in organizations, because it is implemented ex post the beginning of legal proceedings. It uses ESI as it is at the time of those proceedings. The e‐Discovery process (or series of processes) is broadly illustrated by the e‐Discovery Reference Model (EDRM). This reference model was developed to address the lack of standards and guidelines in e‐Discovery identified in the 2003 and 2004 Socha‐Gelbmann Electronic Discovery surveys (Socha and Gelbmann 2004). The EDRM diagram presents an overview of e‐Discovery and symbolizes the transformation of a large volume of general ESI into a small volume of relevant ESI that can be used as evidence in legal procedures. The auto‐ mated part of this process, ‘culling’, is aimed at searching, selecting and filtering ESI without manual review. The model divides the process into six stages: [1] Information Management; [2] Identification; [3] Preserva‐ tion and Collection; [4] Processing, Review and Analysis; [5] Production; and [6] Presentation (figure 2). Those stages are described in detail on the EDRM Website (EDRM 2006). The first stage, Information Manage‐ ment, is related to DA. It symbolizes the discovery of all possible sources of trusted ESI in an organization. They may be business systems, such as fileservers, email servers, ERP systems, (partly implemented) enterprise document and records management systems, (forgotten) backup tapes or personal archives. e‐Discovery spe‐ cialists like to collect data from personal archives and combine this with data recovered from old data backups that were maintained for disaster recovery purposes. They can disclose ESI that has been deleted on purpose (outside of a ‘defensible disposal’) (Henseler 2010a), as has been demonstrated in the Enron and Lehman Brothers cases (Keena 2002; Valukas 2010). This stage is difficult to implement within e‐Discovery for it cannot be controlled. It requires knowledge about how and where trusted ESI has been stored. Most information management efforts within e‐Discovery are crippled by insufficient collaboration among key stakeholders: [1] the business users who need ESI to operate an organization, [2] the IT departments who must implement the mechanics of information management, and [3] the legal, risk, and regulatory departments who understand the duty to preserve ESI.

Figure 2: The EDRM mdel Information overload makes finding and reviewing ESI expensive and increases the risk of failure. The problem is amplified as ESI is more easily generated in a large variety of formats without adding new information, as Paul and Baron (2007) have illustrated. The solution is to collect as much ESI as can be found. e‐Discovery tools use a variety of culling techniques for removing duplicates, filtering based on file extension, date time and / or keywords in order to reduce the volume of ESI. Keyword filtering relies on a set of keywords that is designed based on the context of the investigation, for instance names of persons, projects, places, companies, etc. The retrieval of information from large amounts of data is an important part of this process. This step involves a lot of of manual work done by lawyers searching for evidence in e‐mail which may include millions of documents (Sedona Conference 2004). This situation is rapidly changing. Experiments have shown that finding relevant ESI using keyword search is far from perfect (Krause 2009) and that it is expensive and very inefficient (Oard, et al. 2010). Experiments have shown that technology‐assisted review can outperform manual review (Grossman and Cormack 2011). The need for better search tools and methods is reflected in the growth of the e‐Discovery market (Gartner 2009) and in the growing research interest (Ashley and Engers 2011). New techniques are

285

Geert‐Jan van Bussel and Hans Henseler being introduced that allow a faster review of ESI by using tools such as conceptual search (Chaplin 2008), de‐ tection of near duplicates and visual analysis (Görg, Stasko and Liu 2008), predictive coding, that is used to automatically identify ESI using statistical pattern recognition (Peck 2011) and social network analysis (Henseler 2010b). As a consequence of these developments e‐Discovery experts are getting better in recovering ESI that has never been archived and that was assumed to be deleted. Organizations should realize that it is almost impos‐ sible to delete information that is stored in many different forms and places in various business processes without implementing DA. The introduction of cloud‐based storage is likely to worsen this situation because ESI is stored redundantly and beyond the control of an organization, as the Los Angeles case demonstrates (Murphy 2011). This creates a liability because ESI might be discovered while it should have been deleted ac‐ cording to internal accountability rules. e‐Discovery demands the four dimensions of information. It demands DA to retain what should be remembered and to forget what should be forgotten.

6. Conclusion In this paper, our objective was to analyse how DA and e‐Discovery contribute to the realization of trusted ESI, to the reconstruction of the past and to delivering evidence in a time of information overload. We describe the core objectives of DA and e‐Discovery. DA’s core objectives are improving business process performance and realizing accountability. DA assists organizations in using the ‘information value chain’ and the four dimensions of information to reconstruct past policies, decisions, products, actions and transactions. It enables the use of ESI as evidence and enables business processes to improve performance. DA is a necessity for realizing organizational accountability and delivering evidence. It is implemented ex ante possible legal proceedings. The objective of e‐Discovery is delivering evidence. To find evidence, it uses complex information retrieval and culling technologies. It safeguards its evidence collection and analysis procedures, to ensure the authenticity and veracity of the evidence. It is implemented ex post the beginning of legal proceedings. DA and e‐Discovery are two sides of the same coin. The first is implemented to assist an organization in improving business process performance and in preparing for litigation. The latter is used when an or‐ ganization is asked to be accountable and is subject of litigation. DA identifies trusted ESI and manages it with the ‘information value chain’. ESI is identified, structured, stored, preserved, appraised, deduplicated and irreparably deleted (according to legal rules and regulations). DA will not be able to eliminate the explosion of ESI, but it will reduce additional overload caused by storing the same ESI in different formats. It will ensure that ESI is deleted, when necessary. e‐Discovery will use its information retrieval and culling technologies to locate the needed evidence in the enormous sources of trusted ESI. By deleting ESI the cost of e‐Discovery will be (much) lower, while simultaneously reducing the potential for liability. In combination both mechanisms ensure that organizations have a documented understanding of the past. This is important for e‐Government, for reconstruction of the past is an essential functionality for delivering evidence, and for improving organiza‐ tional transparency and accountability.

References Allman, T.I. (2007) “Managing preservation obligations after the 2006 federal e‐discovery amendments”, Richmond Journal of Law & Technology, Vol 13, No. 3, pp 1‐35. AR Algemene Rekenkamer (2007‐2008) Lessen uit ICT‐projecten bij de Overheid, Deel A, Deel B, Algemene Rekenkamer, ‘s‐ Gravenhage. AR Algemene Rekenkamer (2011) ICT bij de Politie. Rapport 2010, Algemene Rekenkamer, ‘s‐Gravenhage. Ashley, K.D. and Engers, T.M. van (eds.) (2011) The 13th International Conference on Artificial Intelligence and Law. Pro‐ ceedings of the Conference June 6‐10, 201, ACM, Pittsburgh. Barata, K., and Cain, P. (2001) “Information, not technology, is essential to accountability: electronic records and public‐ sector financial management”, The Information Society, Vol 17, pp 247‐258. Bearman, D. (2006) “Moments of risk. Identifying threats to electronic records”, Archivaria, Fall, Vol 62, pp 15‐46. Berman,S. (2010) “Cross‐border challenges for e‐Discovery”, Business Law International, Vol 2, No. 2, pp. 123‐131. Blanchette, J.F. and Johnson, D.G. (2002) “Data Retention and the Panoptic Society. The social benefits of forgetfulness”, The Information Society, Vol 18, pp. 33‐45. Boudrez, F., Dekeyser, H., and Dumortier, J. (2005) Digital Archiving. The new challenge, IRIS, Mont Saint Guibert. Bovens, M. (2006) “Analysing and assessing public accountability. A conceptual framework”, [online], http://www.connex‐ network.org/eurogov/.../egp‐connex‐C‐06‐01.pdf.

286

Geert‐Jan van Bussel and Hans Henseler Chaplin, D.T. (2008) “Conceptual Search. ESI, Litigation and the issue of Language”, Paper read at DESI II. Second Interna‐ tional Workshop on Supporting Search and Sensemaking for Electronically Stored Information in Discovery Proceed‐ ings, London, June. Conrad, J. (2010) “E‐Discovery revisited: the need for artificial intelligence beyond information retrieval”, Artificial Intelligence and Law, Vol 18, No. 4, pp 321‐345. Cox, R.J. and Samuels, H.W. (1988) “The archivist’s first responsibility. A research agenda to improve the identification and retention of records of enduring value”, The American Archivist, Winter/Spring, Vol 51, pp. 28‐42. Croasdell, D.T. (2001) “IT's role in Organizational Memory and learning”, Information Systems Management, Vol 18, No. 1, pp 8‐11. Duranti, L. (1997) “The preservation of the integrity of electronic records”, Proceedings of the DLM‐Forum on electronic records. Brussels, 18‐20 December 1996, EU, Luxemburg, pp 60‐65. EDRM (2006) “EDRM E‐Discovery Reference Model”, [online], http://www.edrm.net. Epler, M.J. (2006) Managing information quality. Increasing the value of information in knowledge‐intensive products and processes, Springer, Berlin, 3rd edition. Gartner (2009) “MarketScope for e‐Discovery Software Product Vendors report 2009”, [online], http://www.gartner.com/DisplayDocument?id=1262421. Görg, C., Stasko, J., and Liu, Z. (2008) “Jigsaw. Supporting investigative analysis through interactive visualization”, Informa‐ tion Visualization, Vol 7, No. 2, pp 118‐132. Grossman, M.R. and Cormack, G.V. (2011) “Technology‐assisted review in E‐Discovery can be more effective and more efficient than exhaustive manual review”, Richmond Journal of Law and Technology, Vol 17, No. 3, Article 11. Groth, P.T. (2007) The origin of data. Enabling the determination of provenance in multi‐institutional scientific systems through the documentation of processes, Southampton. Henseler, J. (2010a) E‐Discovery. Op zoek naar de digitale waarheid, Amsterdam University Press, Amsterdam. Henseler, J. (2010b) “Network‐based filtering for large email collections in E‐Discovery”, International Journal of Artificial Intelligence and Law, Vol 18, No. 4, pp. 413‐430. Hilbert, M. and López, P. (2011) “The world’s technological capacity to store, communicate, and compute information”, Science, Vol 332, No. 6025, pp. 60‐65. Hockx‐Yu, H. (2006) "Digital Preservation in the Context of Institutional Repositories", Program: Electronic Library & Infor‐ mation Systems, Vol 40, No. 3 pp. 232‐243. IDC (2011) “Extracting value from chaos”, [online], http://www.wmc.com/leadership/programs/digital‐universe.htm. Jacobs, A. (2009) “The pathologies of Big Data”, Communications of the ACM, Vol 52, No. 8, pp 36‐44. Keena, J.R. (2002) “E‐Discovery. Unearthing Documents Byte by Byte”, [online], http://www.mnbar.org/benchandbar/2002/mar02/e‐Discovery.htm. Kozushko, H. (2003) “Digital Evidence”, [online], http://infohost.nmt.edu/~sfs/Students/HarleyKozushko/Papers/DigitalEvidencePaper.pdf. Krause, J. (2009) “In search of the perfect search”, [online], http://www.abajournal.com/magazine/in_search_of_the_perfect_search. Latour, B. (1990) “Postmodern? No, simply amodern! Steps towards an anthropology of science”, Studies In History and Philosophy of Science, Vol 21, No. 1, pp 145‐171. Levy, D.M. (2001) Scrolling forward. Making sense of documents in the digital age. Arcade Publishing, New York. Marchand, D.A., Kettinger, W.J. and Rollins, J.D. (2001) Information Orientation: The Link to Business Performance, Oxford University Press, New York (NY). Murphy, B. (2011) “e‐Discovery in the cloud not as simple as you think”, [online], http://www.forbes.com/sites/jasonvelasco/2011/11/29/e‐discovery‐in‐the‐cloud‐not‐as‐simple‐as‐you‐think/. Oard, D.W., Baron, J.R., Hedin, B., Lewis, D.D., Tomlinson, S. (2010) “Evaluation of information retrieval for e‐Discovery”, Journal of Artificial Intelligence and Law, Vol 18, No. 4, pp 347‐386. Paul, G. and Baron, J. (2007) “Information Inflation. Can the legal system adapt ?”, Richmond Journal of Law & Technology Vol 13, No. 3, pp 1‐41. Peck, Judge A. (2011) “Search, Forward: Will manual document review and keyword searches be replaced by computer‐ assisted coding?”, Law Technology News, October 2011. Pederson, A., Routledge, D. and Thurston, A. (1999) Managing Public Sector Records: Case Studies, Vol. 1, Vol. 2, Interna‐ tional Records Management Trust, London. Pipek, V., Wulf, V., and Johri, A. (2012) “Bridging artifacts and actors: expertise sharing in organizational ecosystems”, Computer Supported Cooperative Work, Vol 21, No. 2‐3, pp. 261‐282. Redman, T.C. (2004) “Data: an unfolding quality disaster”, DM Review, Vol 8, No. 8, pp 22‐23, 57. Saracevic, T. (2007) “Relevance: a review of the literature and a framework for thinking on the notion in information science. II.”, Journal of the American Society for Information Science and Technology, Vol 58, No. 3, pp. 755‐776. Rosen, J. (2012). “The Right to be Forgotten”, [online], http://www.stanfordlawreview.org/online/privacy‐paradox/right‐ to‐be‐forgotten. Scholtes, J. C. (2009) Text mining: de volgende stap in zoektechnologie, Maastricht University, Maastricht. Sedona Conference (2004) “The Sedona Principles. Best practices, recommendations & principles for addressing electronic document production”, [online], http://www.thesedonaconference.org/dltForm?did=SedonaPrinciples200401.pdf.

287

Geert‐Jan van Bussel and Hans Henseler Socha, G., and Gelbmann, T. (2004) “Socha Gelbmann Electronic Discovery Survey”, [online], http://www.sochaconsulting.com/Publications/LawTechnologyNews Article 8.04.pdf. Thompson, S.H.T., Srivastava, S.C., and Jiang, L. (2008‐2009) “Trust and Electronic Government Success: An Empirical Study”, Journal of Management Information Systems, Vol 25, No. 3, pp. 99‐132. Toebak, P.M. (2010) Records Management. Gestaltung und Umsetzung, Hier + Jetzt, Baden. Valukas, A.R. (2010) “Report of Anton R. Valukas in the Lehman Brothers Holdings, Inc.”, [online], http://www.scribd.com/doc/28228424/Lehman‐Brothers‐Examiner‐s‐Report‐Vol‐1. Van Bussel, G.J. (2011) “Accountability and the reconstruction of the past”, Castelnova, W. and Ferrari, E. (eds.), Proceed‐ ings of the 5th European Conference of Information Management and Evaluation (ECIME 2011), Academic Publishing Limited, Como, pp 483‐490. Van Bussel, G.J. (2012) ‘Archiving should be just like an AppleTM’ en acht andere, nuttige (?) stellingen, Amsterdam University Press, Amsterdam. Van Bussel, G.J. and Ector, F.F.M. (2009) Op zoek naar de herinnering. Verantwoordingssystemen, content‐intensieve organisaties en performance, VBDS, Helmond.

288

Customer Centricity: A Comparison of Organisational Systems With Social Media Applications Grafton Whyte1 and Adolf Lamprecht2 1 Namibia UST / University of the Western Cape, South Africa 2 University of the Western Cape, South Africa drgwhyte@live.com Abstract: Purpose ‐ A survey of the computing press over the last 30‐40 years would provide evidence that organisational in‐house IS departments regularly fail to deliver information systems on time, to budget or most importantly of all, to specifications demanded of their users. Yet since the popularisation of social networking software application usage has become a normal daily activity for the majority of the people on the planet. Facebook to name just one application has th recently reported having over 1 billion users (1/6 of the planet’s population!). So this research investigates why organisational applications lag behind in providing users with customer centric solutions when compared with social networking applications. Design/Methodology/Approach ‐ A review of the recent literature on the success of systems development projects were used to compile a 28 item survey instrument, which was grouped into 5 themes, this instrument was sent out to 101 respondents in 5 companies in Cape Town. Findings ‐ The results of the survey were unexpected in that overall the majority of participants actually ‘liked’ their organisational applications. The research also found varied results when the data was segmented by age groups with the ‘older’ population of the research sample was scoring organisational software higher social media applications. Research limitations/implications ‐ The scope of this research did not include further research into the possibility that either the older population ‘liked’ the social networking applications less, and therefore scored organisational software comparatively higher. Practical implications ‐ The knowledge that the organisation applications are actually more customer centric than is commonly acknowledge suggests that organisational IT departments are on the right track when it comes to delivering the expected results. Originality/value ‐ Much of the recent literature available do suggest that organisational IS departments are improving their delivery quality and raising client satisfaction, but there was no indication in any of the literature reviewed, that the organisational software had overtaken the social networking applications in providing satisfactory solutions for segments of their customers. Keywords: customer centricity, software development, social networking, organisational applications

1. Introduction In the information technology (IT) environment in South Africa many applications are developed which are technically sound but do not meet the requirements of the customer. Somehow between the user specifications and the development thereof things go astray. Not much attention is given to these discrepancies during the development process because time is of the essence. Customers are on many occasions forced to sign off documentation because they are promised that everything will be sorted out at a later stage. At the end of the project these differences are still present and corporate politics are such, that these requirements are moved into the post implementation process. Customers are then referred to the helpdesk to log calls via their normal incident logging process. This leads to a frustrated customer and creates a very bad relationship with IT and their customers. Conversely, the sheer volume of users on social media would seem to support the notion that social networking applications meet the needs of their users to a high extent with apparently few hick‐ups and disappointments. It would appear that the developers of social media applications have hit the right approach in meeting their users’ needs. This research therefore aimed to identify: What IT needed to possibly learn from the social networking applications to produce more customer centric organisational software? Apart from the main research question, the following research sub questions also had to be answered: What is meant by customer centricity?

Does involving the customer in the development process guarantee a more satisfied customer?

Why is it that some social applications like Facebook are more acceptable and popular amongst users?

289

Grafton Whyte and Adolf Lamprecht

Can we learn something from them and use that knowledge in the software development of large scale systems?

Are we any closer in providing the end user what they want?

The goal of the research was to re‐define what client centric software development means.

2. Literature review At the outset of the literature review on customer centric software development, the question should be asked, what is meant by customer centric software solutions? If the software provider is part of an IT department, a key question is who must the software satisfy? Should it be the internal client being users in other parts of the organisation like HR, operations or claims. Or the external client, the ultimate user or consumer of the service or product the software is being used to deliver. Ultimately, it is external client who pays for the system. For the developer of organisational systems it could be both parties. According to Bannermann & Thorogood (2012), there are five domains of success: process, project management, product, business and strategic. The success on one domain is independent to the success in the other domains. Project success can refer to being on time, within budget, to specification completed, success of the product delivered or success in achieving the business objectives of the project. Various stakeholders have different perceptions of success and the time since completion. They see learning and development as a further measure of success. Their view is that completing a project ‘to scope’ may not be good enough for the client. It is suggested there must be information quality, system quality, service quality, intention to use, actual use, user satisfaction, user satisfaction and net benefits. Their conclusion is a project will most likely fail if there is not sufficient end user input during the development life cycle. Kerzner (2003) has two additional components for success, being completion without disturbing the main business of the company and implementation without changing corporate culture of the company. Corporates expend thousands of man hours in forming a corporate culture and in many instances do not want this to change as a result of a new system implementation. Orngreen, Nielsen, and Levinsen (2008) look at various approaches for instance the human computer interface approach. When the developing team develops a solution for the end user, in their case, for e‐learning, it is entirely focused on the end user and does not take the client who has to set‐up and maintain the system into account. Again the question is, who is the system developed for if there is in fact more than one possible user group? Ktata and Levesque (2010) and Pan (2006) feel that agile developing teams still focus in general on how to deliver efficiently, rather than to deliver what the client wants. Batra (2009) refers to the Agile Manifesto which include controversial statements like ‘Welcome changing requirements, even late in development’. This echoes customer centricity in the true sense. If the customer realizes at any stage that their requirements have changed due to legislation, new requirements or even as a result of inadequate analysis, then these changes should be included into the system. Göransson, Gulliksen and Boivie (2004), feel strongly that user‐centred system design must be closely integrated in the software development process. They, as does Orngreen et.al. (2008), feel that the end user must be involved throughout the development process of the system. Chamberlain, Sharp, and Maiden (2006) did research on the integration possibilities between agile development and user‐centred design as these two fields have much in common. It seems to them logical that the two disciplines must be compatible and as long as some key principles are adhered to. Silva da Silva, et al., (2011) and Fox, et al., (2008) share the same view. Abras, et al., (2004), start with a practical simple problem of you as a customer buying a VCR or today a DVD player and not being able to use the functionality due to poor design, poor documentation etc. They feel that the user must be involved during the design phase. Holzinger, et al.(2005) explains how extreme programming evolved to usability engineering and finally to extreme usability. He feels strongly that the customer should be integrated in the development process and that extreme usability has a proper place in its methodology for the customers’ involvement. If the customer is involved, he gets a feeling of what is achievable during the project and what needs to be phased out to a later

290

Grafton Whyte and Adolf Lamprecht stage. Holziger sees dangers in this process that the customer can get too deeply involved in analysis which may hinder the project but that there are far more benefits than disadvantages. The process involves planning games, mock ups and user story telling. In a follow‐up study Hussain, et.Al., (2009) suggests that agile development still lacks the usability awareness which they would like to see. They obtained their findings from an online survey and refer to various human centred integration (HCI) techniques which fit in well with agile development. Turner and Jones (2007) emphasise the need for closer end‐user collaboration if the final product is to be customer centric. They suggest a move away from technology based development to user based development. Memmel, et. al. (2007) are supporters of the fusion of Extreme Programming and Agile Modelling to create a common engineering approach to becoming more customer centric. They have developed a CRUISER Lifecycle which is similar to Holzinger’s extreme usability. Chang, et. al. (2008) have taken an E‐Restaurant as a case study on how the systems can be customer centric. In order to help the staff identify the customer, they have various statistics of likes and dislikes of the customer, past experiences, tables have orders waiting, orders taken etc and where the customer can interact on a system level. They emphasise that a high quality system should be customer centred in order to be successful. Everything is proactive. This type of system has also been implemented in postal logistic systems and telemedicine. The authors also refer to the use of radio frequency identification (RFID) on a campus in Taiwan to provide guidance for blind students. These are typical examples of highly successful user‐centric software developments. During the research it was found that the term user centred development and all the substitute names are not new at all to the industry. Lieberman et. al. (2006) refer for instance to a research paper by Stiemerling of 1997 and earlier even as far back as 1980, documents which focused on the customer or end‐user during the development life‐cycle rather than on what the technology can offer. It is difficult to really measure customer centric software development and that this is rather a shifting target. As more research is done on the subject and as more models are being developed, then customer centricity will improve only for the customer to set new targets. From the literature review it was determined that most authors feel that user involvement during the development phase was very important for the success of the application as was suggested by for instance Turner and Jones (2007). This became one of the themes during the research. Usability of the software was the emphasis from Holzinger, et al.(2005) and efficiency was another theme used by more than one author which influenced our thinking.

3. Research design Now that it is evident what has been researched on the topic of customer centric software development and what is not, one can focus on researching new areas, rather than duplicating work. The purpose of the methodology is to determine what route to follow in order to obtain answers in a scientific manner for our research questions. Welman, Kruger and Mitchell (2005) state, that quantitative research does not investigate the processes, but focuses on the analysis and measurement of causal relationships between variables. Hageman (2008) also states that with surveys, there is no manipulation; the researcher merely measures the variables of interest. The following table illustrates the main characteristics of the three different approaches. Table 1: Qualitative, quantitative and mixed methods procedures Quantitative Research Methods Predetermined Instrument based questions Performance data, attitude data, observational data, and census data Statistical analysis

Qualitative Research Methods Emerging methods Open‐ended questions Interview, observation, document & audio visual data Text and image analysis

Table adapted from (Creswell 2003)

291

Mixed Methods Research Methods Predetermined & emerging methods Open‐ and closed‐ended questions Multiple forms of data drawing on all possibilities Statistical and text analysis

Grafton Whyte and Adolf Lamprecht A quantitative approach was used for this research as the factors that contribute to the successful development of systems have been well researched and identified in the previous literature review. This study needed to understand the factors that could distinguish between internal corporate systems and social media systems. Hence, on‐line questionnaire on was set‐up on Google containing 28 items. These items were roughly split in two, one half were questions on the respondent’s experience with systems developed for the organisational platform and the other half of the questions on social media. The two halves of the questionnaire were almost identical setting up a comparative study. The target audience was selected to be representative of the complete system development life cycle (SDLC) of software development but with the emphasis leaning towards the end‐user. A total sample size of 101 participants were selected (n = 101). According to (Welman, et. Al. 2011), the sampling error of a sample size of 101 within a combined company size of 1500 employees is for instance approximately 2. The larger the sample size, the smaller the standard error. The test sample of 6.8% of the total population is well within the standards as used by researchers. Data collection was done via email notification with a link (https://docs.google.com/spreadsheet/viewform?formkey=dGVVSWI4ZVVUUXU4S2g4cmtiWS1kUUE6MA) to the on‐line questionnaire. All respondents were emailed, the purpose was explained, the end‐date was communicated as well as the confidentiality of their answers. The method of data collection was probability. For this purpose, a random sampling table was used as the sample size is relatively small. The questionnaire was run over 7 days and answered using the 5‐point Likert scale where: 5 = strongly agree 4= agree 3 = neutral 2 = disagree 1 = strongly disagree.

4. Research results This section introduces you to the data which was collected on‐line as well as the analyses of the data. The correlation coefficient was used to determine whether there was a significant relationship or association between two variables, being the Pearson product moment correlation coefficient (r).

4.1 Demographics Table 2 gives a breakdown of the sample demographics. The focus was not on differences between the gender groups, age groups or number of employees at the companies where the research was conducted, but merely to illustrate the composition of the sample group. Table 2: Sample demographics Gender

Age (yrs)

Company size (employees)

Male Female Totals

60 (59%) 42 (41%) 102

18‐25 26‐35 36‐45 46‐55 >55

14 (14%) 42 (41%) 26 (25%) 18 (18%) 2 (2%) 102

1‐50 51‐100 101‐200 201‐300 >300

10 (10%) 8 (8%) 5 (5%) 12 (12%) 67 (66%) 102

4.2 Thematic analysis The thematic approach was used to try and make sense out of and evaluate the research data. This was to identify and group a list of questions into common themes to derive a more meaningful view of the data. These themes were categories identified in the review of the literature. The themes identified were: Community

User involvement in the software development process

Usability of the software

292

Grafton Whyte and Adolf Lamprecht

Efficiency

Frequency of use

These themes will be introduced in this chapter and explained in more detail in the following chapter in terms of the questions grouped under each theme, the reason for the grouping and the results. 4.2.1 Community Social network platform Q23. I prefer non organisational applications (e.g. Facebook & Twitter) above costume organisational applications. Q24. I frequently use social networking sites e.g. Facebook, Twitter, Mixit. Q25. I use social networking sites to share my personal life, achievements, activities, and content with others. Q26. I primarily use social networking sites e.g. Facebook, MySpace to connect and communicate with others. The above 4 questions were analysed and the correlation coefficient between Q24 ‐ Q25 and Q25 –Q26 was high with a combined average of almost 0.9 as can be seen in table 2. Table 2: Correlation Question

Question

Correlation coefficient

Q23 Q24 Q25

Q24 Q25 Q26

0.4401 0.8510 0.9233

All correlations were significant between 95% and 99% levels of confidence. 4.2.2 User involvement This refers to the extent to which the users were involved during the development of applications. The following 3 questions were grouped together for this theme: Organisational platform: Q10. I provided my input in the development of the application that l am using. Q11. I can suggest improvements and enhancements to the application. Social network platform: Q20. I can suggest improvements and enhancements to the application. There were no significant correlations between the items and overall satisfaction with either organisational or social media applications. 4.2.3 Usability This refers to the look‐and‐feel, functionalities and the amount of training required prior to effective use of the software. Q27 asked the participants directly whether they preferred social networking sites above organisational software. Q23 from the Community theme was a similar type of question and the responses were also similar as can be seen from the high correlation coefficient of 0.985. This would serve as proof that the participants voted consistently the same for the same type of questions on both platforms as could be seen from table 3 where there is a high correlation between the same type of questions asked on both platforms as can be seen from table 3. The following 12 questions were grouped together for this theme: Organisational platform:

293

Grafton Whyte and Adolf Lamprecht Q5. I like the look and feel of the application. Q6. I prefer an organisational application that has a graphic user interface (GUI). Q7. The application has many functions and features. Q8. The application meets my requirements and needs. Q12. I am required to undergo some training before using the application. Q14. Overall, l like the Organisational Application that l am using Social networking platform: Q16. I like the look and feel of the application. Q17. I prefer an application that has a graphic user interface (GUI). Q18. The application has many functions and features. Q19. The application meets my requirements and needs. Q21. I received some formal or informal training before using this application. Q27. I prefer using social networking sites over in‐house developed organisational applications. Q28. Overall, l like the Non‐Organisational Application that l am using. Table 3: Correlation coefficient between questions Question

Correlation coefficient

Correlation between Q23&Q27 Correlation between Q6&Q17 Correlation between Q5&Q16 Correlation between Q7&Q18 Correlation between Q8&Q19 Correlation between Q12&Q21 Correlation between Q14&Q28

0.985 0.942 0.966 0.989 0.960 ‐0.457 0.934

4.2.4 Efficiency Efficiency was identified to determine how efficient the participants thought the organisational applications as well as the social networking applications were. For this purpose, the following 3 questions form part of this theme: Organisational platform: Q9. The application enables me to perform my job functions efficiently. Q13. I look forward to future upgrades of this application. Social networking platform: Q22. I look forward to future upgrades of this application. 4.2.5 Frequency of use This theme was chosen to identify how often the software application was used. The following 2 questions were grouped together for this theme: Organisational platform: Q4. I frequently use the application Social networking platform: Q15. I frequently use the application

294

Grafton Whyte and Adolf Lamprecht From Figure 1 below, it becomes more evident that more users used the organisational software on a regular basis (Q4) than the social networking software although both categories of software were used frequently by most users. 45 40

Participants

35 30 25 20 15 10 5 0 4

Strongly agree

15 4 39

15 24

Agree

Neutral

Disagree

Strongly disagree

Figure 1: Frequency of use

5. Discussion This section discusses the results of this study, some of which were unexpected makes certain recommendations for practice and further research. As can be seen in the previous section, 5 themes were selected from the literature review and the 25 of the research questions were grouped together in the 5 themes:

5.1 Community The result of question 23 was unexpected in the sense that 42% of the respondents were neutral and did not prefer social media applications (e.g. Facebook, MySpace etc) above custom organisational applications. The correlation between Q23 –Q24 was therefore low at only 0.4401. This was a clear indication that the majority of the respondents did not prefer social networking sites above organisational applications. This was not a clear indication if the users were using social networking sites on a regular basis to interact with each other and to share information or whether they preferred social networking sites above organisational applications. This uncertainty was mainly because a substantial number of users responses were neutral to these questions, especially Q23, where 42% gave a neutral response. For Q24, Q25 and Q26, the positive reaction was however on average above 50%, and if you remove the neutral responses, 65%. Therefore it was safe to state that the majority of participants use social networking sites frequently, but do not necessarily ‘like’ them more than organisational applications.

5.2 User involvement This refers to the degree of input which the user was able to give during the development life cycle of the software, on the organisational as well as the social network platform. It was possible to answer the research question partially from Q10 and Q11 that although only 38% of the respondents said that they were involved (strongly agree and agree) in the development of the organisational applications and another 21% of responses were neutral, 70% of these respondents said that they had ways to suggest improvements to these systems. Q20 referred to social network sites where 28% of respondents said that they could make suggestions to improve these applications, compared to 70% on organisational applications. 41% were neutral and 32% could

295

Grafton Whyte and Adolf Lamprecht not make suggestions to improve these applications. It was surprising that such a low percentage felt that they could make suggestions to improve the applications. Just over a quarter of the respondents believed that they had any influence on the future of their favourite social networking applications. In order to be able to form some sound conclusions, one would like to know what correlation between user involvement and satisfied users, i.e. the usability theme and the efficiency theme will be. This could determine if the percentage of users involved during the development phase and number of users who could provide input to improvements of company systems, had a positive impact on the usability and efficiency of the software.

5.3 Usability A surprising result for Q28 was the fact that although only 11% of the users did not like the social networking site, only 59% of the participants actually liked the site (see Graph 5). On the organisational side, 75% of the participants liked the application. One would have expected this percentage to be significantly higher on the social networking side and higher than on the organisational platform. Further research could be directed towards this question which may cause new questions like why the participants did not change to other social networking sites and what the reasons were for not particularly liking these sites. As was expected and can be seen in table 3 in the previous section, the correlation coefficient between Q12 and Q21 was negative at ‐0.457. While users usually require at least some form of training for organisational applications (60% of users), only 8% of users of social networking sites received some form of training. A trend was picked up on Table 3, by using the correlation coefficient between the same question on the organisational platform and social networking platform. There is a very strong relationship between Q5 and Q16. Participants who liked the look and feel of the application on the organisational platform, also did so on social networking platform. This indeed signalled a strong ‘Yes’ vote for the question on how far social networking sites are removed from organisational applications in term of user friendliness, functionality and meeting the users requirements. This brought the next and most important question, Q8 and Q19. From Figure 2 below, and calculations done, it is noted that 78% of users feel that the organisation applications met their needs and 71% felt that the social networking application had met their requirements.

Figure 2: Usability

5.4 Efficiency Only 5% of the respondents indicated on Q9 that the organisational application did not enable them to perform their job function effectively. These 5% could be generally negative employees whom you will never be able to satisfy or they could all be in one department where there is a serious problem? 72% of the

296

Grafton Whyte and Adolf Lamprecht respondents did indicate that they could perform their job functions effectively with the software, as can be seen from graph 5. 83% of the respondents furthermore indicated that they were looking forward for future enhancements and upgrades. This was far higher than the 58% of respondents who were only looking forward to upgrades of social networking sites. Upgrades in the organisational environment could be to make the system more effective and stream lined and it could also include upgrading for new products or services, changes to legislation and user‐friendliness.

5.5 Frequency of use 80% of the participants said that they used the organisational application on a regular basis. Without drawing unfounded conclusions, this would mean that such a participant has more than one application which they use during their daily working hours. This was in line with many production departments where the staff had more applications which they used, one being the main application, then there is a workflow application, another application for storing documents online, etc. 65% of the respondents indicated that they use the social networking sites on a regular basis. It would be interesting to see if there is an age correlation between the use of social networking sites and the age of the participants. If you however looked at the age distribution of the participants as can be seen in table 4, you would see that only 2% of the participants were above 55 years of age. Over 80% of the respondents were between the ages of 26 and 45 as can be seen from table 4. An age sensitive analysis was performed to determine if there was a significant difference for participants who frequently used social networking sites and came to the following conclusion as can be seen on graph 3. There is a direct inverse relationship between the age group and the percentage of that specific age group who frequently use social networking groups. If you study table 4, it will become evident as you move up the age groups, the percentage of strongly agree & agree decreases almost linearly. Table 4: Age differentiation for Q15 Age Strongly agee & Agree Neutral Disagree & strongly disagree Sub total

# 12 0 2 14

18-25 % 86% 0% 14%

# 28 10 4 42

26-35 % 67% 24% 10%

# 27 6 9 42

36-45 % 64% 14% 21%

# 7 1 9 17

46-55 % 41% 6% 53%

# 0 1 1 2

>55 % 0% 50% 50%

6. Conclusions The most significant result from the research was that the participants were very positive towards the organisational applications and especially the middle to older working generation (age 46 to older than 55) did not prefer social networking applications above organisational applications as can be seen from the blue line of graph 3. This was totally unexpected. Why do IT projects still fail? Fail in whose eyes could be another perspective on this question. It may have failed because it was over budget, not delivered within the promised time or not delivered what it was supposed to deliver but according to this comparative research, the evidence suggests, that the customers are happy. The end user of the organisational application may very well not know that the project was over budget, although the business sponsor will have a different view on it. If one or two functionalities were not included, but the new application or upgraded application is a great improvement on the previous or non‐ existent application, then this new application is still be seen a success in the eye of the user. Does involving the customer in the development process guarantee a more satisfied customer? The first portion of this question was answered in the literature review. The latter part of the question can be answered from the research data. Involvement of the customer in the development process does not guarantee success, but it helps to ensure that the customer gets what they want. The project may still be a failure because the analysis was not done correctly, both the development team and the customer had impractical ideas or the objective was not achievable with the technology and resources available, but one thing is almost a certainty, you will get buy‐in from the customer because otherwise he now forms part of the failure.

297

Grafton Whyte and Adolf Lamprecht Why is it that some social applications like Facebook are more acceptable and popular amongst users? Can we learn something from them and use that knowledge in the software development of large scale systems? This was interesting because here the researcher obtained unexpected results from the research. The older the users were, the more they started to lean towards favouring organisational applications rather than social media networking applications as can be seen from graph 3 of the research. The following were some of the possibilities why the organisational applications do not lag behind the social networking applications:

Regular upgrades to organisational software, helps employees perform their daily tasks better with fewer interruptions and fewer manual actions. Therefore they are looking forward to these changes, rather than thinking ‘extra training, job loss possibilities. The research showed that 83% of respondents were looking forward to upgrades.

Regular upgrades where the user was involved in the changes for the upgrade, makes the user feel part of the upgrade. This was one of the recommendations from Holzinger, et al.(2005).

The younger generation, i.e. Generation‐Y, have grown up with technology. They see applications, even organisational applications as an aid in doing their work, rather than an obstacle.

If the new organisational application is marketed before implementation of such a system, this will go far for ensuring a welcomed product.

7. Recommendations After completing the research, the following recommendations can be made:

Upgrades do not only have the purpose of adding new functionalities to the organisational application but also provide the user with an up‐to‐date or current system. This seemed to be what the users wanted from the research.

A communication channel should be made available where the users can contribute and provide feedback on new systems and system changes.

Agile development will not be the last and most effective way of developing applications. Many of the authors in the literature review feel that it is important to stay up to date with development methodologies and adapt over time as more effective ways of developing software become known.

Further research should focus on the following:

A larger research sample could be obtained.

Conduct additional research in the form of interviews on unsatisfied organisational software users to determine which factors contribute to them not being satisfied.

Conduct further research to determine what impact regular updates of organisational software has on the satisfaction level of users.

References Abras, C., Maloney‐Krichmar, D. and Preece, J. (2004), User‐Centered Design. In Bainbridge, W. Encyclopedia of Human‐ Computer Interaction, Thousand Oaks: Sage Publications, pp. 1‐14 Bannerman, P.L. and Thorogood, A. (2012), Celebrating IT Projects Success: A Multi‐domain Analysis, IEEE Computer society, Hawaii International Conference on System Sciences, January 2012, pp. 4874‐4883 Batra, D. (2009), Modified agile practices for outsourced software projects, Communications of the ACM, Volume 52 Issue 9, September 2009, pp. 143‐148 Bolton, M., (2004), Customer centric business processing, International Journal of Productivity and Performance Management, Vol. 53 Iss: 1, pp.44 – 51 Chang, C., Kung, C‐C. and Tan, T‐H. (2008), Development and implementation of an e‐restaurant for customer‐centric service using WLAN and RFID technologies, Machine Learning and Cybernetics, 2008 International Conference, July 2008, pp. 3230 ‐ 3235 Chamberlain, S., Sharp, H., and Maiden, N (2006), Towards a Framework for Integrating Agile Development and User‐ Centered Design, Springer Link, Lecture Notes in Computer Science, 2006, Volume 4044/2006, 143‐153 Costabile, M. F., Fogli, D., Mussio, P., Piccinno, A. (2007), Visual Interactive Systems for End‐User Development: A Model‐ Based Design Methodology. IEEE Transactions on SMC, Part A 37, 6, pp.1029‐1046 Creswell, J.W. (2003), Research design: Qualitative, Quantitative and Mixed Methods. Approaches, Second edition, SAGE Publications, International Educational and Professional Publisher, London UK

298

Grafton Whyte and Adolf Lamprecht Fox, D., Sillito, J., Maurer, F. and Calgary, AB (2008), Agile Methods and User‐Centered Design: How These Two Methodologies are Being Successfully Integrated in Industry, Agile 2008 Conference, IEEE Computer Society, August 2008, pp. 63‐72 Göransson, B., Gulliksen, J. and Boivie, I. (2004), The usability design process – integrating user‐centered systems design in the software development process, Wiley online library, vol. 8, issue 2, April/June 2003, pp. 111‐131 Hageman, A. M. (2008), A review of the strengths and weaknesses of archival, behavioural and qualitative research methods: Recognising the potential benefits of triangulation, Emerald Group Publishing Limited, Advances in Accounting Behavioural Research, Volume 11, pp. 1–30 Holzinger, A., Errath, M., Searle, G., Thurnher, B. and Slany, W. (2005), From extreme programming and usability engineering to extreme usability in software engineering education (XP+UE → XU), Computer Software and Applications Conference, 2005. COMPSAC 2005. 29th Annual International, , IEEE Computer Society, vol. 1, pp. 169‐ 172 Holzinger, A., Searle, G. and Nischelwitzer, A. (2007), On Some Aspects of Improving Mobile Applications for the Elderly, Lecture Notes in Computer Science, Springer Link, Volume 4554/2007, pp.923‐932 Hussain, Z., Slany, W. and Holzinger, A. (2009), Current State of Agile User‐Centered Design: A Survey, HCI and Usability for e‐Inclusion Lecture Notes in Computer Science, 2009, Volume 5889/2009, Springer Link, pp.416‐427 Kerzner, H. (2003), Project management: A systems approach to planning, scheduling, and controlling, John Wiley & Sons, Hoboken, NJ Lieberman, H., Paternò, F., Klann, M., and Wulf, V. (2006), End‐User Development: An Emerging Paradigm , Human‐ Computer Interaction Series, Springer Link, Volume 9, pp.1‐8 Memmel, T., Gundelsweiler, F. and Reiterer, H. (2007), Agile Human‐Centered Software Engineering, Proceedings of HCI 2007, The 21st British HCI Group Annual Conference, University of Lancaster, UK, pp. 167‐175 Orngreen, R., Nielsen, J., and Levinsen, K. (2008), Designing E‐Learning Solutions with a Client Centred Approach, ECIS 2008 Proceedings. Paper 204 Pan, S.L. and Pan, G. (2006), Customer‐Centric IS Application Development: Lessons From a Case of Developing an Online Auction Site, Communications of the Association for Information Systems: Vol. 18, Article 18, pp. 395‐412 Patel,A., Seyfi, A., Tew,Y., and Jaradat, A. (2011) Comparative study and review of grid, cloud, utility computing and software as a service for use by libraries, Library Hi Tech News, Vol. 28 Iss: 3, pp.25 – 32 Silva da Silva, T., Martin, A., Maurer, F. and Silveira, M. (2011), User‐Centered Design and Agile Methods: A Systematic Review, Agile conference (Agile), 7 ‐13 August 2011, IEEE Society, pp. 77‐86 Turner, I.L.; Jones, K.W. (2007), User‐centric Innovations: A Survey Part 1, Collaborative Technologies and Systems, 2007. CTS 2007. International Symposium, IEEE Computer Society, 25 May 2007, pp. 212‐223 Walsham, G. (2006), Doing interpretive research, European Journal of Information Systems (2006), vol. 15, pp.320–330 rd Welman, C., Kruger, F. and Mitchell, B. (2011), Research methodology, 3 edition Published by Oxford University Press, Cape Town

299

Using the Inkwenkwesi Model to Identify the Knowledge Requirements of a new Project Entrant Dr Grafton Whyte1 and Desire Minnaar2 1 Namibia University of Science and technology in Namibia 2 University of Western Cape, South Africa drgwhyte@aol.com desire.minnaar@gmail.com Abstract: IT projects fail for many reasons. Knowledge is a critical resource within organizations and projects but is often overlooked as a cause of project failure. Changes to project teams can affect the knowledge balance within the team and new entrants often have to learn large amounts of information before actively contributing to the project. The Inkwenkwesi model identifies eight domains of knowledge; People, Process, Organization, Product, Service, Suppliers, Customers and the External Environment. This research aims to identify the knowledge requirements of a new project entrant to become productive sooner, and test the Inkwenkwesi Model as a tool for assisting new project entrants acquire the knowledge required to become productive sooner. Keywords: Inkwenkwesi Model, project knowledge, project requirements, project entrant

1. Introduction IT project failures are often attributed to poor project management practices and change management. Knowledge management is however, often overlooked as a cause of project failure. Knowledge has been identified as a critical resource within organisations and the management thereof can contribute to project success. Projects are made up of project teams with the purpose of achieving an objective, product or service within a defined scope and time frame. New members joining a project team often have to learn large amounts of information before actively contributing to the project team. The Inkwenkwesi model provides a holistic knowledge view of an organisation (Whyte, 2011). Here was an opportunity to conduct research that focused on identifying the knowledge requirements of new project entrants based on the Inkwenkwesi knowledge model. Inkwenkwesi is Xhosa for star. This paper has two broad goals: 

1) Identify the knowledge requirements of a new project entrant which will enable them to become productive sooner and,



2) Test the Inkwenkwesi Model as a tool for assisting new project entrants acquire the knowledge required to become productive sooner when joining a project.

The literature is reviewed with respect to types of knowledge that occur within a project. The Inkwenkwesi model is introduced and the research design and methodology discussed. The results and findings are presented and finally some recommendations are made.

2. Literature review Large amounts of research exists that describe reasons for Information Technology project failures (Reich, 2007; Chan et al, 2008; Kang & Hahn, 2009 ; Karlsen & Gottschalk , 2004 ) These include but are not limited to: a lack of project management and processes, inadequate scope and change management (Reich 2007). Knowledge management however, has been overlooked as a cause of project failure (Disterer 2002). In support of this view, it has been shown that effective knowledge management increases the likelihood of project success (Faraj & Sproull 2000). Yoo & Kanawattanachai (2001) prove that the coordination and management of knowledge between different teams has a strong influence on the success of a project. Tiwana, Bharadwaj, & Sambamurthy (2003) also report that knowledge integration is of greater importance to project success than the relationships between IT project teams and the business. Kasvi et al (2003) remark that, in addition to competence, project success is dependent on accumulated, collective and individual knowledge. It’s important to note here, that our attention is drawn to the fact that both individual and collective competence is required for success. Chan et al (2008) reinforces this statement, saying that skills possessed by team members are critical for success.

300

Grafton Whyte and Desire Minnaar

2.1 Knowledge within a project The literature refers to four main types of knowledge critical to IT projects these are: Project Management knowledge, Business Domain knowledge, Technical knowledge and Organisational knowledge. 2.1.1 Project Management knowledge Project Management Knowledge refers to the knowledge of coordinating, planning and managing the various IT project activities (Zhao & Zuo, 2011). It involves the planning, organizing, controlling the project and managing project risk, in order to meet the project objectives within various time and budget constraints (Chan & Rosemann, 2001; Goles et al, 2008). Reich (2007) refers to this as process knowledge in her research. It is the knowledge about project time frames, tasks, processes, methodologies and project structure. This knowledge clarifies what is expected from each team member within the project and associated timelines. Chan et al., (2008) places great emphasis on team task, coordination skills and knowledge. They state that this knowledge is more important and has a greater impact on project success than application domain and development method knowledge. Research done by Goles et al., (2008) has found that project management skills and knowledge are one of the highest ranked skills for IT firms today. 2.1.2 Business Domain Knowledge The second type of knowledge is Business Domain Knowledge. This refers to knowledge about the “customer” organisation that the IT project is delivering to. It includes knowledge about the business processes, rules, activities, functional areas, entities, business operations and policies, customer business objectives and stakeholder needs (Zhao & Zuo 2011; Tiwana, 2004; Kang & Hahn, 2009; Lee et al., 1995; Chan & Rosemann, 2001). Reich (2007, pp.9) expands on this and states that it includes knowledge about the industry and the business opportunity or problem currently at hand. Current project members would have a good understanding of the organisation and its processes; however this may be a gap for a new member joining the project. Kang & Hahn (2009, pp3) stress the importance of domain knowledge stating that without it the project team would not know what to build in the IT solution. Business domain knowledge includes the ability to integrate business needs with technology and thus requires the knowledge of functional area processes and business process design in order to meet the needs of the organisation (Goles et al., 2008; Chan et al., 2008). 2.1.3 Technical Knowledge The third category of knowledge is Technical knowledge. Zhao and Zuo (2001, pp.268) refers to this as project engineering knowledge. This knowledge refers to the technical know-how to implement information systems and the activities involved in the systems development life cycle (Zhao & Zuo, 2001; Goles et al, 2008). This category of knowledge is IT specific, i.e. what the various technologies are and how they are implemented. 2.1.4 Institutional Knowledge The last major knowledge category critical to IT projects is Institutional knowledge. This knowledge refers to the history and values of the organisation. Reich (2007) advises that institutional knowledge is not so much about the facts of the organisation, but rather about how facts are to be interpreted to understand “what is really going on”. Chan & Rosemann (2001, pp. 42) refers to this as company-specific knowledge. In their research on managing knowledge of enterprise systems they claim that systems cannot be successfully implemented without taking individual company-specific factors into accounts. Company-specific knowledge includes organisations values, culture, staff morale, behavior and attitudes within the organisation.

2.2 Overview of knowledge types Software development is a knowledge intensive process. It requires specialized knowledge in many domains to conceptualize and design software solutions that meet business needs (Kang & Hahn, 2009). A study conducted by Tiwana (2004) concluded that high levels of knowledge integration contributes positively and influences the success of software development. Business and Technical domain knowledge needs to be integrated to improve effectiveness and efficiency of software development. Reich (2007, p.11) advises that knowledge integration can benefit project performance and is crucial when a projects attempts to solve difficult problems. On the other hand she suggests that process and domain knowledge are the most important knowledge areas for project team members. Kang & Hahn (2009) indicates

301

Grafton Whyte and Desire Minnaar that acquiring technology and methodology knowledge precedes domain knowledge, in other words an individual would acquire technology and methodology knowledge before acquiring domain knowledge. Kang & Hahn (2009) refer to methodology knowledge as the knowledge related to the process of managing a software project, referred to in this paper as Project Management knowledge. According to Chan et al (2008) team task skills, here classified as Project management knowledge, are more critical than other skills in a project. Goles et al. (2008) focused their research on the most critical IT knowledge required by organisations today and into the future. Their findings reflect that Project Management knowledge, specifically project planning, budgeting and scheduling, was ranked the highest critical skills for IT firms. It was clear from the findings that Technical knowledge did not appear to be critical to IT firms.

2.3 Inkwenkwesi model The Inkwenkwesi model, proposed by Whyte (2011), postulates a holistic knowledge view of an organisation. The Inkwenkwesi model, hereafter referred to as the Ink Model, postulates that there are five key knowledge domains for IT projects within the boundary of the organization and three outside. The five domains of the knowledge star represent People, Process, Organisation, Product and Service. Suppliers, Customers and the External Environment are the knowledge domains outside of the organisation boundary. Organization Boundary

Service

Process

Organization

Customer

Suppliers

People

Product

External Environment

Figure 1: Inkwenkwesi model (Whyte G, 2011) A project team can be viewed as a micro organisation. The organisation boundary in the Ink module thus refers to the boundary of the project and project team. With this in mind, the five corners of the knowledge star thus highlight the various kinds of knowledge that exists within an IT project team. Within projects tacit and explicit knowledge exists which may be mapped to the knowledge domains of the Ink model in accordance to the types of knowledge identified within IT projects earlier in this paper. The Ink model will be used as a frame of reference for identifying the knowledge requirements of a new project entrant.

3. Research design and methodology Several knowledge types have been identified in the literature. The four broad knowledge categories include Project Management, Business Domain, Technical and Institutional knowledge. The Inkwenkwesi Model identifies a further eight (8) knowledge types. Five (5) within the boundary of the project, i.e. Service, Product, Process, People and Organisation knowledge and three (3) immediately outside of the project boundary, i.e. Environmental, Supplier and Customer. Altogether these twelve knowledge types are tested to understand the relationships that may exist between the perceived level of the respective knowledge types acquired by the new project entrant and the perceived rate at which the individual felt productive within the project. Correlations were calculated between the level of knowledge acquired for the respective knowledge types and the perceived time to becoming productive on the project. Furthermore the study aims to identify which knowledge types have a greater impact on the rate of productivity and whether these knowledge types can be mapped against the Inkwenkwesi model. For this purpose a survey questionnaire was the chosen method for answering the research questions.

302

Grafton Whyte and Desire Minnaar The research survey consisted of four (4) open-ended questions and thirteen (13) close-ended questions. Twelve (12) of the thirteen (13) close-ended questions were based on the twelve (12) knowledge types. For each of these knowledge types, the respondents were required to rate their level of knowledge before they were productive on the project as well as after they were productive on the project, in the form of a Likert scale ranging from 1 (Poor Knowledge) to 7 (Strong Knowledge). Two values were thus captured per knowledge type. The last close-ended question required respondents to rate the overall time it took them to become productive on the project, again ranging from 1 (Very Slowly) to 7 (Very Quickly).

3.1 Research sample The target population in this study included IT staff that had just joined an IT project within an environment unfamiliar to them. In other words, specifically project members that have recently joined a project that was new to them and therefore caused them to experience some level of discomfort due to a lack of knowledge at the point of joining the project, thus impacting on their performance and productively. The sample selected for this study consisted of IT professionals employed by a leading South African insurance company that had been working on projects within the IT industry. A total of 50 IT professionals were surveyed, of which 45 responses were collected and analyzed, resulting in an acceptance rate of 90%. The sample included a variety of IT roles, i.e. project managers, developers, architects, and business analysts.

3.2 Instrument design and collection method An online web-based survey tool was used to create the survey. An email including a summary of the research objective and a website link to the online survey was sent to the 50 IT professionals employed by the IT department. The website link in the body of the email directed respondents to an intranet SharePoint website. All responses were recorded online, exported to an excel spreadsheet and prepared for analysis.

3.3 Survey reliability Reliability is defined as the degree to which the survey provides stable and consistent results under similar conditions (Litwin M, 1995). It is the measure of stability when the measurement is repeated in an identical population. The Test-retest reliability assessment was applied to test the reliability of the survey. The survey was completed on two occasions, two months apart, by a pilot test group. The reliability assessment was calculated using Cronbach’s Alpha’. The Cronbach’s Alpha was calculated for all the questions, i.e. Twelve knowledge types (for before and after productivity) and one question based on the time-to-productivity. A Cronbach’s Alpha of .98 was calculated, demonstrating a high degree of reliability and internal consistency. According to DeVellis (2003), reliability is considered to be good if alpha values are greater or equal to 0.7.

4. Results and discussion The Pearson Correlation Coefficient calculation was used to determine the correlation coefficient between the twelve knowledge types and the time to productivity. It was first calculated for the before-productivity knowledge value to time-to-productivity, and thereafter for the after-productivity knowledge value to time-toproductivity. The twelve knowledge types were analysed and are reported in three categories; 

1) During the project knowledge types – these are knowledge types that showed a positive correlation as knowledge acquired during the project which contributed to the productivity of the new project entrant,



2) Prior to project knowledge types – these are knowledge types that showed a strong correlation as knowledge acquired prior to the project which contributed to the productivity of the new project entrant and,



3) Insignificant knowledge types – these are knowledge types that seemed not to have a strong correlation at any stage of the project and did not seem to contribute positively to the productivity of the new entrant.

4.1 Presentation and discussion The table below illustrates the correlation and significance p-values for each of the knowledge types as a result of the data analysis process completed. Values displayed are for before-productivity and after-productivity on

303

Grafton Whyte and Desire Minnaar the project. All before-productivity and after-productivity values are correlated to the time-to-productivity and classified according to their respective categories.

Knowledge Type

Customer

Before-productivity (Correlated to time-to-productivity) (N=45)

After-productivity (Correlated to time-to-productivity) (N=45)

Pearson Correlation

Significance

‘During the Project’ Knowledge Types 0.27 0.04

Service Supplier

Significance

0.38

0.01

0.34

0.01

0.36

0.01

0.38

0.01

0.4

Prior to Project Knowledge Types 0.59 0

0.47

Process

0.44

0.39

People

0.46

0.34

0.01

Project Management

0.45

0.11

0.23

Organisation

0.42

0.14

0.18

0.14

0.17

Technical

Business Domain

Insignificant Knowledge Types 0.22 0.07

Product

0.18

0.12

0.19

0.11

Environmental

0.16

0.14

0.23

0.06

Institutional

0.09

0.28

-0.08

0.29

Table 1: Pearson Correlation and Significance values

4.2 During the Project’ knowledge types The first category, ‘During the Project’ knowledge types consists of Customer, Service, and Supplier are the only knowledge types with an after-productivity correlation greater than the before-productivity correlation to time-to-productivity and are also statistically significant. Suggesting that not only do these knowledge types contribute positively to the new entrant’s time to productivity, but it increases over time (denoted by the increase in the correlation coefficient values) as the new entrant acquires more knowledge in these areas during the project. The Customer before-productivity to time-to-productivity relationship is considered a weak positive relationship, (coefficient of 0.27). The Customer after-productivity to time-to-productivity relationship is however strengthened, and considered to be a moderately positive relationship (correlation of 0.38). The increase from before-productivity to after-productivity correlations indicates that it is the Customer Knowledge gained during the project may result in reduced time to productivity, since the relationship to the time it takes to become productive is strengthened for customer knowledge gained during the process of becoming productive. Customer knowledge also reflects the greatest increase (from 0.27 to 0.38) for knowledge types in the ‘During the project’ category. Supplier and Service knowledge also reflect a greater after-productivity correlation to time-to-productivity compared to the before-productivity correlation, meaning that the supplier and service knowledge gained during the process of becoming productive on the project may result in faster times to productivity for the new project entrant. An increased after-productivity correlation with time-to-productivity value reflects that the Supplier and Service knowledge gained during the project may have a greater positive impact on the time to productivity for the new entrant compared to the existing supplier knowledge the new entrant entered into the project with.

304

Grafton Whyte and Desire Minnaar

4.3 Prior to Project’ knowledge types The ‘Prior to Project’ knowledge types is made up of Technical, Process, People, Project Management and Organisation knowledge. 4.3.1 Before Productivity These five knowledge types have been categorized together for two main reasons. Firstly, the beforeproductivity values of these five knowledge types have a strong positive relationship with the time-toproductivity. (Correlations values are greater than 0.4) The second reason for this classification is that the after-productivity value for these five knowledge types has a weaker, although positive, correlation to time-toproductivity. This means, that it is the Technical, Process, People, Project Management and Organisation knowledge that the project entrant entered into the project with, i.e. knowledge gained prior to joining the project that may result, in the new entrant becoming productive faster. These relationships are also statistically significant since their p-values are less than 0.05. 4.3.2 After Productivity The after-productivity values correlations to time-to-productivity are less than the before productivity correlations for these five knowledge types. This indicates that these knowledge types may have had a positive impact on time-to-productivity, but not as great as the knowledge acquired prior to joining the project. In other words, it’s the knowledge gained prior to joining the project that may have a greater positive impact on the time it takes the new entrant to become productive, than the knowledge gained during the project in these areas. Only Technical, Process and People knowledge types have a moderately positive to strong positive relationship with the time it takes to become productive. Although these relationships are not as strong are their respective before-productivity values, however they are statically significant. Project Management and Organisation knowledge gained during the project however reflect a negligible relationship to time-to-productivity. These relationships are not statistically significant. The Project Management and Organisation knowledge gained prior to the project share a significant and strong positive relationship to the time it takes the new entrant to be productive. The knowledge gained during the project in these areas however, is shown to be insignificant and makes very little difference to how soon the project entrant becomes productive.

4.4 Insignificant’ knowledge types Business Domain, Product, Environment and Institutional knowledge types are classified in this study as insignificant knowledge types. These knowledge types reflect negligible or no relationship to the time it takes to become productive. This applies to knowledge gained prior to joining the project and during the project. Knowledge gained in these areas seem to have very little impact on the time the new entrant felt it took to become productive on the project. The relationship between the knowledge gained prior to and during the project, to the amount of time it took the new entrant to become productive on the projects are not only weak or non-existent, but also not statically significant.

4.5 Inkwenkwesi Model Analysis This section maps how well the Inkwenkwesi model identifies the knowledge types required by the new project entrant in order to become productive sooner. The Inkwenkwesi model, (Ink Model), includes eight of the twelve knowledge types used for analysis in this research. 4.5.1 During the Project’ knowledge types All the knowledge types classified as ‘During the Project’ knowledge types, i.e. Customer, Service and Supplier are included in the Inkwenkwesi model. These three knowledge types were the only ones identified, as speeding up the process of becoming productive during the project when acquired during the project. Figure 2 below, illustrates these knowledge types as mapped on the Inkwenkwesi Model. The knowledge types highlighted in green on the Ink Model makes reference to the ‘During the Project’ Knowledge types, as found

305

Grafton Whyte and Desire Minnaar on the Ink Model. In this regard, the Ink model identifies all the knowledge types a new project entrant requires during the project in order to become productive sooner. Organization Boundary

Service

Process

Organization

Customer

Suppliers

People

Product

External Environment ‘During the Project’ Knowledge type ‘Prior to Project’ Knowledge type ‘Insignificant’ Knowledge type

Figure 2: Inkwenkwesi Model with classified knowledge types 4.5.2 Prior to Project’ knowledge types The knowledge types highlighted in Orange in Figure 2 illustrates the ‘Prior to Project’ Knowledge types found on the Ink Model. In the previous section, Technical, Process, People, Project Management and Organisation were identified as ‘Prior to Project’ knowledge types. The Ink Model identifies three, People, Process and Organisation knowledge. Technical and Project management knowledge are the two ‘Prior to Project’ knowledge types not included in the Ink Model. 4.5.3 Insignificant Knowledge Types Business Domain, Product, Environment, Institutional knowledge were identified as Insignificant knowledge types. Figure 2 highlights the Insignificant knowledge types included in the Ink Model in purple. The Ink model consists of two insignificant knowledge types. These are Environment and Product knowledge. Overall the Ink Model is shown to be a good model for identifying the knowledge types required by a new project entrant in order to get productive sooner. All ‘During the project’ knowledge types are included in the Ink model. These identify the knowledge requirements during the project. Three of the five orange types, ‘Prior to Project’ knowledge types are included in the Ink model. These are knowledge types identified as knowledge to be gained prior to joining the project in order to become productive sooner, and lastly only two out of four insignificant knowledge types are included in the Ink Model. Overall this makes for a good model to be used to identify the knowledge types of a new project entrant.

5. Conclusion Twelve knowledge types were analyzed. These knowledge types were tested in terms of how knowledgeable the project entrant felt before productivity was achieved in the project and after productivity was achieved. Lastly the new project entrant was required to rate how soon productivity was reached within the project. Customer, Service and Supplier knowledge were identified as the knowledge types that may have a strong impact on the time it takes to become productive during the course of the project. Acquiring these knowledge types during a project, may reduce the time to productivity.

306

Grafton Whyte and Desire Minnaar Technical, Process, People, Project Management and Organisational knowledge types may have a strong positive impact on the time it takes the new entrant to become productive when entering a project. It is important to note, that these knowledge types may result in faster productivity times, when the new project entrant enters the project knowledgeable in these areas. Gaining knowledge within these areas during the project has a lesser impact on the time to productivity compared to entering the project already knowledgeable in these areas. Business Domain, Product, Environmental and Institutional knowledge types were classified as Insignificant Knowledge. These results appear to show that being knowledgably in these areas is not likely to reduce the amount of time it takes a new project entrant to become productive. These results are displayed irrespective of when knowledge in these areas were gained, i.e. before the project or during the project. The Inkwenkwesi Model is shown to be a good model for identifying the knowledge types required by a new project entrant. All the knowledge types required during the project, i.e. Customer, Service and Supplier are covered by the model. Three of five knowledge types identified as knowledge types gained prior to the project that may reduce time to productivity are also included in the Inkwenkwesi model, i.e. Organisation, Process and People. Lastly, only two out of four insignificant knowledge types are included in the Inkwenkwesi Model, Environmental and Product. The Inkwenkwesi is identified as a good model to be used to identify the knowledge types required by a new project entrant to become productive sooner.

6. Recommendations Based on this research, the following recommendations can be made to organisations that run IT projects. 

Project managers can use the Inkwenkwesi Model with the classified knowledge types, to identify the knowledge types required during the project for any new persons joining the project. If project managers focus on ensuring knowledge about the Customer, Supplier and Service are shared with new project members early in the project; it may result in new members becoming productive sooner.



The Inkwenkwesi Model, with the classified knowledge types, may also be used during the process of selecting project members to join a project. ‘Prior to Project’ knowledge types have been shown to be associated with reduced times to productivity. Although the Inkwenkwesi model only includes three of the five Prior to Project’ knowledge types, i.e. Process, People and Organisation, there may be merit in including these knowledge areas as part of the selection criteria when identifying potential candidates for projects. Ensuring the potential candidates also have People, Process and Organisation knowledge prior to joining the project may reduce times to productivity for the potential candidate.



The Inkwenkwesi model with the classified knowledge types may be used to identify knowledge gaps within the existing project team. Productivity may be increased in the project teams, if the reason for lowered productivity is as a result of a lack of knowledge in the team. The Inkwenkwesi model with the classified knowledge types may be used to assess the collective project knowledge that exists in the team and identify weaker knowledge areas that need to be addressed, in order to increase productivity.

References Chan, C, Jiang, J & Klein,G 2008 ,’Team task skills as a facilitator for application and development skills’ ,IEEE Transactions on Engineering Management , vol. 55, no 3, pp. 434-441 Chan, R, & Rosemann, M 2001, ‘Managing knowledge in enterprise systems’, Journal of Systems and InformationTechnology, vol. 5, no. 2, pp. 37-54 Lee, D, Trauth, E & Farwell, D 1995,’Critical skills and knowledge requirements of IS professionals: a joint academicindustry investigation’ , MIS Quarterly , vol. 19 , no. 3, pp.313-340 Faraj, S & Sproull, L 2000, ‘Coordinating expertise in software development teams’, Management Science, vol. 46, no. 12 ,pp. 1554-1568. DeVellis, R . 2003 Scale development: Theory and applications, 2nd edition. Thousand Oaks, CA: Sage Disterer G 2002, ‘Management of project knowledge and experiences’, Journal of Knowledge Management, vol. 6, no. 5, pp 512 – 520 Goles, T , Hawk, D & Kaiser, K 2008, ‘Information technology workforce skills: the software and IT services provider perspective’ Information Systems Frontiers, vol. 10, no. 2, pp. 179-194 Kang, K & Hahn, J 2009, ’Learning and forgetting curves in software development: does type of knowledge matter?" ICIS Proceedings. Paper 194 Available online: http://aisel.aisnet.org/icis2009/194 , viewed 9 January 2012 Karlsen, J , Gottschalk P 2004, ’Factors Affecting Knowledge Transfer in IT Projects.’ Engineering Management Journal 1, vol. 6, no. 1 pp. 3-10

307

Grafton Whyte and Desire Minnaar Kasvi, J., Vartiainen, M, & Hailikari, M 2003,’Managing knowledge and knowledge competences in projects and project organisations’, International Journal of Project Management, vol. 21, no 8, pp 571 – 582 Reich, B 2007,’Managing Knowledge and Learning in It Projects: A Conceptual Framework and Guidelines for Practice’, Project Management Journal, vol. 38, no. 2, pp. 5- 17 Tiwana, A 2004, ‘An empirical study of the effect of knowledge integration on software development performance,’ Information and Software Technology, vol. 46, pp.899-906. Whyte G, 2011, Inkwenkwesi (K Stars) Programme, PowerPoint presentation distributed via email on 21 Monday 2011 (unpublished) Whyte G, (1995),’ Understanding user perceptions of successful Information Systems’, , unpublished thesis (Ph.d.), Cranfield University. Yoo Y, Kanawattanachai P , 2001,’ ‘ Developments of transaction memory systems and collection mind in virtual teams, International Journal of Organizational Analysis, vol. 9 Iss: 2, pp.187 - 208 Zhao, D, & Zuo, M 2011, ‘Knowledge Transferred across Projects and Its Characteristics in an IT Service Enterprise ‘, Information Technology, Computer Engineering and Management Sciences (ICM), 2011 International Conference, vol. 3, pp. 267 – 270

308

An Investigation into the Effectiveness of Storytelling as Means of Sharing Tacit Knowledge Grafton Whyte1 and Monica Ralake2 1 Namibia University of Science and technology in Namibia 2 University of Western Cape, South Africa drgwhyte@live.com rminkie@yahoo.com Abstract: Purpose – The aim of the study is to investigate the effectiveness of storytelling as means of sharing tacit knowledge at Eskom Distribution in the Western Region. There is much concern among many South African organizations, that the imminent retirement of many senior executives and subject matter experts is going to leave a vast tacit knowledge gap and loss of organizational memory which may takes years to recover, if possible at all. There has been much interest in storytelling as a means of capturing this knowledge, but to date there has been no research within the Knowledge or Information Management disciples looking at the efficacy of storytelling as a means of capturing tacit knowledge. Design/methodology/approach – In total, eight stories were collected from four regional executives (storytellers) and were analysed using an interpretive approach from six randomly selected knowledge workers (storytakers). Findings – The findings of this study clearly demonstrated that valuable tacit knowledge can be captured through storytelling. In addition the results suggest that the storytelling form allows for a rich application of the story beyond the original intent of the storyteller. The research confirmed that tacit knowledge sharing can add considerable value to the organisation’s competitive environment. The researcher recommended factors to consider when implementing the use of stories and storytelling as a tacit knowledge capturing method including a solid process for encouraging knowledge workers with long service to share their experiences. Research limitations/implications – The generalizability of the research is

limited by the fact that data was collected from only one organization. Despite the limitation this research presents empirical evidence in support of the effectiveness of storytelling as a means of capturing tacit knowledge and thereby provides a solid base upon which further research can be built. Originality/value – This

paper investigates the effectiveness of storytelling as a tool for sharing tacit knowledge the efficacy of which, to the author’s knowledge, has never been substantiated. This research paper presents evidence in this regard. Keywords - Stories, Storytelling, Storyteller, Storytaker, Tacit knowledge, Knowledge Sharing

1. Introduction Denzin and Lincoln (2005) state that a research design should involve ‘a clear focus on the main research question, the purpose of the study, what information will answer specific research questions most appropriately, and which strategies are most effective for obtaining answers to the questions. The main objective of this study is to shed light on the effectiveness of storytelling as a means of transferring tacit knowledge at Eskom Distribution in the Western Operating Unit. The results of the study could contribute to a better understanding of the practice of storytelling and the process of knowledge transfer within organisations. Eskom is a South African company that generates approximately 95% of the electricity used in South Africa and approximately 45% of the electricity used in Africa. Eskom generates, transmits and distributes electricity to industrial, mining, commercial, agricultural and residential customers and redistributors. Additional power stations and major power lines are being built to meet rising electricity demand in South Africa Organisational stories are a step beyond testimonials. Stories are frequently about real people doing real work and involved with real challenges. The stories in this study are collected from regional executives (storytellers) and then shared among colleagues in other parts of the organisation (storytakers). Once the stories have been listened to the understanding of the storytaker is checked against three criteria; (i) meaning, (ii) lessons learned and (iii) application of the story. The more correspondence there is between the storytellers’ explanation of the story and the storytakers’ understanding of the story the more effective we would consider the story to be as a means of knowledge transfer. Tacit knowledge is difficult to communicate or share. The capturing of tacit knowledge has been one of the main challenges in knowledge management (Santoro & Brezillon, 2005). There are number of ways that can be used to capture tacit knowledge, one of them being storytelling. Since the development of speech human beings have communicated through telling stories. In oral traditions storytellers were held in high esteem, they entertained, touched the emotions and passed on knowledge to the storytakers. Storytelling is a part of

309

Grafton Whyte and Monica Ralake human’s daily life resulting to it becoming a potential approach in knowledge sharing. However, to date the researcher is unaware of any study that has looked at the effectiveness of storytelling as a means for transferring tacit knowledge within an organisational context. This research is a first attempt at evaluating the effectiveness of storytelling by addressing the main research question:

2. Is storytelling an effective means of sharing tacit knowledge? 2.1 Literature review In this section a brief history of storytelling and the effectiveness of storytelling as means of sharing tacit knowledge in an organization will be discussed. In order to start answering the research question it is important to understand how organizational stories should be shared and what are the current and future trends. In the modern business world, storytelling (or narrative) is emerging as an important informal method of communication. Storytelling is regarded as important in conveying experiences of work whilst communicating shared knowledge and learning and maintaining organizational memory (Lehaney et. al, 2004). Studies indicate that storytelling plays a significant role in sharing tacit knowledge resulting in shaping the knowledge base of organizations. It is very important to understand the technique and the context of Storytelling for the holistic management of organizational knowledge. Stories are a fundamental form of knowledge and communication and are particularly suited to knowledge management (Ma & Keppell, 2004). As much as there are several dissimilarities in story type and utilization, they do share some common characteristics. One of the most important characteristic is that stories exist in the realm of knowledge, not information. First, stories not only transfer information, but also meaning and knowledge. The information they contain is seamlessly incorporated into the story through the use of context. And since stories create clusters or chunks of information, they are easier to pay attention to and to remember. The assumption is that stories cannot be used to transfer all kinds of organisational knowledge which is why it is important to demonstrate that storytelling is a powerful tool and the contexts in which it can be used. Particularly, because it is very difficult to use it intentionally as a management instrument. For example teaching core capabilities within an organisation by storytelling would be very difficult. For these forms of knowledge transfer formal education is widely used. Still stories allow the transfer of knowledge with all its complex connections (Davenport et al. 1998). Stories can successfully transfer knowledge about managerial systems, norms, values and moral of a company. They can be easily identified with, are a powerful in transferring knowledge rich in tacit dimensions (Swap et al. 2001). Stories allow the listener to comprehend new experiences and to make judgements about the person, objects, and beliefs of the storyteller. Stories help develop general attitudes and beliefs (Adaval et al. 1998).

Storytelling engages the participants' imagination and emotions, and it opens the minds of listeners.



It offers a way to explore complex issues from multiple vantage points.



The storyteller feels valued for his or her expertise.



People remember information better if it's in story form.



The exercise builds trust among participants, which improves cooperation.

Numerous definitions of organizational story have been proposed. Browning (1992) defined stories as “communications about personal experience told in everyday discourse,” reflecting the role of stories as the reporting of historical events through normal organizational conversation. Hansen, borrowing from Feldman(1990), claims that stories shared in organizations are “socially constructed accounts of past events that encode culture and are therefore important to members of the organization” .Boje (2000) defines organizational stories as the “narratives that are shared because something outstanding or out of the ordinary has happened”. The above definitions suggest the belief that the organizational story is a type of communication with often-questionable accuracy in order to be relevant and to satisfy various needs and wants of multiple organizational members.

310

Grafton Whyte and Monica Ralake As for individuals or the society at large, stories are very important for the organisation as well. Organisational strength and weakness are reflected by stories told by employees. Various organisational parts such as divisions, regions and teams create stories that describe the employee relationship, cultural dilemmas, lessons learned and management style. It is important that organisational leaders are aware of stories told by their subordinates and use then to the benefit of their organization. Unknown, hidden or ignored stories may threat managers’ interests.

3.1 Storytelling as a Knowledge Management tool Bailey (2005) cited Larry Prusak, the founder of and executive director of the Institute of Knowledge Management (IKM), keynote address at a national conference on knowledge management in the United States where he emphasized what was important about knowledge management in a single word: Storytelling. Prusak also mentioned that “knowledge is not in a database and it is not in computer applications but it is in our stories”. The act of telling stories is considered as an old skill but in a new context: Knowledge Management (KM) (Snowden, 1999). Storytelling in KM is used as a technique to describe complex issues, explain events, understand difficult changes, present other perspectives, make connections and communicate experiences. Storytelling approach is used to communicate experiences and transferring tacit knowledge (Taifi, 2010). Stories are beneficial to people and they have a capability to externalize the tacit knowledge which is why they are regarded as one of the most useful technique in knowledge management. Storytelling could be used as a solution to organizations faced with a serious retirement crisis, where large or significant numbers of experienced workers are retiring. These near retirees can share their experiences using storytelling prior to them leaving. Enterprises worry that this knowledge will disappear when the workers leave. Much of their value lies in tacit knowledge (such as decision logic, methods and best practices) that workers have accumulated through experience. The loss of tacit knowledge could hurt any part of operations. If the loss affects mission-critical processes or tools, it could damage the business severely. (Gartner 2009) The whole point of knowledge management is to make sure that the knowledge present in an organization is applied productively for the benefit of that organization. Storytelling is considered an effective tool that can be used to make sure that important knowledge is shared and utilized effectively throughout the organisation. It is vital for organisations to capture stories from experienced workers and retain the knowledge for future use.

3.2 Knowledge Sharing at Eskom Eskom has approximately 39 200 employees and quite a number of stakeholders worldwide. Due to the size of organization, knowledge management is one of the most important aspects. Eskom as a business focuses more on ensuring that sustainable development becomes a reality. It plays a major role in accelerating growth in the South African economy by providing a high-quality supply of electricity to satisfy the needs of the country. Knowledge sharing has always been an important part of the business strategy, with management implementing programmes to facilitate the sharing of the organization’s depth and wealth of knowledge with all those affiliated with the organization. To effectively promote a Knowledge sharing culture, the organization has numerous sources of information such as Eskom Information Centre, organizations’ library workshops, courses, mentorship and seconded programs. Top management support has been very effective in Eskom as indicated by the existence of KM technologies such as collaborative tools. Communication is almost instantaneous, even across a wide geographical spread. An important factor in the effective knowledge sharing depends on the degree of information and communication technologies in the organization. Tacit knowledge is subconsciously understood and applied, difficult to articulate, developed from direct experience and action (maybe the SECI model where Socialisation results in knowledge sharing though story telling). Usually, shared through highly interactive conversation, story-telling and sharing of experiences. In Eskom there is a clear map of different categories of knowledge stored in the repository. It is observed that the tacit knowledge is shared through the events such asBe smart Be Safe work shop – this workshop gives emphasis on knowledge sharing. Both external and internal knowledge is shared in this event. The external knowledge is imported through external speaker while internal

311

Grafton Whyte and Monica Ralake knowledge is disseminated by the employees. Interesting topics on site incidents and safety precautions subjects are covered in storytelling. Technology show – this is a technology event. It presents the future of organization’s research and development activities. The speakers are invited from the organization along with its worldwide subsidiaries. Best practices are shared through a dedicated repository. Employees share their achievements and experiences with best practices. External and Internal Trainers conduct training sessions for the employees. This leads to the development of mentors one of the objectives of which is to make fresh recruits to be project-ready with domain knowledge and process expertise relevant to their roles. Research Methodology This study presents the research purpose, approach, and strategy used to address the research problem as mentioned in the introduction. As cited by Hussey and Hussey (1997), “this research project sought to analyse and explain (the purpose of the research), through mainly qualitative methods (the process of the research) using deductive logic based on existing theories, the role of stories and storytelling as knowledge sharing practices (the logic of the research) and the outcome is one of applied research (applying the research to a particular organisation). This is in line with the overall research problem as identified in the introduction. There are three main sections to these are 

the research purpose



research approaches and



research strategy.

The methodology of this study is of a descriptive nature. There are a number of techniques that are currently used to share tacit knowledge in the organisation and storytelling is one of them, however only storytelling as a knowledge sharing technique will be investigated in a highly knowledge intensive industry. Required information for this study will comprise relevant information about Eskom Distribution Western Operating unit. For the success of the study, facts about storytelling and its importance as an effective knowledge sharing method was collected. Related theories about Organizational Storytelling and Knowledge sharing are a must to assemble required information. The research methods include therefore literature review, interviews and case studies of organisational stories. Interviews at Eskom, will give an empirical base for the research problem questions as stated in the introduction. The research presents a number of variables connected to the issue on how storytelling can be used as a competitive tool. The researcher identifies the factors that influence storytelling to be an effective means of sharing knowledge in an organisation. The identified factors make the study to be explanatory. This study uses a case study approach to investigate employees’ understanding on storytelling as a knowledge transfer tool and the perception of the employees on the facilitators. Barriers of deploying storytelling approach in the organization are also presented. The data collection process for case studies is more complex than those used in other research strategies. What makes it so complicated The researcher further (apart from what) needs to follow certain formal procedures to assure quality control during the data collection process Utilization of archival records as a source of evidence will be excluded from this study. The researcher decided to use open-ended interview as it was the most appropriate. The Interviews Process Based on experience, our candidates were selected and divided into two groups; storytellers and storytakers.

4. The storyteller interviews Interviews were divided into 2 parts where:

312

Grafton Whyte and Monica Ralake Four storytellers (Organization Executives) with their multiple stories will be recorded using suitable prompts and interrogation. Each story will be reviewed and the storyteller will be asked to interpret their stories focusing on: 

Meaning,



Learning points and,



Application of the story

5. The storytaker interviews Interviews were also divided into 2 parts where: 

The 7 storytakers (Middle Managers and supervisors ) will be requested listen to the stories



Storytaker’s thoughts will be interpreted based on giving: 

Meaning



Learning points



application

Conducting in-depth interviews follows the same general process as for other research: plan, develop instruments, collect data, analyze data, and disseminate findings. More detailed steps are given below.

5.1 Preparation for Interview The list below will serve as a check list for the interview process: 

Less distracted setting will be chosen.



Main purpose of the interview will be explained.



Terms of confidentiality will address.



Explaining the format of the interview will take place.



Duration of the interview will be estimated.



Contact information of the interviewer will be provided.



Presenter will be given a chance to clarify any doubts about the interview if any.



Prepare a Method for recording data such as tape recording will also be prepared



(V Dapzury & S Pallavi, 2002)



Procedure of the Interview

The following points should be taken into consideration during the interview session: 

Occasionally verify the tape recorder (if used) is working.



One question to be asked at a time.



An attempt to remain as neutral will be practiced.



Responses will be encourages.



Appearance when note taking will be considered.



Transition between major topics will be provided.



Control of the interview is not to be lost.



(V Dapzury& S Pallavi, 2002)

6. Data collection and analysis The results of this study are based on a qualitative investigation of storytelling as an effective way of sharing tacit knowledge between employees. Content analysis was then used to identify coherent themes shared between the storytellers and storytakers. As mentioned in the research methodology section, the sample in

313

Grafton Whyte and Monica Ralake the study was limited to workers at Eskom only. The collected data assisted the researcher to come up with evidence to confirm/affirm the main research question. Data in form of eight stories was collected from four people within the top management at Eskom. Each storyteller had corresponding storytakers from the general workers. For the purposes of confidentiality the researcher will make use of the following names as shown on table 1 below to refer to different people. Table 1: evidence to confirm/affirm the main research question Storyteller Storyteller-1 = Business Planning & Strategy Manager

Storytaker Storytaker-1 - Programme Management Storytaker-2 – Finance

Storyteller-2 = General Manager Storyteller-3= Regional Engineering Manager

Storytaker-3 - Project Service Storytaker-4 - Customer Services Storytaker-5 - Engineering

Storyteller-4= Information Management Manager

Storytaker-6 - Information Management Storytaker-7 – Planning

The recorded conversations from storytellers were then converted into mp3 audio format which in turn was played whilst the storytakers were listening. This format allowed the researcher to play it at different speeds for the storytakers to understand them as they are. The other reason for selecting this audio format was that it allows streaming (does not have to wait for full download). Interviews from four people were recorded and then given to the storytakers to check if they understood the same concepts as shared by the storytellers. Each storyteller had two stories to be shared. The stories were limited to two as each had only 15 minutes and also considering that this was done during working hours the researcher didn’t want to prejudice the company of productive time. An analysis of each story and how the storytaker understood it is presented below.

7. Analysis of stories Storyteller 1, the first two stories were collected from a Business Planning and Strategy Manager. Her first story mainly focused on service environment where she had to provide service to functional departments. Professionalism, commitment to deadlines and marketing were the key learning points of her experience. Storyteller-1mentioned that professionalism entails a number of attributes such as Knowledge, “skilling yourself as an employee, learning what your job is about, what your value chain is about” and making an effort to research about things one might not know or understand. It is very important that no matter what you do in an organisation “You can never let your professionalism drop”. The storyteller also emphasised on negotiating deadlines for people within the service posts and further confirmed that people should be able to agree on realistic deadlines and inform the partners if one cannot meet them for any reason or another. Over delivering was pointed out as one way to impress the customer bearing in mind that “the one time you don’t perform” will be ones downfall as it be always be held against you as a service provider much bigger than all the time you have been performing well. The second story expressed the need to expand your mind as an employee so that not only the business could benefit from that but you as employees can also benefit from it as well. Being flexible and willing to gain knowledge from outside your environment means you are building up experiences which will open paths and more career opportunities. If you opt to limit yourself by only focusing on the job you are hired for or capacitated for then you will not stimulate your mind. Storyteller-1 further stated that choosing to limit yourself will also not “allow you to be creative” and the strongest performer in you environment. She/he has learnt that hard work, flexibility and mind expansion are the key factors to empower yourself in a business. After listening to the recorded conversation Storytaker-1 said that there were a number of things that came up from the story include: 

Being open/willing to learn anything not along career path

314

Grafton Whyte and Monica Ralake 

Exploration opens up different career paths



The feelings of achieving through changing careers

This will come as a profit to the business and on a personal level as one gets rewarded by changing careers within the organisation. The researcher observed strong similarities between the recorded conversations of both the storyteller and storytaker. Both storytakers managed to get it straight as said and more interestingly the listeners were very motivated by the story. According to the Storytaker-2 professionalism is the main meaning conveyed by the story. Agreeing with the storyteller she also suggests that employees need to always behave in a professional manner. Eskom employees need to remember that they are not only colleagues but also internal customers and should provide an efficient service to each other. Committing to deadline was one of the learning points obtained by the storytaker. Storytaker-2 felt the story could be shares on inductions for new employees, also pointed out mentorship as one way that could be used to share experiences. Storyteller-2’s attempt is to depict the usual dilemma faced by many when working under pressure. Storyteller-2 shared his experience when the blackouts hit parts of the Western Cape leaving the whole of Cape Town in the dark, as he was one of the people who were involved in the development of the 2006 Cape Recovery Plan. It took well experienced professionals, with relevant skills to come up with such a solid plan. Commitment and dedication were some of the driving factors for the effectiveness of the plan. Storyteller-2 also highlighted the fact that it is very important to have a good working relationship with both internal and external colleagues at all times. Storyteller-3, The Regional Engineering Manager focused his first story on safety, where he mentioned the importance of always being cautious and wearing necessary personal protective equipment (PPE) when on site or environment that requires wearing one. The storyteller then made an example of his grandmother’s routine where she would do all the necessary safety measure all the time when she was to embark on a journey, and emphasized a point that if his grandmother could manage to have such a routine and stick to it, why can’t Eskom employees do the same especially with necessary resources provided. Telling a story or making examples is better way to enforce regulation within an organisation that is one of his learning points of his experience in the organisation. As he further states stories are not easily forgotten. Story two contained an authentic safety event that was real, relevant and known to some employees in the organization. The story was more based on a successful project that was completed within agreed deadline. Customer complaint was the main reason of initiating the project, as they needed power regardless the of the network crisis that was not capable to take more customers. Because Eskom is more customer focused, the complaint was then taken into consideration such that a plan was put in place to provide power for customers. As complicated as it was to actually commence with the project, it was eventually executed. The project was successfully completed within an agreed-on period. Due to the effectiveness of the implementation process, the business managed to save costs and time. Skill, experience, dedication and communication were pointed as best ingredients for a successful team. Storytaker-5 confirmed that she had to listen to Storyteller-3’s story twice as she couldn’t understand it at first. Having listened to the story the second time she described the story as a safety cautious message. She further confirmed that the employees need to be more proactive than reactive on safety related matters. She emphasised the point that employees need to be safety cautious at all times and not only when they foresee danger. Utilizing safety resources provided by the organisation was a learning point and further relating it to the story where an elderly woman from the most remote village managed to stick to her safety principles, employees need to do the same. According to Storytaker-5 the story is worth being shared throughout the organisation. She believes that executives are in better position to spread his story and suggested that safety related functions or workshop are the best places to share the story as an opening statement. Trust and believing in your customer was one factor that Storyteller-4 emphasised in his first story. His story was about a customer who could be judged as unknowledgeable due to the dress code and the way the customer presented himself. According to the storyteller, the customer came to Eskom offices and requested

315

Grafton Whyte and Monica Ralake one megawatt which was unreasonable as a small town in that province consumes half megawatt. But then again the customer had done his research and knew exactly what he wanted. As one of his highlight experiences the storyteller mentioned the importance of politeness, trusting the customer and trying not to judge customer by their appearance. The second story emphasised on proper planning and not making decisions based on irrelevant factors. Storyteller-4 shared a story where they went to one of the provinces to construct a power line and because they were tired when they got there, the colleagues wanted to construct the line without thinking about the future. In that instance Storyteller-4 argued that in the event of natural hazards like floods the power line would be damaged and convinced them to follow the decision. After ten years it happened that there were floods in that province and that power line is the only one that was not affected Storytakers were very excited and humbled that s/he got the pleasure of listening to a story from one of the most experience executive in the business. Prior to providing feedback on the story, Storytaker-6 thanked the researcher for selecting him, and further expressed how privilege s/he is to be the one listening. Storytaker-6 described the story as great example say â&#x20AC;&#x153;judging a book by its coverâ&#x20AC;? and further confirmed that in fact it is the main meaning of the story for her/him. The storytaker related the story to different sections within the business where employees treat each other differently because of grade or seniority. The story taught him that employees should treat their customers both internal and external as first class customers where the best services are provided. Storytaker-6 believed that the story is valuable and communication department can distribute the recording as means of sharing knowledge and learning from past experiences The storytaker was quite touched with the story as it delivered a number of learning points to all workers. Teamwork and planning was identified as the most important factors leading to effective organisation. The storytaker identified pockets of individualism at the organisation. He gave example of workers who had stayed with the organisation for more than twenty years as the ones with a lot to share based on experiences and the organisation should take advantage of that. The storytaker thinks that if all people within the organisation had time to share experiences then the business will be successful and workers will benefit as well.

8. Conclusions and recommendations This research investigated the question. Is storytelling an effective means of sharing tacit knowledge? Specifically, the study looked at the technical knowledge conveyed by a storyteller and the knowledge acquired by a storytaker to understand the degree of congruence between the two parties. The data collected provided confirmatory evidence to support the notion that storytelling is an effective means of sharing tacit knowledge within an organisation. In fact the degree of congruity between storytellers and storytakers and the potential to motivate storytakers was impressive opening up some exciting avenues for further investigation. One such opportunity is the possibility that fairly dry safety regulatory knowledge could be enlivened and made more interesting by embedding them into real life stories, a process which could significantly aid retention. The selection of company executives as the main storytellers had very positive effect on the transfer of knowledge as there was an eager anticipation on the part of storytakers to learn from these senior practitioners. Other factors such as the skill of the storyteller and the nature of the story were other factors that seemed to impact the transfer of knowledge in the story. The methodology adopted in this research focused on 3 criteria to test effectiveness; (i) meaning, (ii) lessons learned and (iii) application of the story. The reason for these tests is firstly, for a story to have any use at all it needs to be understood (meaning). Knowledge transfer is achieved if storytakers can discern the lessons to be learned from stories, but this is no guarantee the knowledge will be effective. It is only the ability of storytakers to apply lessons learned to their own circumstances that begins to address questions of effectiveness. This breakdown proved useful for eliciting understanding from both storytellers and storytakers and subsequently for analysing and comparing their responses. Nearly all the storytakers were able to successfully apply learning from stories to their own situations.

316

Grafton Whyte and Monica Ralake The data collection process would provide a useful model for collecting stories within large organisations, the process employed here was quite simple and could be replicated and scaled-up quite easily. Training interviewers would also be quite painless. The challenge would be deploying or sharing this knowledge throughout organisations in a useful way, when needed. The technology is not the issue as there is no shortage of software tools on the market, one such solution are in-house wikis. The authors suggest there are two basic approaches to this problem of deployment that organisations need to consider: Push approaches – here stories could be used as part of in-house training programmes to create scenarios for which participants need to devise solutions. One useful application of this approach is induction training usually given to new employees. On-demand (or Pull) approaches – here organisations actively market throughout the organisation repositories or wikis that employees can refer to that contain stories or other knowledge artefacts. The generalizability of the findings in this study is limited by the fact that it was conducted within a single organisation, Eskom a South African state-owned enterprise. Therefore, further research is needed that looks at more diverse sample of stories from storytellers and storytakers, and perhaps spread across multiple organisations and industries. Finally, the contribution of this research is that it is the first, in the knowledge of the authors, to investigate the effectiveness of storytelling as a means of sharing tacit knowledge between storytellers and storytakers in an organisational setting.

References Adaval, R & Wyer R. S., (1998) “The role of narratives in consumer information Processing”, Journal of Consumer Psychology 7 (3), pp. 207 - 245Bailey, T. (2005), “The Story of JPL Stories” , ASK Magazine. Available at: http://askmagazine.nasa.gov/issues/11/special/index.html accessed on 12th Jan 2011. Boje M.D 2000, “Narrative Methods for Organizational and Communication Research”, Introduction to Narrative Methods. Browning, L.D., (1992), “Lists and stories as organizationalcommunication”,Communication Theory, Vol. 2 No.2, pp.281-302. Dapzury, V. andPallavi, S., (2002), “Interview as a Method for Qualitative Research” Southern Cross University and the Southern Cross Institute of Action Research (SCIAR) http://www.public.asu.edu/~kroel/www500/Interview%20Fri.pdf accessed on the 15 March 2011 Davenport, T.H., Delong D.W., and Beers, M.C., (1998). “Successful Knowledge Management Projects”. Sloan Management Review, Vol. 39, No. 2. Denzin, N. K. and Lincoln, Y.S., (2005).”The SAGE Handbook of Qualitative Research Third Edition” SAGE Publishing, California. Feldman, S. P., (1990). “Stories as Cultural Creativity: On the Relation between Symbolism and Politics in Organizational Change.” Gartner (2009), “Case Study: 3M uses storytelling to Uncover Tacit Knowledge”, ID Number: G00162392 Hussay, J. and Hussay, (1997), “Business Research: A Practical Guide for Undergraduate and Postgraduate Students London”, MacMillan Press Ltd Lehaney, B. Clarke, S. Coakes, E .and Jack, G., (2004), “Beyond Knowledge Management”, Idea Group Publishing, USA; pp 14-23 Ma, K. F. and Keppell, M., (2004). ”Knowledge management: The relevance of storytelling in the management of knowledge in organisations” O’Neill, J.W., (2002), “The Role of Storytelling in Affecting Organizational Reality in the Strategic Management Process”, The Journal of Behavioral and Applied Management – Summer/Fall 2002 – Vol. 4(1) Santoro, F. and Brezillon, P., (2005), “Group Storytelling Approach to Collect Contextualized Shared Knowledge” , In the Proceedings of the 16th International Workshop on Database and Expert Systems Applications, DEXA ’05 22 – 25 August 2005, Copenhagen, Denmark, available at (http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1508303&isnumber=32313> Viewed on 20 November 2010 Swap W, Leonard D, Shields M and Abrams L, (2001) “Using Mentoring and Storytelling to Transfer Knowledge in the Workplace,”, Journal of Management Information Systems 19 (1), pp. 95 – 114 Taifi, N., (2010), “Tacit Knowledge and the Knowledge Management Systems” <http://knowledgemanagement-review.blogspot.com/2010/04/tacit-knowledge-management.html > Viewed on 20 March 2011

317

The Role of is in Residential Energy Management Ming Hock Yew, Alemayehu Molla and Vanessa Cooper RMIT University, Melbourne, Australia minghock.yew@rmit.edu.au Abstract: While the increase in worldwide residential energy consumption is a concern, a larger concern is the breakdown of residential energy use which is shifting towards life‐style oriented appliances and devices. Life‐style appliances and devices drive the biggest growth in residential energy consumption making them a concern requiring intervention, given also that life‐styles are behavioural in nature with potential for modification. Previous research involving use of IS in residential energy management has not benefited from new technologies, in particular new information technologies (IT). Thereby, the full potential of information systems (IS) is still not harnessed for residential energy management warranting research into its roles in order to guide design and deployment of IS for improved residential energy consumption. The objective of this research is to study the roles of IS in residential energy management. By providing a clear role of IS in residential energy management, we set the grounds for further research into a framework for the design and deployment of a residential energy management IS in order to improve on residential energy efficiency. The paper commences with a review of the factors contributing to residential energy consumption and also use of IS in residential energy management. The roles of IS in residential energy management are then derived from the functional affordance based on studies into the four areas of residential energy management process, technologies used in residential energy management, actual Implementation of IS for residential energy management and environmental behavioural theories and models. From the research it is found that IS for residential energy management fulfils the overarching IS roles of automating, informating and transforming processes. Of these, a specific key role found is in the channelling of interventions to the residential energy end user in order to modify behaviour towards improved energy efficiency. Keywords: residential energy, energy informatics, environmental behaviour, energy behaviour, green IS, green IT

1. Introduction: Residential energy consumption ‐ what and how big is the problem anyway? Fourteen percent of world‐wide energy end consumption is attributable to the residential sector and this is growing by 1.1 percent each year (EIA 2012). In comparison, 41.5 percent of world‐wide electricity (a major form of energy used) is used by industry and its share has decreased from 53.5 percent in 1973 to 41.5 percent in 2010 (IEA 2012). While the increase in residential energy’s share of total energy consumption is a concern, the bigger concern is the breakdown of residential energy consumption which is shifting towards more life‐ style oriented appliances and devices as evidenced in the case in Australia (Petchey 2010). This trend decreases energy efficiency and necessitates intervention, as life‐styles are behavioural in nature with potential for modification. Residential energy users feel the impact when energy cost increases, both due to increased consumption and increased energy price. In the state of Victoria (Australia), “electricity prices have increased by an average 10 percent in the past year and by an average 33 percent over the past five years. Gas prices increased by 6 percent in the past year and 22 per cent over five years” (Wells 2012). Management of residential energy consumption has been carried out mainly via the economic approach (pricing, tariffs, or energy efficiency labelling) policies (Zhang, Siebers et al. 2012). The effectiveness of this approach has not been widely researched but the increase in residential energy consumption does provide an indication of the limited success of economic based management approach. At the same time, research on residential energy consumption adopted either an analytical approach (Hirst, Goeltz et al. 1981) or a modelling approach (Swan and Ugursal 2009, Zhang, Siebers et al. 2012). Theories and models of environmental/energy behaviour were also used in some research (Lehman and Geller 2004, Fischer 2008, Matthies 2009) to explain residential energy consumption. These researches provided useful inputs for managing residential energy consumption towards overall consumption reduction. This paper studies the use of information system (IS) in residential energy management to determine its role and hence its value adds. Residential energy use is a form of energy end use which is differentiated from energy generation/transformation and transmission. It also concerns energy use in a personal capacity as opposed to energy use in a business setting. The ensuing sections look at factors contributing to residential energy consumption and where IS is used before discerning its role. For consistency in terminology, energy use refers to manner or kind of application of energy, where else energy consumption refers to the quantity of energy used ((ISO) 2011).

318

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper

2. Factors contributing to residential energy consumption The first challenge facing policy makers or researchers concerned with managing residential energy use is determining and understanding the factors contributing to residential energy use. On the surface residential energy use appears to be attributable to many inter‐related factors. A comprehensive understanding of these factors is key to determining the role and value of IS in managing residential energy use. Hirst et al. (1981 Hirst, Goeltz et al. (1981) first proposed a relationship relating residential energy use to three categories of factors: Energy Use = f (capital stock, technical efficiency, behaviour). Capital stock is defined as “the structure in which they live and the energy‐using equipment they own” while technical efficiency is defined as “the ways in which the household uses (operates) that stock” (Hirst, Goeltz et al. 1981). Both capital stock and technical efficiency are in turn influenced by technological factors such as house construction technology and energy efficient design of appliances. These groups of factors were used in modelling approaches such as when Seligman and Darley (1977) introduced bottom‐up and top‐down modelling approaches to study residential energy use. Their models are based on common determinants of energy use comprising of housing stock (a part of capital stock), appliances (type and quantity), climate and the residents themselves. The resident factor was seen to be driven by energy related policies, economic factors (such as energy price), household profile, and energy use behaviour. Other researches also proposed a mix of overlapping factors, with Van Raaij and Verhallen (1983) looking at similar contributing factors comprising of socio‐demographic factors, family life‐style, energy prices, energy‐related behaviour, cost‐benefit trade‐offs, effectiveness and responsibility, feedback, information and home characteristics. Some researchers were more focussed on a specific set of contributing factors. This includes Bin and Dowlatabadi (2005) who proposed an alternative paradigm, called the Consumer Lifestyle Approach (CLA), to explore the relationship between consumer activities (energy consumption) and its environmental impacts, although consumer lifestyle is ultimately behaviour driven. Energy use can be seen in the larger context of environment behaviour in order to better understand the psychology behind it. Kollmuss and Agyeman (2002), argued that “the question of what shapes pro‐ environmental behaviour is such a complex one that it cannot be visualized through one single framework or diagram”. Given the complexity, the environmental behaviour (or aspects of it) studied have ranged widely, with all research reviewed studying a different behaviour or aspect. Pro‐environment and environmental detrimental behaviour were researched based on norms and attitudes by Pelletier and Dion (1999), Matthies (2009), Steg and Sievers (2000), Stern (2000) and De Groot and Steg (2007). Lehman and Geller (2004) took a different tack and focused on the interventions which they felt were important in shaping environmental friendly behaviour, suggesting intervention strategies to increase a variety of pro‐environment behaviours and decrease behaviours that damage the environment. These intervention strategies can be broadly categorised into antecedent strategies (information/education, verbal or written prompts, models and demonstrations, commitment, environmental alterations) and consequence strategies (rewards and feedback). The extant research has shown that residential energy consumption is a function of factors related to capital stock, climate, relevant economic factors, energy related policies, and the resident (behaviour and household profile). This understanding then allows us to look further at better management of residential energy use and the roles IS plays in it.

3. Use of IS In residential energy management An Information system (IS) ties together the various elements of an information to provide a complete knowledge solution (Watson, Boudreau et al. 2010). Three overarching roles of IS has been identified as automate, informate (informate up and informate down) and transform (Zuboff 1985, Chen, Boudreau et al. 2008). Use of IS for residential energy consumption can be examined from both the energy end user perspective (the perspective of interest here) and the energy provider and regulator perspective. From the research on factors contributing to residential energy management climate, economic factors, and energy related policies are external factors beyond the control of the residential energy user. In these areas governmental departments and utility providers have leveraged on IS to manage energy provision and consumption. Powered by IS, smart grids are now becoming mainstream. E‐government infrastructure has also leveraged on IS for residential energy billing as well as policy dissemination and enforcement. However, these represent use of IS for

319

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper residential energy management by the residential energy providers or regulators, and not so much by the residential energy users themselves. Capital stock (referred to in the previous section) is partially within the control of the residential energy user as far as decision to purchase the house and appliances are concerned. Conveying information to aid the purchase decision and to some extent automating the purchase process are roles of IS here. In comparison, behaviour and household profile are internal factors in which the residential energy user would have conscious or unconscious control over behaviour. In this aspect, the role of IS is not too clear as research has focussed more on residential energy use rather than the use of IS to manage this behaviour. This includes research which attempted to explain residential energy use via models by Fischer (2008), Chen, Cook et al. (2012), and Bin and Dowlatabadi (2005). Other researchers analysed residential energy consumption directly with the help of statistics in order to determine the contributing factors and their significance (Bin and Dowlatabadi 2005). A number of the researches (Seligman, Darley et al. 1978, Midden, Meter et al. 1983, Darby 2000, 2006, Fischer 2008) investigated the effectiveness of feedback through provision of information on energy consumption, and this was found to have a significant impact on effective residential energy management. Darby (2006) and Fischer (2008) showed that among the interventions that are aimed to improve residential energy efficiency, feedback offers the most potential in amending the consciousness of energy user and modifying energy use behaviour for improved energy efficiency. This is where IS potentially plays a central role in channelling such feedback. Most of the residential energy management systems investigated in previous studies prior to 2009 has not benefited from new technologies, in particular new information technologies (IT). The researchers, at that time, faced considerable constraints in acquiring cost effective smart meters for wide‐scale field research which Fischer (2008) had quoted as “unlikely to be installed widely” at the time of her research. Burgess and Nye (2008) also allured to technological hurdles for the widespread roll‐out of smarter metering The situation is now different with smart plugs/sockets costing less than AUD50 (Jaycar 2012) allowing consumers to self‐ monitor electrical energy usage at appliance level. Fischer (2008) had also allured to research gaps in studies on energy consumer feedback done concerning the small sample sizes of the studies and projects and timeliness of measurement and feedback. A possible reason for the small sample sizes could be the cost of carrying out field research where considerable hardware and software was required. Hardware required includes metering devices, networking equipment, input/output (I/O) devices, data storage and computing equipment. The software required manages the hardware, database and also provides computing power for residential energy management. In this respect, mobile and cloud computing could be leveraged on to reduce the need for extensive hardware and software. In mobile computing smart phones, which have become ubiquitous these days, eliminate the need for dedicated displays to report energy consumption and provide related information. Cloud computing eliminates the need for on premise servers or computers, eliminating another cost hurdle in the use of IS for residential energy management. Testimony of the cost advantage to be leveraged from cloud computing in residential energy management can be seen in the implementation of cloud based residential energy management IS by Google (PowerMeter) (Google 2011) and Microsoft (Hohm) (Mundie 2012) which the two technological giants were able to provide for free to the residential energy users. While promising, use of IS for residential energy management is not a widely diffused practice, with correspondingly little research in this area. Antecedent to use of IS for residential energy management, an understanding of the roles that IS can play in managing residential energy use needs to be first established. This allows better crafting of policies and also further research into residential energy management.

4. Discerning the roles of IS in residential energy management The roles of IS in residential energy management could be discerned from an analysis of its functional affordance. Hartson (2003) defined functional affordance as “a design feature that helps users to accomplish work (i.e. the usefulness of a system function)”. For residential energy management, work could be seen in the context of attaining energy efficiency. The IS design feature that helps the residential energy user attain energy efficiency hence constitutes its functional affordance. Four areas of study are used to identify the IS design features for functional affordance:

The residential energy management process

320

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper

Technologies used in residential energy management

Actual implementation of residential energy management IS

Environment behaviour theories and models

An understanding of the residential energy management process would yield requirements that translate into functional affordance. Research on the technologies used in IS for residential energy management allows a nomological mapping of these technologies to EI technologies defined by Watson, Boudreau et al. (2010). This then leads to an understanding of the functional affordance of IS for residential energy management from the assimilated use of these technologies. A study of full system implementation of an IS for residential energy management as well as environmental behaviour theories and models allows a review of the functional affordance of this form of IS. On top of proposing environmental behavioural theories and models Lehman and Geller (2004), Matthies (2009) and Fischer (2008) had also highlighted the value in the use of IS in conjunction with their theory or model leading to the identification of functional affordance of this form of IS. Functional affordance identified here leads us to the discovery of roles IS can play in residential energy management. The relationship between these four areas of study, functional affordance and roles of IS in residential energy management, which represents the methodology in identifying the roles of IS in residential energy management, is illustrated in figure 1 below.

Figure 1: Methodology in identifying roles of IS in residential energy management Before considering the role of IS in residential energy management, this form of IS should first be recognized as a form of energy informatics (EI) which is defined as “analysing, designing, and implementing information systems to increase the efficiency of energy demand and supply systems” (Watson, Boudreau et al. 2010). However, energy informatics has not been thoroughly researched (Watson, Boudreau et al. 2010) and this applies for both business and residential energy. Watson, Boudreau et al. (2010) further describes three types of technology that are present, or should be present, in an intelligent energy system: flow networks, sensor networks, and sensitized objects. A flow network is “a set of connected transport components that supports the movement of continuous matter”. A sensor network is a “set of spatially distributed devices that reports the status of a physical item or environmental condition”. A sensitized object is a “physical good that a consumer owns or manages and has the capability to sense and report data about its use”. A review of the technologies used in the IS for residential energy management to nomologically map these against the EI constituent technologies is hence in order. This provides a clearer sense of the role of this form of IS as a type of EI. On top of this, the three overarching IS roles of automate, informate (informate up and informate down) and transform are nomologically mapped to the functional affordances of IS for residential

321

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper energy management in order to categorize each role that IS plays and allow further research into each of these category of roles.

5. Functional affordance based on the residential energy management process The residential energy management process, from the perspective of the residential energy user, is first reviewed in order to better understand the functions required of IS. A simplified representation of the process flow can be seen in figure 1. Energy Use Intervention • Feedback on energy consumption • Information/feedback from other sources (e.g. energy bill/policies) Inputs to residential energy user Energy Consumption Capital Stock Decision • Appliance Usage Making • House and Appliance Acquisition

Energy Consumption Data Collection

Figure 2: Residential energy management process The process starts with a once off decision on capital stock. This is where the residential energy user decides on the type and specification of house and appliance to acquire, which has bearings on energy consumption. Energy consumption takes place as the appliances are used. Concurrent with energy use, energy consumption data is collected for billing or analysis purpose. Data collection could be via manual meter reading or smart meters connected to the utility provider. Together with inputs from external sources (such as regulatory body or utility providers) feedback is provided to the user while making capital stock decision or during energy use. This feedback could be in the form of energy bills or any other energy related information and could be sent via hardcopy letters or electronic means. From the study of the residential energy management process, it can be seen that the information related processes can be distilled as the following functional affordances:

energy consumption data acquisition and compilation (automate role)

informating (down) the energy consumption data

transforming residential energy use.

Transforming is achieved by improving the visibility of energy consumption and also modifying the energy use behaviour towards energy efficiency. Segregating the roles of IS in residential energy management based on these three overarching IS roles allows more focussed studies along these lines in order to realize fully the roles IS plays or can play in residential energy management.

6. Functional affordance based on technologies used in residential energy management In essence an IS for residential energy management can be constituted of four technological areas: metering, input/output (I/O), network/communications, and computing technologies. These would facilitate the collection and compilation of residential energy consumption data as well as reporting of energy consumption and interaction with the residential energy user. Previous research on the use of IS for residential energy management tends to focus on specific technologies and not the system as a whole. For instance, in residential energy management, Darby (2006) reviewed sensitized objects in the form of smart meters as well as display technology, emphasizing the importance of these technologies in the quality and quantity of feedback to the energy user, leading to better energy management. Wireless sensor networks were researched by Erol‐ Kantarci and Mouftah (2011) and network communication gateways were investigated by Kushiro, Suzuki et al. (2003) to ensure optimum network communications for the IS. These are but parts of the whole picture which need to be rationalized and assimilated in order to realize a total understanding of an IS for residential energy

322

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper management and its functional affordance. The four technological areas for residential energy management IS are tabulated in table 1 in relation to the EI three technologies defined by Watson, Boudreau et al. (2010), Table 1: Technological areas and component technologies of residential energy management IS

Technological Area

Component Technologies

Metering technologies

Smart meters Smart sockets / plugs Other sensor devices (eg thermocouples/thermometers, hygrometer) Smart appliances Displays (LCD, LED, touchscreens) Mobile devices (smartphones, tablets)

Input/output (I/O) and mobile technologies 3 Network/commu nications technologies

Computing technologies

Wireless communication devices (Wireless access point, routers, switches) Power‐line communication devices (routers, switches) Mobile internet devices (modems, dongles, smart phones) Common data communication protocols (eg TCP/IP) and data interchange format (eg XML) Cloud computing (SaaS, PaaS) Interactive mobile applications Interactive web applications Social media (Facebook applications)

Energy Informatics (EI) Technology Sensitized objects

Part of sensor network Part of sensor network

Part of sensor network

Literature

Darby (2010); Efthymiou and Kalogridis (2010); Fischer (2008); Hargreaves, Nye et al. (2010); Molina‐Markham, Shenoy et al. (2010) Darby (2006); Fischer (2008); Roth and Brodrick (2008) Kushiro, Suzuki et al. (2003); Erol‐Kantarci and Mouftah (2011)

Mundie (2012); Google (2011); Nader (2011); Pitt, Parent et al. (2011); Wang, de Groot et al. (2009)

Each of the EI technologies are vital for complete functionality of an IS for residential energy management and hence make up its functional affordance. The functional affordance of the IS as a whole can thus be derived its functions as a:

sensitized object: where the IS sense and report the state of appliance connected to the meters and energy consumed through them (data about its use)

sensor network: where the IS reports the status of the appliance being monitored (physical item) or the energy consumed (environmental condition).

7. Functional affordance based on implementation of IS for residential energy management Two technology giants rolled out their cloud based IS for residential energy management in an effort to test the technology in the market. These were Microsoft’s Hohm (launched on 24 June 2009 by Microsoft) and Google’s PowerMeter (launched on 5 October 2009 by Google) (LaMonica 2010, 2010, Nader 2011). In both cases, these were free energy monitoring tools to raise awareness about the importance of giving people access to their energy information. Google PowerMeter included key features like visualization of residential energy consumption, the ability to share information with others, and personalized recommendations to save energy. Microsoft Hohm also went beyond energy usage reporting and provided savings recommendations, which could range from placing new caulking on windows to removing air leaks to installing a programmable thermostat. These recommendations are tailored based on specific circumstances in the consumer’s home including house features, usage patterns and appliances. In addition, consumers were able to compare their energy consumption with that of others in their area and connect with the Microsoft Hohm community to find referrals and exchange ideas. These implementation of residential energy management system IS somewhat widens its functional affordance beyond just monitoring and reporting energy consumption, but to include energy conservation knowledge and advice as well. They largely play an informate role with some elements of an automate role by way of automating energy use data collection, surveys and to solicit inputs from energy users. Besides

323

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper automation of energy and user information processing, IS can be used to monitor and control household systems and appliances. This was a role played by Fujitsu’s in new apartment projects in Australia (Fujitsu 2012) and Panasonic also in new apartment projects in Singapore (Zengkun 2011) here again introducing a new dimension to the functional affordance of this IS. The functional affordance, based on actual implementation, and framed against the three overarching roles of IS can thus be summarized as:

automating the monitoring and reporting of energy consumption

informating on energy conservation knowledge and advice

automating the monitoring and control household systems and appliances.

8. Functional affordance based on environmental behavioural theories and models In the previous section, residential energy use was found to be a function of capital stock, climate, relevant economic factors, energy related policies, and the resident (behaviour and household profile). For external factors beyond the control of the residential energy user, IS plays a passive role by channelling information on climate, economic factors, and energy related policies to the residential energy user. IS plays a more crucial and active role in internal factors that are within the control of the residential energy user such as behavioural modification. Previous research alluded to a critical role played by IS in channelling interventions to modify residential energy use behaviour towards improved energy efficiency. For instance, Fischer (2008) concluded that the need for frequent feedback on actual consumption with interactivity and granularity extending to appliance‐specific breakdown makes “smart” metering, electronic data processing and communication an especially useful. Feedback to the residential energy user constitutes part of the behaviour modification approach proposed in social and behavioural research (Darby 2000, Lehman and Geller 2004, 2006, Fischer 2008, Matthies 2009) with regards to an individual’s environmental or energy behaviour. Lehman and Geller (2004) suggested intervention strategies to increase a variety of pro‐environment behaviours and decrease a variety of behaviours that damage the environment. The interventions, in particular feedback on energy use, point to an infomate role which IS can fulfil in managing residential energy and can be organized as follows:

Antecedent Strategies ‐ (a) information/education, (b) verbal or written prompts, (c) modelling and demonstrations, (d) commitment, (e) environmental alterations

Consequence Strategies – (a) rewards, (b) feedback

Fischer (2008) presented a heuristic model of environmental decision making (figure 3), based on the norm activation theory, where behavioural outcome is seen as a result of both conscious and habitual behaviour. In conscious behaviour, norm activation precedes any motivation for action. Norm activation is made up of three areas of consciousness (environmental problem, relevance of one’s behaviour and own sense of control). Only with this set of consciousness will the person reflect on changing his or her behaviour in order to solve the problem. Upon being conscious of the problem (norm activation completed) the next phase comprising of different motives (personal norms, social norms and “other motives” such as minimizing costs) for action follows. Norms may conflict with each other or with other motives, which brings the person to the next phase of evaluating costs and benefits (both in the tangible and intangible sense). During the evaluation phase, norms and motives may also be redefined in with the availability of new information. For example, in some cases marketing campaigns might trigger impulse action. Decisions on energy use can be categorized by:

How energy is used (intensity, mode of usage). This relate to curtailment behaviour (repetitive efforts to reduce energy use) such as lowering thermostat settings (Abrahamse, Steg et al. 2005).

What appliances to purchase or upgrade. This is a once‐off behaviour that impacts efficiency of the equipment

Specific actions arise after evaluation phase and if these are performed regularly they could develop into new routines or habits. Habitual behaviour is therefore seen as a perpetual influence to day‐to‐day action affecting the outcome. Routinized or habitual behaviour that is not energy efficient leads to sub‐optimal energy use (Fischer 2008). The only way to break out of habitual behaviour is to form new norms which is where energy end users transit from subconscious (habitual) behaviour to conscious behaviour (norm activation). Such a theory relies heavily on feedback to modify behaviour and as such would be suitable for intervention strategies suggested by Lehman and Geller (2004)

324

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper

Figure 3: Heuristic model of environmentally relevant behaviour (adapted from Fischer (2008)) Integrating intervention strategies and the heuristic model of environmental relevant behaviour realizes a set of functional affordance for the IS used in residential energy management as illustrated in figure 4.

Figure 4: Integration of a behavioural theory and concept The integration is achieved in several way, as described based on the functional affordances of the IS – (a) automating energy use data collection including surveys to solicit inputs from energy end users and manage user learning, (b) informating the residential energy user by way of providing feedback on energy consumption (informate down) or utility company or energy policy maker by way of reporting energy consumption (informate up, (c) monitoring and controlling household systems and appliances. The intervention strategies steers the residential energy user (norm activation, motivation, evaluation and action/behaviour processes) towards energy efficient behaviour, consciously or sub consciously given a time period long enough for environmental detrimental habits to be broken, hence playing a transform role.

9. Conclusion The role of IS in residential energy management, derived from functional affordance of such IS can thus be summarized as in table 2 below. These roles were discussed in the context of the three technologies of EI (Watson, Boudreau et al. 2010) and three overarching roles of IS (Zuboff 1985, Chen, Boudreau et al. 2008). IS for residential energy management fulfils the automate, informate and transform roles. Functional affordance based on the various studies reveals common roles for such as IS which can be summarized as:

automating the energy consumption data acquisition, compilation and reporting (automate role)

automating the monitoring and control of household systems and appliances

325

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper

informating (down) the energy consumption data and energy conservation advice to the residential energy user and informating (up) energy consumption data to regulatory bodies or utility providers

transforming residential energy use by modifying energy use behaviour towards energy efficiency

Of these, a key role found is to channel interventions to the energy end user (informate down). This research contributed by determining key roles of IS in residential energy management framed along the three overarching roles of IS. This sets the grounds for further research into a framework for the design of a residential energy management information system in order to improve on residential energy efficiency. Table 2: Roles of IS in residential energy management Functional Affordance Based On Residential energy management process Technologies Used in Residential Energy Management Actual Implementation of IS for Residential Energy Management Environmental Behavioural Theories and Models

Roles of IS in Residential Energy Management energy consumption data acquisition and compilation (automate role) informating (down) the energy consumption data transforming residential energy use. sensitized object: where the IS sense and report the state of appliance connected to the meters and energy consumed through them (data about its use) sensor network: where the IS reports the status of the appliance being monitored (physical item) or the energy consumed (environmental condition). automating the monitoring and reporting of energy consumption informating on energy conservation knowledge and advice automating the monitoring and control household systems and appliances. automating energy use data collection including surveys to solicit inputs from energy end users and manage user learning informating the residential energy user by way of providing feedback on energy consumption (informate down) or utility company or energy policy maker by way of reporting energy consumption (informate up) monitoring and controlling household systems and appliances. Steering the residential energy user towards energy efficient behaviour (transform role).

References (ISO), I. O. f. S. (2011). ISO 50001: 2011. Energy management sysetms ‐ Requirements with guidance for use. Abrahamse, W., et al. (2005). "A review of intervention studies aimed at household energy conservation." Journal of Environmental Psychology 25(3): 273‐291. Bin, S. and H. Dowlatabadi (2005). "Consumer lifestyle approach to US energy use and the related CO< sub> 2</sub> emissions." Energy Policy 33(2): 197‐208. Burgess, J. and M. Nye (2008). "Re‐materialising energy use through transparent monitoring systems." Energy Policy 36(12): 4454‐4459. Chen, A. J. W., et al. (2008). "Information systems and ecological sustainability." Journal of Systems and Information Technology 10(3): 186‐201. Chen, C., et al. (2012). "The user side of sustainability: Modeling behavior and energy usage in the home." Pervasive and Mobile Computing(0). Darby, S. (2000). "Making it obvious: designing feedback into energy consumption." Proc. Energy Efficiency in Household Appliances and Lighting. Darby, S. (2006). "The effectiveness of feedback on energy consumption." A Review for DEFRA of the Literature on Metering, Billing and direct Displays 486. Darby, S. (2010). "Smart metering: what potential for householder engagement?" Building Research & Information 38(5): 442‐457. De Groot, J. and L. Steg (2007). "General beliefs and the theory of planned behavior: The role of environmental concerns in the TPB." Journal of Applied Social Psychology 37(8): 1817‐1836. Efthymiou, C. and G. Kalogridis (2010). Smart Grid Privacy via Anonymization of Smart Metering Data. Smart Grid Communications (SmartGridComm), 2010 First IEEE International Conference on. EIA, U. (2012). Residential Energy Consumption Survey (RECS). Erol‐Kantarci, M. and H. T. Mouftah (2011). "Wireless sensor networks for cost‐efficient residential energy management in the smart grid." Smart Grid, IEEE Transactions on 2(2): 314‐325. Fischer, C. (2008). "Feedback on household electricity consumption: a tool for saving energy?" Energy Efficiency 1(1): 79‐ 104. Fujitsu (2012). Sustainability Case Study. Lend Lease – Convesso Concavo Project. Fujitsu Australia and New Zealand. Google (2011). "Google PowerMeter: A Google.org Project ". Retrieved 24‐9‐12, 2012, from http://www.google.com/powermeter/about/. Hargreaves, T., et al. (2010). "Making energy visible: A qualitative field study of how householders interact with feedback from smart energy monitors." Energy Policy 38(10): 6111‐6119.

326

Ming Hock Yew, Alemayehu Molla and Vanessa Cooper Hartson, R. (2003). "Cognitive, physical, sensory, and functional affordances in interaction design." Behaviour & Information Technology 22(5): 315‐338. Hirst, E., et al. (1981). "Residential energy use and conservation actions: Analysis of disaggregate household data." NASA STI/Recon Technical Report N 81: 23686. IEA (2012). Key World Energy Statistics, International Energy Agency. Jaycar (2012). Retrieved 18 Dec 2012, 2012, from http://www.jaycar.com.au/productResults.asp?MID=3&SSUBID=840&SUBCATID=1003&keyform=CAT2#3. Kollmuss, A. and J. Agyeman (2002). "Mind the gap: why do people act environmentally and what are the barriers to pro‐ environmental behavior?" Environmental education research 8(3): 239‐260. Kushiro, N., et al. (2003). "Integrated residential gateway controller for home energy management system." Consumer Electronics, IEEE Transactions on 49(3): 629‐636. LaMonica, M. (2010). "Google PowerMeter tracks home electricity via Wi‐Fi." Retrieved 20‐8‐2012. LaMonica, M. (2010). "Microsoft Hohm scores home energy efficiency." Retrieved 20‐8‐2012, 2012. Lehman, P. K. and E. S. Geller (2004). "Behavior analysis and environmental protection: Accomplishments and potential for more." Behavior and social issues 13(1): 13‐32. Matthies, E. (2009). Planning Interventions on a Theoretical Basis. Application of a new integrative influence model of pro‐ environmental behaviour to promote energy efficient behaviour. Conference Energy Efficiency and Behaviour. Midden, C. J. H., et al. (1983). "Using feedback, reinforcement and information to reduce energy consumption in households: A field‐experiment." Journal of Economic Psychology 3(1): 65‐86. Molina‐Markham, et al. (2010). Private memoirs of a smart meter. Proceedings of the 2nd ACM Workshop on Embedded Sensing Systems for Energy‐Efficiency in Building. Zurich, Switzerland, ACM: 61‐66. Mundie, C. (2012). "Hohm Helps Consumers Save Money and Energy." Retrieved 20‐8‐2012, 2012, from http://www.microsoft.com/en‐us/news/press/2009/jun09/06‐24EnergyUsagePR.aspx. Nader, W. (2011). "Real‐time Power Monitoring, Home Automation and Sustainability." Pelletier, L. and S. T. Dion (1999). "K., & Green‐Demers, I.(1999). Why do people fail to adopt environmental protective behaviors? Toward a taxonomy of environmental amotivation." Journal of Applied Social Psychology 29: 2481‐2504. Petchey, R. (2010). End use energy intensity in the Australian economy. Pitt, L. F., et al. (2011). "Integrating the smartphone into a sound environmental information systems strategy: Principles, practices and a research agenda." The Journal of Strategic Information Systems 20(1): 27‐37. Roth, K. and J. Brodrick (2008). "Home energy displays." ASHRAE Journal 50(7): 136‐137. Seligman, C. and J. M. Darley (1977). "Feedback as a means of decreasing residential energy consumption." Journal of Applied Psychology 62(4): 363. Seligman, C., et al. (1978). "Behavioral approaches to residential energy conservation." Energy and Buildings 1(3): 325‐337. Steg, L. and I. Sievers (2000). "Cultural theory and individual perceptions of environmental risks." Environment and Behavior 32(2): 250‐269. Stern, P. C. (2000). "New environmental theories: toward a coherent theory of environmentally significant behavior." Journal of social issues 56(3): 407‐424. Swan, L. G. and V. I. Ugursal (2009). "Modeling of end‐use energy consumption in the residential sector: A review of modeling techniques." Renewable and Sustainable Energy Reviews 13(8): 1819‐1835. Van Raaij, W. F. and T. M. M. Verhallen (1983). "A behavioral model of residential energy use." Journal of Economic Psychology 3(1): 39‐63. Wang, C., et al. (2009). A service‐oriented system for optimizing residential energy use. Web Services, 2009. ICWS 2009. IEEE International Conference on, IEEE. Watson, R. T., et al. (2010). "Information systems and environmentally sustainable development: energy informatics and new directions for the IS community." MIS quarterly 34(1): 23. Wells, R. (2012). Power struggle: homes cut off. The Age. Victoria, Fairfax Media. Zengkun, F. (2011). Energy‐saving features for Punggol flats in pilot project. Straits Times. Singapore, Singapore Press Holdings. Zhang, T., et al. (2012). "A three‐dimensional model of residential energy consumer archetypes for local energy policy design in the UK." Energy Policy 47(0): 102‐110. Zuboff, S. (1985). "Automate/informate: the two faces of intelligent technology." Organizational Dynamics 14(2): 5‐18

327

Does the Need to Belong Drive Risky Online Behavior? Antony Zogg and Tony Hooper Victoria University of Wellington, Wellington, New Zealand antony.z@xtra.co.nz Tony.Hooper@vuw.ac.nz Abstract: One of the most fundamental characteristics of the human condition is the need for social belonging. Social group formation is fundamental to our evolutionary survival. Online communities on social networking sites, reflect the real world human need for socialization and acceptance, and use mediated communication mechanisms to facilitate those relationships. They also reflect the behavioural mechanisms humans use subconsciously to gain acceptance or re‐ admittance to groups in the offline world from which they may have been ostracised. The act of mimicking personal behaviour, even unconsciously, has been shown to enhance the likelihood of relationship development in both parties (Lakin, Chartrand, and Arkin 2008). Similarly, people align themselves with those who reflect their values (Posner 2010). Such value congruence builds trust and encourages people to share more information about themselves online on those e‐ business websites that reflect their values (Cazier, Shao, and Louis 2007). People often post personal and, at times, inappropriate information, video clips and photos of themselves on social networking sites like Facebook. While in some cases this is a result of ignorance about how to configure privacy settings (Collins 2010), in others, there appears to be a lack of concern for personal privacy online (Acquisti and Gross 2006). Using a combination of online survey and semi‐ structured interviews this research explores the disinhibition effect (Suler 2004) that online conditions may create, allowing people to reveal more private information about them than perhaps they should. It found that there is a decreased need for social affiliation in online communities than offline. While people seek to reflect the values of their friends on social networks to gain approval, the investigation found insufficient support for this being the result of mimetic behavior. Similarly there was insufficient evidence of the need for social affiliation being more important than personal security as a driver of risky information posts on social networks. While online communities create a sense of belonging, and therefore of security, the aspiration towards group membership in an online community was not found to be a driver of risky behavior. Belonging to an online social group may enhance perceptions of protection and security but not to the extent that it violates individual privacy perceptions. This research will be of particular interest to managers and marketing professionals who engage with their customers via social networking services. The use of mimetic behaviour to encourage a sense of value congruence in customers could promote interaction and engagement between the business and its online community in a way that will be reflected in their purchasing behaviours. This research provides insight into the transition between similar behaviours in the online and offline worlds. Keywords: belonging, social group formation, risky behaviour

1. Introduction People from pre‐teens to adults post personal and, at times, inappropriate information about themselves online, thereby potentially putting themselves at risk (Gray and Christiansen 2009). The cause may be ignorance on how to configure privacy settings (Collins 2010) or a lack of concern for personal privacy online (Acquisti and Gross 2006). The consequences of risky online behaviour are wide ranging and can include identity theft, re‐identification, being vetted by potential employer’s online, marketing firms gaining access to personal information, and sexual predation (Gross and Acquisti 2005). It would appear that online social network (OSN) communities reflect the human need for socialization and acceptance that they experience in the real world (Baumeister and Leary 1995). When humans are ostracised, they go to great lengths to get re‐ admitted to the group (Lakin, Chartrand, and Arkin 2008) . In order to gain acceptance and inclusion, the act of mimicking a person, even unconsciously, appears to encourage relationship development between the mimicker and the mimickee (Lakin, Chartrand and Arkin 2008). The demonstration of value congruence has been shown to encourage the alignment of people with shared values (Posner 2010). Similarly value congruence builds trust encouraging people to share information about themselves online in business websites that reflect their values (Cazier, Shao, and Louis 2007).This research investigates whether mimetic behavior to gain social acceptance may lead people to indulge in risky online behavior in order to demonstrate value congruence and gain acceptance in their chosen online community. By extension, a strong sense of social acceptance and shared values may make people feel so secure that they indulge in risky behaviour as a result. The purpose of this research is to establish whether online communities reflect the behavioural dynamics shown to exist in the real world. This would contribute to an understanding of whether mimetic behaviour and the demonstration of value congruence may also be a driver of risky online behaviour. If so, the perceived protection and security afforded by memberships of online groups, may be changing perceptions of personal privacy.

328

Antony Zogg and Tony Hooper

2. Literature review One of the fundamental needs of humans is to be able to build and maintain social relationships that are meaningful (Baumeister and Leary 1995; Cialdini and Goldstein 2004). The more we like a person, the more we are likely to consciously (Cialdini and Goldstein 2004), or subconsciously (Gabriel et al. 2010), engage in behaviour that will meet with their approval, and lead to acceptance by them. While the forming and maintenance of friendships should be easy, a significant amount of effort is committed to interpersonal activities (Baumeister and Leary 1995), although the ultimate outcome is the need to belong (Bergsieker, Shelton, and Richeson 2010). Changes to one’s sense of belonging may result in strong emotional responses (Baumeister and Leary 1995). Mimetic behaviour has been demonstrated as a way of building rapport and as a mechanism to deal with the threat to the need to belong (Lakin, Chartrand and Arkin 2008). Exclusion from the group may trigger unconscious mimetic behaviour with another member of the group as a way of getting re‐admitted to the group (Lakin, Chartrand and Arkin 2008). (Van Baaren et al. 2009) have shown that mimicry, even unconscious mimicry, can be a by‐product of social interaction with a person one likes. The body language, tone of voice, facial expression and gestures of the person from whom approval is sought is mimicked without the awareness of the mimicker (van Baaren et al. 2009). In the event that a person does not feel drawn to another person there is an unconscious effort not to mimic that person (van Baaren et al. 2009). If a person construes themselves as being different from others, they may mimic others to become more connected with them. Research conducted between strangers showed that mimicking behaviour changed the way people engage with and perceive others, and that the mimicked person feels similar and positive towards the mimicker (van Baaren et al. 2009). While belongingness and conformity to the views of others contributes to a person’s self esteem, it appears that as long as a person already has a strong sense of belonging, the need to live by beliefs and views shared by others is not as important to self‐esteem. So where a person does not belong, conforming to the values and beliefs of others becomes more important (Gailliot and Baumeister 2007). Research into value congruence shows some similar features. (Pirson and Malhotra 2008) have shown that stakeholders consider value congruence as most important in any business or organization with which they intend to associate. Similarly, (Cazier, Shao and Louis 2007) have demonstrated that people interacting online with businesses are more likely to share personal information with those businesses with which they can feel value congruence. (Posner 2010) has shown that people would wish to associate with a group or organisation that is in alignment with their personal values, resulting in greater job satisfaction, motivation and the reduction of stress. Value congruence appears to increase with the passage of time (Posner 2010). Involving employees in drafting a mission statement that reflects their values, may lead to value congruence and greater loyalty to the company (Verma 2009). If a mission statement is drafted on the basis that the values of the employees are reflected in it, this can lead to employees identifying with the mission and committing themselves to carry it out. Value congruence then, can be a powerful motivator of staff in the organisation (Verma 2009), can lead to greater trust and the reduction of stress, in both online and real world situations. A sense of belonging, of value congruence and increased trust can be expected to lead to a reduction in stress and an increase in perceived safety, lessening the need for personal privacy. In research carried out on an online community of investment professionals in South Korea, (Park et al. 2010) showed that two social factors contributed to the intention to seek or share information. Some members of virtual communities share information with a view to improve their reputation within the community. There is also an associated reward for this type of behaviour in terms of social capital where there may be a financial reward (Park et al. 2010). Other members of the community are driven by a sense of belonging that makes them more likely to share information with other community members, and are more trusting of the information they seek from others. The authors concede that these behaviours are applicable to other virtual communities including online social networks where people seek and share information interactively (Park et al. 2010). The question arises whether mimetic behaviour and the need for social acceptance may give rise to people posting risky personal information on social network sites. Similarly does posting risky personal information on social network sites reflect a need to belong to the group and/or a need to demonstrate value congruence? The following hypotheses were identified:‐

329

Antony Zogg and Tony Hooper H1: Humans mimic online behaviours including risky behaviours to gain acceptance in online communities H2: The need for personal safety online is secondary to the need for social affiliation H3: Humans reflect the values of their friends on social networks to gain their approval

3. Methodology Weber’s comments that the rhetorical divide between positivism and interpretivism no longer serves us, and should be scrapped (Weber 2004), forms the philosophical basis for this investigation. It seems that there is a place for all research philosophies with some suited to certain research types and not to others, justifying the views of (Onwuegbuzie, Johnson and Collins 2009), that there is room for a mixed approach to research. Building on the literature survey and the arguments presented above an exploratory investigation was undertaken using an anonymous online survey followed by a series of semi‐structured interviews.

4. Limitations Limited sample size (130 survey responses) and limited interview candidates provide clear constraints on the generalisability of these findings. Due to the fact that Facebook friends of the researcher were scattered around the globe, interview participants had to be selected from the Wellington region, meaning that the interviewees did not represent all of the nations involved in this research. In the content analysis, the Weft‐ QDA software allows for data categories to be created for analysis. Each response to a question had to be coded by the researcher. In a case where a comment could fit a number of different categories, the researcher had to exercise his discretion. Similarly the interpretation of the graphs produced by the quantitative analysis and the interpretation of the freeform survey questions was dependent on the experience and beliefs of the researcher. While the sample sizes used in this research are too small for making generalizations from, the findings were considered to be worth sharing in a conference situation as a prelude for refining the research questions and methodology for a further investigation.

5. Summary of survey data and gaps in knowledge The majority of respondents to this survey reported a sense of community and belonging online. Opinions diverge on the question of the comparison between real world groupings and online communities. In the online world, the five senses are not utilised in communication and the absence of body language, tone of voice and other non‐verbal cues often lead to misunderstandings. By contrast in the real world people communicate face‐to face and engage the five senses extensively in communication (Suler 2004). There was evidence to suggest that people try to rebuild their real world communities online. Respondents reported to belonging to online family groups, school groups and sporting groups that reflect the real world. Many also indicated that they only friend people on Facebook who are already trusted friends in the real world. Online communities differ from real world groupings in some important respects. The survey result indicates that affiliation needs online are of lesser importance than in real world groups, and this may indicate why little evidence of human mimicry online could be found. Accordingly, the survey data did not appear to support the hypotheses humans mimic online behaviours, including risky behaviours, to gain acceptance in online communities. The survey data also indicates that while most only befriend people online that they already know and trust in the real world, the majority of respondents are concerned with their personal safety and privacy online. This indicates that personal safety online is of greater importance than social affiliation online. The survey data does not support the hypothesis that the need for personal safety online is secondary to the need for social affiliation. In spite of the risks, there was an indication that the majority of respondents were comfortable sharing information online with people they trust. While respondents seemed to be aware that there are risks online, there appeared to be some ignorance as to what those risks actually were. On the question of online mimicry, the majority of respondents reported changing their communication styles online depending on the person they were communication with. A lot of thought and consideration was given to who to admit as Facebook friends, so their actions online were neither unconscious nor automatic. What did not emerge was the motivation for that behaviour and whether it was a conscious or unconscious

330

Antony Zogg and Tony Hooper behaviour. One could argue that because of the unconscious nature of mimicry, respondents may not be aware that they are engaging in mimicry online. One could also argue that out of an infinite number of behaviours that constitute mimicry, only three were considered in this research. What is also not clear is whether, as happens in the real world, people tend to mimic each other more on Facebook than on Linkedin or any forum or blog. The survey data demonstrated that both values and trust are important factors in determining the amount of personal information shared online and that people are not prepared to compromise their values online. This was demonstrated by the fact that the majority of respondents indicated their willingness to share personal information online with trusted friends. If the information to be shared was controversial, the majority of respondents indicated that they would not share it. The data offers strong support for the hypothesis that humans reflect the values of their friends on social networks to gain their approval.

6. Summary of interview data The interview data reflected the view that the five senses required to communicate effectively in face‐to face communication and those senses are eroded in an online context. The logical conclusion here is that the real world need to belong is not as strong or is not replicated online. This means that there is limited scope online for human mimetic behaviour. Accordingly, the interviews confirmed the survey finding that mimetic online behaviours to gain acceptance in online communities, including risky behaviours, are not supported. This is further confirmed by statements from the majority of interviewees that they consciously imitated rather than mimicked the behaviour of others online as a way of being clearly understood as opposed to being accepted. The majority of interviewees carefully select their online friends from their real world groups, and configure their privacy settings. These behaviours do not support the second hypothesis that the need for personal safety online is secondary to the need for social affiliation. In spite of the risks, the majority of interview respondents indicated that they were comfortable sharing information online with people they trust. While there is an awareness of the risks of sharing too much personal information online, interviewees, like their counterparts in the surveys, were unaware of the nature of one of these risks. Of eight interviewees, seven were not aware that Facebook sells their personal information to other businesses, marketing companies and other interested organisations. It was demonstrated in the surveys that respondents share more personal information online with people and communities that they trust and that share their values. From the interviews it was clear that respondents trusted different social networks more or less than others. Accordingly they varied the type of personal information they were prepared to post on those sites. A logical explanation for those behaviours may be that Facebook espouses similar values to those of one user whereas Linkedin espouses similar values to those of another user. Another explanation may be that users present their persona in different ways to different people or communities, the nature of their postings depending on the social networking site they are engaged on and the people with whom they wish to relate (Suler 2004). Either way it reflects the hypothesis that humans reflect the values of their friends on social networks to gain their approval.

7. Discussion This research was designed based on the assumption that either users engage in online risky behaviours, or they do not. It did not provide for the possibility that users may behave in different ways when faced by different online situations as discussed in the Cyber Psychology literature (Suler 2004). A user involved in one online experience may be more disinhibited when communicating with friends than communicating online with a parent, for example. Suler’s definition of disinhibited and inhibited online behaviours can explain to some extent, why some of the behaviours measured in this research were mixed (Suler 2004). In the one online context, the user is not engaging in risky online behaviour, but in another context, that same user is engaging in what might be considered to be risky online behaviour. This research indicates that risky online behaviour is not a tendency for a specific user, but is dependent on the online situation being faced by that user (Suler 2004). While online communities are similar to some extent with real world groups, they are also very different, particularly in the way people communicate. In the real world people meet face‐to‐ face and communicate using a rich tapestry of verbal and non‐verbal cues. Body language, tone, facial expressions, enthusiasm and

331

Antony Zogg and Tony Hooper emotion all add to understanding of the message being received. However, online the five senses do not come into play and this blurs the delineation of a person’s personal boundary (Suler 2004). Suler argues that this may be the reason why people reveal information online that is normally kept secret in the real world (Suler 2004). There is evidence in the survey data that people divulge different information in different online contexts, like when using different media. Their behaviour may be risky in one context and guarded in another. This suggests that without the face‐to‐face interaction online, the need to affiliate with online social groups may not be as powerful as the need to affiliate with groups in the real world. The majority of interviewees acknowledged a process of self‐moderation before sending out an online message indicating that the action was deliberate. There was little evidence of mimicry online but where people do mimic each other, acceptance rather than being understood, is the motivation for this behaviour. This does not entirely discount the possibility of human mimicry online, because due to its unconscious nature, people may not be aware of their behaviour and this makes it difficult to measure. As indicated above, people often behave riskily on one situation but are very guarded in another. An individual will behave in different ways depending on the media used. It is possible that if the focus of the surveys and interviews shifted away from Facebook to some other forum, the same sample of respondents may have answered very differently. Most individuals interviewed or surveyed selected their online friends carefully, and these people were family and friends in the real world too. The basis of good friendship in the real world is that it is values driven. That indicates that trust and shared values are brought online resulting in respondents sharing more personal information about themselves online. This has been demonstrated in the literature in both the real world and online context from an organisational perspective (Cazier, Shao and Louis 2007; Pirson and Malhotra 2008; Posner 2010; Verma 2009). The survey data demonstrated the importance of values to the respondents. There was a strong assertion that values would not be compromised in any attempt to gain a sense of affiliation in online communities. Values were more important to respondents than the prospect of being accepted or rejected by an online group. The responses to the Likert questions demonstrated the lack of interest in online affiliation, but a keen interest in shared‐values. It is important to view the results of this research through the lens of the disinhibition effect (Suler 2004). It is likely that in a slightly different online context or situation the behaviours of the self same respondents of the survey and the interviewees might well have been different.

8. Implications for business and future research The lessons from this research can help businesses to understand their customers and to provide better products and services. This research confirms studies carried out online that it is important for company values to reflect the values of their customers (Cazier, Shao and Louis 2007; Caldwell, Chatman and O'Reilly 2009; Pirson and Malhotra, 2008 and Posner 2010). Businesses need to promote transparency and to espouse their organisational values in their daily operations and on their websites. More research is needed into online human mimicry as this is such a powerful driver of human behaviour in the real world (Lakin, Chartrand and Arkin 2008; van Baaren et al. 2009; Kühn et al. 2009; Cialdini and Goldstein 2004; Kouzakova et al.2010). Similarly, more research is needed on the disinhibition effect (Suler, 2004) to build websites that provide a context in which customers can maintain anonymity and / or invisibility as they safely engage with the business online.

9. Conclusion The objective of this research was to explore why people indulge in the risky behaviour of sharing too much personal information online. What emerged was the fact that while online and real world communities are similar in some ways, they are very different in others. The major difference relates to our real‐world reliance on body language, tone of voice and other non‐verbal cues to ensure that our message is understood. Online communication cannot rely on body language and other non‐verbal cues to convey the message, thereby dehumanizes the communication process. Because our five senses cannot be used in online communication, we may compensate by divulging information online that might not have been necessary in the real world. Such differences between real world groups and online communities may have resulted in a decreased sense of social affiliation online. In such a situation people may not engage in mimetic behaviour online as there is not as great a need to belong to online communities as there would be in the real world.

332

Antony Zogg and Tony Hooper Our research has shown that online privacy is considered by most people to be of great importance and most will configure their privacy settings. Our respondents showed that they were values driven and would not compromise those values to gain acceptance in online communities, possibly reflecting a decreased sense of social affiliation online. Individuals appear to display a range of online behaviours along a continuum from risky to ultra‐cautious, depending on the online context in which they are engaging with others.

References Acquisti, A., and Gross, R. (2006) "Imagined communities: Awareness, information sharing, and privacy on the Facebook", Paper presented at the Proceedings of Privacy Enhancing Technology, Cambridge, UK, June. Baumeister, R. F., and Leary, M. R. (1995) "The need to belong: Desire for interpersonal attachments as a fundamental human motivation", Psychological bulletin, Vol 117, pp 497‐497. Bergsieker, H. B., Shelton, J. N., and Richeson, J. A. (2010) "To be liked versus respected: Divergent goals in interracial interactions", Journal of Personality and Social Psychology, Vol 99, No 2, pp 248‐264. Caldwell, D. F., Chatman, J. A. and O'Reilly, C. A. (1990), "Building organizational commitment: A multifirm study", Journal of Occupational Psychology, 63: 245–261. Cazier, J. A., Shao, B. B. M., and Louis, R. D. S. (2007) "Sharing information and building trust through value congruence", Information Systems Frontiers, Vol 9, No 5, pp 515‐529. Cialdini, R. B., and Goldstein, N. J. (2004) "Social Influence: Compliance and Conformity", Annual Review of Psychology, Vol 55, pp 591. Collins, J. C. (2010) "Fortify Your Facebook Privacy Settings", Journal of Accountancy, Vol 209, No 6, pp 42. Gabriel, S., Kawakami, K., Bartak, C., Kang, S. J., and Mann, N. (2010) "Negative Self‐Synchronization: Will I Change to Be Like You When It Is Bad for Me", Journal of Personality and Social Psychology, Vol 98, No 6, pp 857‐871. Gailliot, M. T., and Baumeister, R. F. (2007) "Self‐esteem, belongingness, and worldview validation: Does belongingness exert a unique influence upon self‐esteem", Journal of Research in Personality, Vol 41, No 2, pp 327‐345. Gray, D., and Christiansen, L. (2009) "Protecting Adolescents' Personal Information Online: Constraints and Parameters", Journal of Information Privacy & Security, Vol 5, No 4, p 31. Gross, R., and Acquisti, A. (2005) "Information revelation and privacy in online social networks", Paper presented at the Proceedings of the ACM workshop on privacy in the electronic society, Alexandria, Virginia. Kouzakova, M., Karremans, J.C., van Baaren, R. B., and van Knippenberg, A. (2010) "A Stranger's Cold Shoulder Makes the Heart Grow Fonder: Why Not Being Mimicked by a Stranger Enhances Longstanding Relationship Evaluations", Social Psychological and Personality Science Volume 1.1 pp 87‐93. Kühn, S., Müller, B.C.N., van Baaren, R. B., Wietzker, A., Dijksterhuis, A., and Brass, M. (2010) “Why do I like you when you behave like me? Neural mechanisms mediating positive consequences of observing someone being imitated" Social Neuroscience, Vol 5 No 4 pp 384‐92 Lakin, J.L., Chartrand, T. L., and Arkin, R. M. (2008) "I Am Too Just Like You", Psychological Science, Vol 19, No 8, p 816. Onwuegbuzie, A., Johnson, R., and Collins, K. (2009) "Call for mixed analysis: A philosophical framework for combining qualitative and quantitative approaches", International Journal of Multiple Research Approaches, Vol 3, No 2, pp 114. Park, J. H., Konana, P., Gu, B., and Man Leung, A. C. (2010) "An investigation of information sharing and seeking behaviors in virtual communities". Paper presented at the ICIS 2010 Proceedings. from http://aisel.aisnet.org/icis2010_submissions/242 Pirson, M., and Malhotra, D. (2008) "Unconventional Insights for Managing Stakeholder Trust", MIT Sloan Management Review, Vol 49, No 4, p 43. Posner, B. (2010) "Another Look at the Impact of Personal and Organizational Values Congruency", Journal of Business Ethics, Vol 97, p 535. Suler, J. (2004) "The online disinhibition effect", CyberPsychology & Behavior, Vol 7, No 3, pp 321‐326. van Baaren, R., Janssen, L., Chartrand, T. L., and Dijksterhuis, A. (2009) "Where is the love? The social aspects of mimicry", Philosophical Transactions of the Royal Society B: Biological Sciences, Vol 364, No 1528, p 2381. Verma, H. (2009) "Mission statements ‐ a study of intent and influence", Journal of Services Research, Vol 9, No 2, p 153. Weber, R. (2004) "The Rhetoric of Positivism Versus Interpretivism: A Personal View", MIS Quarterly, Vol 28, No 1, pp III.

333

334

PhD Research Papers

335

336

Challenge in Sharing Tacit Knowledge: Academicians’ Behavior Towards Developing a web Portal for Sharing Research Ideas Hafiza Adenan, Dang Merduwati Hashim, Adnan Jamaludin and Haziah Sa’ari Faculty of Information Management, MARA University of Technology, Puncak Perdana campus, Malaysia hafizaadenan@ymail.com merdu61@yahoo.com azie.crystal@gmail.com Abstract: Academicians’ collective memories soft information, such as research ideas, expertise, experiences, academic skills, know‐what, know‐how and know‐why which inevitability it is considered should made accessible.The Higher Education Institution needs to identify, collect, classify, verbalize and diffuse the academicians’ soft information specifically research ideas present in the university for knowledge enrichment. This can be implemented by the academicians actively sharing their research ideas with others. Actively sharing research ideas by academicians will have great impact on the enrichment of their intellectual capability as most of the valuable knowledge resides in one’s brain. However, as there is no specific medium to bring their research ideas into the surface and be visible to others, the precious research ideas still remain in the academicians’ brains. Therefore, the objective of the study is to explore academicians’ behavior toward the development of a sharing research ideas web portal at private university colleges in Malaysia. This study used the qualitative method that is a multiple cases study. The study refers to four private university colleges in Malaysia. In‐depth interview, focus group discussion and document analysis were formed the data collection for this study. The theory of Planned Behavior by Ajzen (1991) was used to determine academicians’ behavior. This study showed that the academicians’ attitude, subjective norms, and perceived behavioral control towards developing a web portal for sharing research ideas all affect their intention to share their research ideas with others. Keywords: academician, sharing research idea, attitude, subjective norm, perceived behavioral control, intention

1. Introduction In Malaysia, in line with globalization changes in the Higher Education Institution (HEI), the role of private universities has been scrutinized and challenged by society. To face the challenges, the academicians in the private universities’ should actively undertake research as part of their job function to share the latest developments, insights and expertise with society and produce high quality findings which will serve as motivation for learning and commercialization. Hence, research is the road to discovery with a proven worldwide acceptance that research contributes positively to society economically. Generally, research process starts with developing knowledge which resides in the minds of the academicians. According to Nonaka and Takeuchi (1995), this type of knowledge is known as tacit knowledge that is knowledge which is rooted in action, experience and involvement in a specific context. Tacit knowledge is comprised of cognitive and technical elements. Nevertheless, Nonaka and Takeuchi (1995) acknowledged that tacit knowledge is almost impossible to be shared without the active participation and cooperation of the knower. As noted, it is commonly known that one of the platforms that enable the knower to share ideas is through information communication technology such as the web portal. Hence, the academicians need specific web portal to capture knowledgeable research ideas and share their research ideas with interested parties, and next, help them to implement research works efficiently and commercialize their research findings successfully.

2. Web portal and higher education institution A web portal is an online tool to aggregate information in one location in an organized manner for dissemination. The most recent application of portals in HEI has been to create a point of access for administrative functions for students, such as registration, financial aid, and academic records, or for staff, such as time sheets, leave balances. Therefore, the use of portals maximizes the efficient use of staff and students’ time (Rajalakshmi, 2010). Today’s portals are increasingly secured applications designated to give users simple, quick, secure access to relevant organizations and personal data. The portal is a place for HEI’s members with common interest on any matters to meet and chat virtually. It is a source of information and knowledge to be shared among the HEI members. The portal through its functions allows the HEI members to share ideas and information and helps them to get to know each other. As a collaboration tool, the portal helps HEI in promoting activities for mutual benefits.

337

Hafiza Adenan et al.

3. Knowledge sharing and higher education institution Knowledge sharing is considered as the foundation of learning and research at HEIs. In the context of higher education, Ali (2010) defines knowledge sharing as the process of exchanging and acquiring knowledge that is needed through informal and formal technical facilities channels. Scholars and practitioners have emphatically argued that a large part of knowledge in organizations is tacit. Unlike explicit knowledge, tacit knowledge essentially resides in the minds of the knower. Thus, it has been determined that knowledge sharing becomes a major focus area in knowledge management as knowledge sharing serves as a link between the strata of individual knowledge, where the knowledge resides and the strata of the organization where knowledge generates its economic and competitive gains. However, the study in 2005 which involved all the 17 public universities in Malaysia found that the existence of a knowledge sharing culture in Malaysian public HEIs was still very low. Only 29.4% of the staff of Malaysian public HEIs have positive culture in their routine work (Suhaime, 2006). The study implies that as academicians show a positive attitude and tremendous feeling of enjoyment in their working environment, their self‐efficacy and knowledge sharing levels improve.

4. Theory of planned behavior and knowledge sharing Theory of Planned Behavior (TPB) is a widely accepted model and is used in finding factors influencing behavior as this theory is a general model of behavior specific that can predict a range of behavior. TPB assumes that individuals’ behavior is determined by behavioral intention and perceived behavioral control. Attitude toward behavior, subjective norm and perceived behavioral control determine the behavioral intention. TPB proposed by Ajzen (1991), explained that behavioral intentions are motivational factors that show to what extent people are prepared to perform a behavior. Attitude has been regarded as influencing behavioral intention. Bock, Kim and Lee (2005) found that an individual’s intention to share knowledge is driven primarily by attitude towards knowledge sharing. Subjective norm indicates that behavior is directed by one’s desire to act as an important referent to how others act or think one ought to act. When applied to the focal behavior, subjective norm echoes the participant’s perceptions of whether the behavior is accepted, encouraged, and implemented by the participant’s spheres of influence. Subjective norm is also considered as an outcome of beliefs that specific individual’s approve or disapprove of performing the behavior. Perceived behavioral control factors are individual natural factors and include the extent of information a person has, together with the skills, abilities, emotions, and needs of that person regarding a specific behavior. It concerns the degree to which an individual thinks that performance or nonperformance of the behavior in question is under his or her free control.

5. Problem statement The research ideas of private university academicians’ are important in producing a knowledge society and helping Malaysia to become knowledge based economy country. However, some of the precious research ideas remain in the academicians’ brains as there is no specific web portal to bring their research ideas into the open to be visible to others. At present, available web portals are becoming an increasingly important tool to help private university colleges to gather intellectual resources. Nevertheless, there is no specific web portal for the academicians to capture and share their research ideas with interested parties and next, help them to implement and commercialize their research findings efficiently. In addition, currently, there is no specific web portal that enables the management of private university colleges to identify what is required by academicians to carry out research works and to offer appropriate rewards or incentives as appreciation for the research ideas most contributed and the research ideas most referred to.

6. Research question and research objective The research question of the study is: What is the behavior of private university college academicians’ towards developing web portal for sharing research ideas? The research objective is to investigate the behavior of private university college academicians’ towards developing a web portal for sharing research ideas..

7. Study’s significances The study’s findings enable the private universities in Malaysia to:

Indicate the academicians’ behavior toward developing a web portal for sharing research ideas.

338

Hafiza Adenan et al.

Justify developing the web portal for sharing research ideas as a robust virtual platform for the motivated academicians or other interested parties to acquire and share the research ideas that resides in one’s brain.

Clarify that the web portal is a platform for the academicians who lack experience in researching works to acquire knowledgeable research ideas from other experience parties and then implement their research works.

Justify that web portal is a platform for the private university colleges’ management to identify the research instruments required by the academicians and other interested parties.

Justify that the web portal is a platform for the management of private university colleges to appreciate the research ideas most contributed and most referred to.

Determine that the web portal for sharing research idea is an aid to private University colleges’ management to improve their reputation and enhance the productivity through producing high quality research.

8. Conceptual framework Figure 1 shows the conceptual framework of the study. The study’s conceptual framework explains that the academicians’ attitude, subjective norm and perceived behavioral control influenced their intention to use the web portal for sharing research ideas. Next, the intention either to use or not to use the developing web portal for sharing research ideas will affect the academicians’ actual behavior towards the web portal for sharing research ideas by private university colleges’ management.

Academicians’ attitude

Academicians’ perceived behavioral control

Academicians’ subjective norm

Academicians’ intention to use the developing web portal for sharing research ideas

Academicians’ behavior toward the developing web portal for sharing research ideas Figure 1: The study’s conceptual framework

9. Developing the web portal for sharing research ideas This section describes the development of the web portal for sharing research ideas.. Specifically, the functions of developing a web portal for sharing research ideas is to helps academicians or other interested people to share and determine the research ideas needed and to transform those ideas into a web base knowledge bank. This makes the research ideas available to interested people at the right time, connects people and builds relationships across various educational or other interested institutions. Next, the sharing research ideas process through the application of the sharing research ideas web portal enable the academicians or other interested parties to implement their research works efficiently. Apart from that, through developing the sharing research ideas web portal, the university management is able to determine the requirements of academicians or other people relating to research works or commercialization. It also functions as a motivational tool by identifying appropriate rewards or attractive incentives to the contributors of the most

339

Hafiza Adenan et al. research ideas and the most research ideas referred to. Figure 2 shows the entity relationship for the developing the sharing research ideas web portal.

Sharing research idea web page Registration page: • Staff of institution • Staff of other institution • Student Sign up page • Username • Password • Top hits idea (contributor) Search page - Searching idea strategies: • Contributor • Subject • Keyword Contribute Ideas page: • Idea title • Idea description • Contributor • Institution Archive page: • Contributor • Subject • Keyword • Year Figure 2: Entity relationship for the sharing research ideas web portal

10. Research methodology In order to achieve the objectives mentioned above, this study used a qualitative research method. This method focuses on understanding the point of view, experience and interpretation of the private university colleges academicians. Hence, in this study, the qualitative research design used is multiple cases study. In this study, it refers to four main cases (four private university colleges in Malaysia) where it is a valuable and useful in the exploratory and descriptive and early stages of research regarding academicians’ behavior toward developing a sharing research ideas web portal. In‐depth interviews, focus group discussions and document analysis were the data collection for this study. Based on the case study methodology of the research adapted from Yin (1994), the initial step in the study involved the development of a conceptual framework on the

340

Hafiza Adenan et al. influence of academicians’ behavior on the developing sharing research idea web portal. This step was then followed by the selection of the pilot case study and four main case studies (the data collection sites) and the design of the data collection protocol. Following the case selection and data collection steps, the individual case reports were developed. A cross‐case analysis of the findings was then undertaken. This analysis provided the basis for the discussions and implications presented in the final section of the article. Figure 3 shows the case study methodology of this study adapted from Yin (1994).

Develop data collection protocol

Case selection

Conceptual framework

Conduct pilot case study

Conduct case study one

Conduct case study two

Conduct case study three

Conduct case study four

Cross case analysis, comparisons and conclusions

Discussion and findings

Figure 3: The case study methodology adapted from Yin (1994)

11. Research design The research design started with a literature review which discusses the literature from numerous sources in order to exhibit an in‐depth knowledge of the research problem. A well‐defined research problem is required before the researcher can commence the process of research design and subsequent data collection (Yin, 1994). After reviewing previous literature, research problems of this study were obtained. The development of the conceptual framework is derived from the literature review discussion and research objective. Next questions for in‐depth and focus group interviews were developed. The constructs in in‐depth and focus group interview related to academicians’ behavior were based on the Theory of Planned Behavior model by Ajzen (1991). Next, the sample design was developed. Both samples for the pilot interview and main case study interview consist of academicians from four private university colleges in Malaysia. The sample of academicians for this study was selected by using purposive sampling and chosen by the four private university colleges’ management. In this study the samples of the main case study consist of forty four academicians. They are from social science and science faculties, the length of working experience as academicians is more than one year, they hold a Master Degree or PhD degree and they had designations as head of program and senior academician The in‐depth interview consists of twenty academicians and the focus group consists of four groups of academicians with six academicians per group. Next, five pilot interviews were conducted to improve the data collection processes before the main case studies commenced. This phase represents the final preparation for data collection which is used more formatively to assist the researcher to develop relevant lines of questioning as well as provide some conceptual clarification for the research design. Permission to conduct the research was obtained from the private university colleges’ management. After obtaining approval, the requested lists of academicians were taken from the private university colleges’ management. Then, the researcher send a letter to each of the academicians selected to inform them about the research and the date that the interview would take place. Each letter clearly explained the objective of the study. In addition, the academicians were informed about the importance of the interview, the confidentiality of their responses and that the data would only be utilized for the purpose of the study. Once the interviews had been conducted, recorded and transcribed, the information was analyzed and interpreted in order to produce the findings. Findings were inducted from the data.

341

Hafiza Adenan et al.

12. Data analysis In this study, the information gathered from in‐depth interview and focus group interviews was recorded into audiotape and converted into text data. Then, the researcher carried out the transcription process and coding. In analyzing the transcripts of textual data from the in‐depth interview and focus groups, the four attributes in TPB that is attitude, subjective norm, perceived behavioral control and intention were used as umbrella categories for the coding of data. They provided a useful sorting device for sub themes relevant to these four TPB attributes. After completing the transcription process, the researcher performed the coding process which aims to derive meaning out of the text data, by dividing it into text and labeling the segments with codes. The coding process also serves to avoid overlap and redundancy. The researcher needed to compile the list of themes, cluster similar topics together, create categories and subcategories. An iterative, inductive process was used in the coding and analysis of data. Thereafter, in‐depth interviews and focus groups interviews, transcribed from audiotape were thematically coded by content analysis, using a coding system devised by the researcher. In order to manage the data efficiently, the researcher used the qualitative data analysis software tool NVivo9. NVivo9 is a computer program which allows one to import and code textual data, edit the text; retrieve, review and recode coded data; search for combinations of words in the text or pattern in the coding and import from or export data to other quantitative analysis software. Specifically, NVivo9 is designed to facilitate the construction of relational networks identifying the content and structure of respondents’ opinions. NVivo9 involves the development of a tree‐like structure of relationships between elements referred to as nodes.

13. Study’s findings 13.1 Academic entrepreneurs’ attitude All the academicians from four private university colleges in Malaysia supported the development of a web portal for sharing research ideas. They admitted that the sharing research idea web portal would help them to identify required information and knowledge to complete their research process. The academicians said that through the sharing research ideas web portal, they would be able to build up teamwork and relationships with people interested in their research works and in addition, help them in the commercialization process. At the same time, the academicians agreed that developing the sharing research idea web portal is the appropriate platform for them to communicate with others regarding research matters.

13.2 Academicians’ subjective norm The people around academicians, specifically their superiors, colleagues and friends encourage them to use the sharing research ideas web portal. According to those people, the sharing research ideas web portal will help to produce innovative and creative academic entrepreneurs in future. This is evident from the encouragement given by the university management through rewarding active and innovative research ideas contributors. These people admitted that when the academicians actively commercialize their findings, indirectly, the university achieves a good reputation and gathers better profit.

13.3 Academicians’ perceived behavioral control As known, the academicians justified that the sharing research ideas web portal gives benefit to their research works. Therefore, they admitted that the application of developing the sharing research ideas web portal enables them to enhance their skill, knowledge and experience.

13.4 Academicians’ intention The academicians’ attitude, subjective norm and perceived behavioral control towards developing the sharing research idea webs portal are encouraging, thus, all of them have the intention of using the sharing research ideas web portal.

342

Hafiza Adenan et al.

13.5 Academicians’ behavior As all of the academicians have the intention of using the sharing research idea web portal, their behavior is towards applying themselves to the development of the sharing research ideas web portal.

14. Conclusion The private university colleges’ academicians behavior towards developing the sharing research idea web portal is encouraging. They identify that the functions of the sharing research ideas web portal will enable them to achieve their mission that is to produce high quality commercialize research findings. Apart from that, they clarify that through the sharing research ideas web portal, they are also able to collaborate with relevant parties and build up relationships with other people who are actively and interested in the research works. As for recommendation, in future, the private university colleges in Malaysia should establish the knowledge sharing center and equip it with the appropriate and efficient information communication technology which supports communication via technology. Efficient communication technological facilities will encourage the academicians to share their research ideas through technology. In addition, the private university colleges’ management should upgrade the sharing research idea web portal to be accessible to other institutions abroad.

References Ali, M.M.S.,(2010) “Cultural influences on knowledge sharing in a Kuwaiti higher education institution: a case study approach,” Unpublished PhD thesis. Ajzen, I. (1991).”The Theory of Planned Behavior”. Organizational Behavior and Human Decision Processes. Vol.50, pp 179‐ 211. Ariffin, A. H. (2004). Managing knowledge in FIT academic environment: A proposed knowledge sharing framework. Unpublished Master degree dissertation, Universiti Utara Malaysia, Malaysia. Arokiasamy, L., Ismail, M., Ahmad, A., & Othman, J. (2009). “Background of Malaysian Private institutions of higher learning and challenges faced by academics”.The Journal of International Social Research. Vol. 2, No.8, pp 60‐67. Bock, G.W., Zmud, R.W., Kim, Y.G. Kim and Lee, J.N. (2005). “Behavioral intention formation in knowledge sharing examining the roles of extrinsic motivators social‐psychological forces and organizational climate”. MIS Quarterly. Vol.29, No.1, pp 87‐111. Farsi, J.Y and Talebi, K. (2009) “Application of knowledge management for research commercialization”, World Academy of Science, Engineering and Technology. Vol.49, pp 451‐455. Iqbal, M.J., Rasli A, Low H.H, Bilal Ali M, Hassan I & Jalaee A. (2011) “Academic staff knowledge sharing intentions and university innovation capability”, African Journal of Business Management. Vol. 5(27), pp.11051‐11059. Mohayidin, M.G. et.al, (2007). “The application of knowledge management in enhancing the performance of Malaysian universities”, The Electronic Journal of Knowledge Management, Vol. 5, No.3, pp.301‐312. Nonaka and Takeuchi, H. (1995). The knowledge creating company. New York, NY: Oxford University Press. Rajalakshmi, S., Wahida Banu, R.S.D., Nithiyanandam, S., and Maguteeswaran, R. (2010). “Developing an Social Networking Repository Education Web Portal for sharing and capturing in Higher Education – The Info‐Ca‐Sh,” International Journalof Innovation, Management and Technology., Vol.1, No.1, pp. 82‐86. Saba F, McDowell D (2007). ‘Knowledge management for teachers: The collection, organization, and sharing of educational wisdom”. Educ.Technol., pp. 39‐44. Shaari, R., Rajab, A., & M.Y. Rosman. (2010). “The organizational issues of knowledge sharing among academic staffs in the Malaysian Public Universities.” International Journal of Knowledge, Culture and Change Management.” Vol.10, issues 6, pp 133‐148. Sohail, M.S. and Daud, S. (2009). Knowledge sharing in higher education institutions. Journal of Information and Knowledge management Systems. Vol.2, No.39, pp.125‐142. Suhaimee, S., Abu Bakar, A.Z., & Alias, R.A, (2006). Proceedings of the PostgraduateAnnual Research Seminar, Skudai, Johor. Yin, R. (1994).Case study research: Design and method (2nd.ed). Beverly Hills, Sage Publishing.

343

Acceptance of new Technologies in HR: E‐Recruitment in Organizations Nicoleta Valentina Florea 1 and Mihaela Badea 2 1 Management‐Marketing Department, Faculty of Economic Sciences, Valahia University of Targoviste, Romania 2 Doctoral School, Valahia University of Targoviste, Romania floreanicol@yahoo.com, badea.mihaela@gmail.com Abstract: Technology can be used to enhance performance, quicken decision making, and provide cost saving. The Internet has changed everything. With Internet recruiting, a company can place notice of a vacancy on its website or list it with one of online job or career websites. The use of Internet and world‐wide‐web to match candidates to jobs through electronic databases that store information on jobs and job candidates is Internet recruiting.Web‐site is a quick and cheap recruitment source and can be very efficient in retaining employees for long term in organization. E‐Recruitment is one of the most successful e‐Commerce applications as a method for quickly reaching a large pool of the potential job seekers. E‐ Recruitment is becoming increasingly popular since the Internet has no geographical boundaries. To attract new employees, some organizations use their e‐Recruitment websites or portals as a means of identifying internal job candidates for new positions or promotions. In this paper we analyse the strategies and the methods of recruitment through the Internet in regard to the traditional methods of recruitment used by large organizations from Dambovita County. We also analyse the advantages and the disadvantages of using such method, what kind of organization rather use e‐Recruitment, as well as, what kind of employees are rather attracted by e‐Recruitment. This paper is a descriptive research based empirical data sources. The empirical data used for the research is for the most part provided by the data of the large organizations from Dambovita County. The article is presenting the way in which the organizations use technology increases or decreases its positive net effect on them. Technology is, after all, nothing than a tool. Through the Internet, HR can develop a powerful recruitment program, that helps manage the highly competitive and time‐consuming process of finding skilled personnel. Keywords: new technologies, e‐recruitment, internet, employees, efficiency, large organizations

1. Introduction “It would seem that the Internet was created specifically for human resources”‐Doran A., 2001. In the twenty‐first century, we are constantly inundated with new ideas, new products, and new processes. Technology influences every aspect of our lives, and it changes constantly. It seems that “work‐saving” technologies emerge almost daily, and, if adopted, they affect organizational policies, practices, and workflow. There is no doubt that technology can impact all facets of society. In some ways, technology simplifies work, and in other ways it makes the workplace more complex. Technology offers both obstacles to success and increased opportunities. Technology is neither bad nor good, but neutral (Waddill D.D. and Marquandt M.J., 2011). The growth of the Internet means that a significant amount of recruitment is now carried out online. E‐ Recruitment is defined as “the use of technology to attract candidates and aid the recruitment process” (Cushway B., 2012).The use of the internet for recruitment purposes is undoubtedly the most striking recent development in the field, but its practical significance remains a question of debate (Torrington D., Hall L. and Taylor S., 2005). The method of matching job seekers to employers that has emerged over the last few years, and is growing more quickly than any other means, is Internet recruitment [Cooper D. et al., 2003].E‐ Recruitment is now a substantial component of the recruitment strategy for a wide range of organizations worldwide, as well as an increasingly popular method for seekers in searching and applying for jobs (CCH WKB, 2010). E‐Recruitment is one of the most successful e‐Commerce applications as a method for quickly reaching a large pool of the potential job seekers [Torres‐Coronas T, Arias‐Oliva M., 2005]. The HR challenges‐ “Better, faster, cheaper…, and more strategic”‐ influence the development of what is now a broad marketplace of tools to support the automation of the recruiting process (Reynolds D.H. and Weiner J.A., 2009). Thanks to the web, company and job information can be put in the hands of job seekers quickly than ever before. Job seekers can act on this information instantly. The organization’s web site can create the ability to attract and retain the right people ([Cohen D., 2001). Many organizations present their vacancies list on their

344

Nicoleta Valentina Florea and Mihaela Badea own web page (Pell A., 2008). Web‐site is a quick and cheap recruitment source (Price A., 2000) and can be very efficient in retaining employees for long term in organization (Bach S., 2005). It has been demonstrated that if the candidates submit online the resume on their own, they are tempted to remain much longer as employees than through other recruitment channels. Organizations have long been concerned with attracting and retaining highly talented employees. The primary reason for this is that depend on the skills and talents of their workforce to compete in an ever‐changing global environment (Guental H.G. and Stone D.L., 2005). In order to facilitate the recruitment process, organizations are increasingly using electronic HR systems, including web‐based job sites, portals, and kiosks to attract job applicants.

2. Research regarding the use of e‐Recruiting in organizations Almost all recruiting is now done on the web (Doran A., 2001). Many technologies has appeared, such as Internet and other new communication methods, which help organizations recruit more rapidly and more efficient the candidates. Many organizations have implemented an interactive web page or a panel survey for students (ex. Tesco). Organizations use website to attract candidates in 82 percent and use online advertising (Czerny A., 2004). Although estimates vary, surveys show that between 70 and 90 percent of large organizations now use e‐recruiting systems, and it is anticipated that over 95 percent of organizations plan to use them in near future (Guental H.G. and Stone D.L., 2005). Cisco mentioned that one of the advantages of using the Internet as a recruitment source is the advertising reach those people who does not actively seeking a job. Organizations have reduced recruitment costs by 85% using online recruitment ads (Bach S., 2005). Furthermore, firms such as Walt Disney World and Cisco are using e‐ Recruitment websites or web‐based portals to help establish “brand identities” that distinguish them from their competitors (Guental H.G. and Stone D.L., 2005). For example, for Disney World the organization’s website plays a pivotal role in attracting new employees to the firm. The applicants can review unique information about the organization’s “brand identity” on the organization’s website to determine whether their personal goals and values fit with the organization’s culture. It is evident that e‐HR systems have become important means of helping organizations establish a brand identity, attract talented workers, and retain valuable employees. In 2002 iLogos survey found out that already 91% for Global 500 companies were using their corporate websites for recruiting (Wuttke F., 2008). The National Online Recruitment Survey (2003) found that the average online job seeker is 33 years old with more than 11years’ work experience and they have been with the same employers for more than four years, and has visited more than five online sites in a quest for new employment. Statistics reveal that 80% of the world’s US 500 companies use corporate websites for recruiting (Epstein R. and Singh G., 2003). Multinationals are proactively seeking the best talent worldwide. The method of online recruiting allows organizations to transcend geographical boundaries seeking the best in people. Nike, with e‐Recruitment obtained an average time to fill job positions reduced from 62 to 42 days and the recruitment costs reduced by 54% (Bondarouk T., 2009). As a method of recruitment, the Internet is less used in Britain then the U.S. (S. Bach, 2005). All information can be updated and access easily. Importantly, as the organization announces vacancies on the site, to continue ensure candidates for its image (Stredwick J., 2000). Research by the British Market Research Bureau has suggested that an internet search is the preferred job hunting method for one in four adults (Cushway B., 2012).In 1997, only about 11% of US employers were using the web for outside recruiting, today 80%, and is expected in the next few years to reach 100% (Cohen D., 2001). In USA, a survey of Bernard Hodes Advertising shows that, consulting Internet is on second place after reading printed media or media needed in recruitment communication (Segretain E., 1998). An IRS survey established in 2004 that 84% of employers made some use of electronic recruitment. It was noted by IRS that the Internet is now a fundamental part of the recruitment process (Armstrong M., 2007). Internet recruiting will definitely continue to play a very prominent role as the world becomes more digitized (Bondarouk T., 2009). The HR Department is an important gatekeeper to a significant amount of critical information (Doran A., 2001). Information is becoming accessible through corporate networks and secure connections through the Internet.

345

Nicoleta Valentina Florea and Mihaela Badea

3. E‐Recruitment‐ advantages and disadvantages While traditional recruiting is characterized as a sequential batch process, e‐Recruitment is characterized as a continuous and online process in which some of the recruiting activities may be performed concurrently (Torres‐Coronas and Arias‐Oliva M., 2005). In today’s increasingly electronic world, the Internet (with its big capacity for text, graphics, interaction, and multimedia presentations) is an important tool for applicants and for organization to interact in an e‐business and e‐Commerce era. Technology has allowed organizations to send monthly email, newsletters and updates jobs right to the target candidates. E‐Eecruitment refers to marketing and promotional efforts aimed at recruiting prospective college students by electronic means (web, sms, e‐mail) (Doran A., 2001). Advantages The main advantages are winning time and money for candidates and for organization, too, though e‐ Recruitment has become an indisputable tool and the main competitor for mass‐media (Bournois F. and al., 2007). Web‐based recruitment activities range from prospective students portal; online submission of online applications; online examination; online confirmation and enrolment. Organizations e‐mail information and job opportunities directly to them at little cost. The process of e‐Recruitment consists of attracting, screening and tracking applicants, selecting, and offering jobs or rejecting candidates. A typical approach is to advertise the vacancy on an on‐line recruitment site (Armstrong M., 2007). This will provide job details and information about the organization. Fees for Internet job sites tend to be lower than for other employment sources, such as advertising and recruitment agencies. More than 30% of all recruitment advertising budgets was spent on the Internet in 2000 (Bodea C. et al., 2003). Firms can choose to list their openings on one of the more than 30,000 Internet job boards (Heneman R. and Greenberger D., 2002). For instance, online systems helped Cisco Systems attract more than 500,000 individuals in one month and enabled them to hire 1,200 people in three months’ time (Guental H.G. and Stone D.L., 2005). Some reports that online recruiting costs 95% less than traditional recruiting. For example, some estimates indicate that the cost of online recruitment is $900 as compared to $8,000 to $10,000 for traditional systems. In the same way, research shows that organizations can reduce hiring cycle times by 25% when using online recruitment, and can use these systems to provide easy and inexpensive realistic “virtual” previews to job applicants. Some indicators of using Internet: 71% of employers are using sites for employments, 92% from interviewed prefer Internet for the rapid access of the most recently posted jobs, 89% appreciate that there are specific criteria to search for a vacancy (Bournois F. et al., 2007) shows that e‐Recruitment strategy is the integration and utilization of Internet technology to improve efficiency and effectiveness of the recruitment process. Most organizations understand this and have begun the evolution by integrating e‐Recruitment strategy into their hiring process. The objectives of recruitment and selection process should be choosing a great number and more qualitative candidates at a less cost to satisfy the personnel needs of the organization (Armstrong M., 2006). Benefits of e‐ Recruitment (Hopkins B., 2003): speeding up the recruiting process; cutting out intermediaries; ethicality of computer‐based selection; improved accessibility to information. Apart from the reasons for using online recruiting noted above, these systems may also increase the effectiveness of the recruitment process by (Guental H.G. and Stone D.L., 2005):

Reaching large numbers of qualified applicants, including those in international labour markets,

Reducing recruitment costs,

Decreasing cycle time,

Streamlining burdensome administrative processes,

Enabling the organization to evaluate the success of its recruitment strategy.

Advantages obtained using e‐Recruitment (Bournois F. and al., 2007): winning time; lower costs comparing with mass‐media; relatively simple and usable technique for all; growing productivity compared with analyzing of written and submitted resumes. E‐Recruitment has several other advantages: low cost, quick response time, broad range of applicants, more educated applicants (Bondarouk T., 2009), worldwide accessibility (Galanaki E., 2002) at their own convenience‐ 24 hours, 7 days a week. It has been demonstrated a shorter recruitment cycle and lower cost‐per‐hire. Now, not only those people who look for a job, but passive job seekers who

346

Nicoleta Valentina Florea and Mihaela Badea would be interested in some working conditions or the recruitment message can easily browse the online advertisements. There were identified six categories of the e‐Recruitment sources (Torres‐Coronas and Arias‐Oliva M., 2005): 1. General purpose job board, 2. Niche job board, 3. E‐Recruitment application service provider, 4. Hybrid (online and off‐line) recruiting service provider, 5. E‐Recruitment consortium, 6. Corporate career website. Table 1: Summary of six categories of the e‐Recruitment sources E‐Recruitment sources 1.

2. 3.

Advantages

Disadvantages

Brand recognition, Experience, high‐traffic, best tools, large candidate base, Gathering of passive job seekers, focused search Low application, high cost

High cost, low quality application, limited candidate relationship Low brand recognition

Expertise in advertising industry Low service cost, direct and immediate link to corporate career site

Integration issues with existing systems, low traffic Strong image as a conventional media, low traffic, low tech

Candidate relationship management, integration with existing systems

Potential conflicts among members, low exposure, low tech Needs for IT specialists, high costs

Sample participants Monster.com, Hotjobs.com Dice.com, MKJobs.com Recruit USA, Lawson

New York times, Wallstreet Journal Direct Employees.com, NACElink 94% of Fortune 100 companies, 81% of Fortune 500 companies.

Adaptation after Torres‐Coronas and Arias‐Oliva M., HRM. Managing knowledge people, 2005, p.89 Other main advantages of using e‐Recruitment (Cushway B., 2012):

It can speed up the recruitment process and reduce administration,

It can improve efficiency of the recruitment process,

It has the potential to reduce costs,

The ability to reach a wide pool of applicants,

The ability to focus on a target applicant pool,

The ability to advertise vacancies widely internally through the use of the company Intranet,

It promotes the image of the company as a modern, up‐to‐date employer,

It offers 24‐hours, seven‐days‐a‐week access,

It can reach a global audience,

It makes handling high volumes of applicants easier,

It provides a databank of applicant information,

It can be used to provide more information about the organization and the job,

It makes it easier for the applicant to apply for a job.

One of the advantages of the web is that applicants can screen themselves in a confidential and safe‐ environment, globally accessible, information‐intensive, user‐friendly (Cohen D., 2001).

347

Nicoleta Valentina Florea and Mihaela Badea Disadvantages Some analysts have argued that are malfunctions of using such systems. For example, replacing traditional recruiters with computerized systems may make the recruitment process much more impersonal and inflexible and, therefore, have a negative impact on applicants’ attraction and retention rates. Applicants may perceive that online systems are more likely to invade personal privacy than other recruitment sources. As a result, applicants may be less willing to use e‐Recruitment systems than traditional systems to apply for jobs (Guental H.G. and Stone D.L., 2005). On the other side, the disadvantages of e‐Recruitment are that it could create disparate impact as economically disadvantaged groups might not have access to online facilities, due to social and economic classes, creating what scholars have termed a “digital‐divide” (Lancaster J., 2003). E‐Recruitment may exclude members of some minority groups which do not have access to the Internet for different reasons as well as people which are not familiar with the medium or which do not want to use it due to security matters (WuttkeF., 2008).Further, employees from the older age groups might perceive themselves to be less tech‐ savy than the current “digital‐generation” of employees, and therefore feel a sense of alienation (Bondarouk T., 2009). The employers advertising jobs are bombarded with hundreds of applications (Torrington D., Hall L. and Taylor S., 2005]. To prevent this it is necessary to make use of software for short‐listing, which is looking for keywords in CVs, but this inevitably will miss good candidates. Criticism have also been made about poor standards of ethicality on the part of cyber‐agencies, that are posting fictional vacancies, are circulated to employers unsolicited CVs, are copying CVs from competitors’ sites and send them on to employers without authorisation (Torrington D., Hall L. and Taylor S., 2005). Other main disadvantages of using e‐Recruitment (Cushway B., 2012):

Not everyone has access to the Internet,

Ease of applying for jobs might attract a greater number of unsuitable applicants,

It could limit applicants from groups who cannot fully use the technology, e.g. certain disabled groups,

If not carefully designed the process could discourage some applicants or give an unfavourable impression of the organization,

Some tests could potentially give rise to allegations of discrimination,

The process may be seen as impersonal, and put off some candidates.

Over time, as the industry grows, professional standards will be established and a regulatory regime established, but for the time being such problems will remain.

4. Types of online recruiting The main types of online recruiting are:

job sites‐ these are operated by specialized firms and can contain over 100,000 vacancies with 6 or 7 million “hits” a months. Companies pay to have jobs listed on the sites, which are not usually linked to agencies.

agency sites‐ are run by established recruitment agencies. Candidates register online but may be expected to discuss their details in person before their details are submitted to a prospective employer (Florea N.V., 2010).

media sites‐ which may simply contain a copy of an advertisement appearing in the press, but may include an external description of the vacancy and the company and provide a link to the company’s website.

interactive web page or panel survey – many organizations implemented this type of recruitment and it is addressed to students(e.g. Tesco) (Stredwick J., 2000).

E‐college recruiting‐ managers now use e‐mail and web to bypass campus career centres, and e‐letter students with exciting information to get them interested in their career. Resume robots and people finder programs identify candidates effortlessly.

Cv‐tequeis a database formed by existing resumes on the Internet (Guillot‐Soulez C., 2009);

348

Nicoleta Valentina Florea and Mihaela Badea

blog employment(le blog‐ emploi): some candidates choose to submit their applications on blog and some sites offer recruitment administration of these blogs;

recruitment through SMS: using SMS in recruitment start develop in France; this method is used especially to recruit for a job for the last minute, or for specialized jobs or for mass recruitment;

virtual professional networks: it refers to sites on the Internet to build professional networks (ex. Viadeo, Xing).

job dating‐ this method is often coupled with job boards and other recruitment methods: more organizations are grouping on the same site and make quick connections with the employees to convince them candidate (Benchemam F. and Galindo G., 2009);

speed dating‐ allows the meeting between candidate and recruiter as soon as possible.

Starting with 1990, using Internet, recruitment starts to develop more and more. Internet has modified recruitment practices for organization and for candidates as well (Guillot‐Soulez C., 2009):

for organization Internet is a new vector to receive the resumes;

for candidate Internet modifies how to search a job through the rapid dissemination of resumes and even by submitting resumes spontaneous.

Many organizations are finding ways of using Intranet‐based systems to speed up their recruitment process (Hopkins B., 2003). The Intranet can be a big asset to an organization (Doran A., 2001). Technology can be used to (Cushway B., 2012):

Advertise vacancies on your company’s website, on commercial job sites or job boards, or on social networking sites,

Track and process applications,

Select candidates through the use of online testing and by gathering applicant details.

The most common practices used for online recruitment involve (Guental H.G and Stone D.L., 2005):

Adding recruiting pages to existing organizational websites,

Using specialized recruiting websites,

Developing interactive tools for processing applications,

Using online screening techniques.

The organization’s image on the Internet can be a map and a barometer of organizational health. The laptop carrying case has replaced the briefcase as a business status symbol (Doran A., 2001). Technology development (table 2) and its use in human resources management activities, like e‐Recruitment and e‐selection, not only saves costs but also enables organizations to provide much more information to applicants, which can easily be updated. Table 2: Internet users for Romania in 2010 and 2012 Country Romania

Users 2010 7,786,700

2012 9,642,383

Population 2010 2012 21,959,278 21848504

Penetration(%) 2010 2012 35.5 44.1

Source‐ www.internetworldstats.com In light of these research findings, the following guidelines are suggested for HR professionals (Guental H.G. and Stone D.L., 2005):

Guideline 1: Organizations,that are well‐known or have prominent reputations,may benefit more from online recruitment tan those that are not well‐known.

Guideline 2: Organizations should use e‐Recruitment as one source of applicants, but should not use it as the only source for recruiting applicants.

349

Nicoleta Valentina Florea and Mihaela Badea

Guideline 3: Organizations should capitalize on the strengths of recruiting and use it when large numbers of candidates are needed or when they are searching for individuals in high‐level job requiring fairly high education levels. So, they should use e‐Recruitmentto target candidates in specific labor markets.

Guideline 4: Organizations should recognize that e‐Recruitment may not be the preffered source for most job applicants and may not always reach the types of applicants that can help the organization meet its goals. So, organizations should be aware that e‐Recruitment may not attract the most qualified job applicants.

Guideline 5: Organizations should ensure that their websites are designed to be easy to use, simple to navigate, and focus on attracting candidates rather than screening them for jobs.

5. Case study of large organizations from Dambovita County, Romania The analysis made in this article is only a part of our PhD research, which is using the Benchmarking model to analyse and compare the recruitment, selection, employment and integration strategies used by large organizations. This paper is about what recruitment sources the large organizations are using, and how they use e‐Recruitment as a new technology. The paper is a descriptive research based on both sources and empirical data sources. The empirical data is for the most part provided by the large organizations from Dambovita County. In the survey we made, at the question about which sources the analysed organizations use, organizations had answered (table 3): Table 3: The most used recruitment sources in organizations Organization Erdemir Oțel Inox Mechel UPET ELSID ARCTIC

Internal recruitment sources 40% 10% 60% 50% 60% 10%

External recruitment sources 60% 90% 40% 50% 40% 90%

The main internal recruitment methods used by organizations analyzed are: staff recommendations, display, and lists of foster employees. Intranet is used only by two of six large analised organizations. Intranet is not used as recruitment source, because the organizations have not developed such a program, Intranet being used only by OTELINOX and Arctic. Reasons why organizations turn to internal recruitment:

Arctic and Mechel OTELINOX use this method to provide career plans and personal development of their employees, to motivate employees performance and promote equal opportunity employee performance results and appreciate their results.

ELSID, ERDEMIR and UPET use internal recruitment due to low cost compared to external recruitment, knowledge organization, motivate employees and assessing results.

Table 4: The most used methods of internal recruitment Organization Erdemir Oțel Inox Mechel UPET ELSID ARCTIC

1‐very important SR D SR D D SR

2‐ important L L, I L L L D, I

3‐ insignifiant D SR D SR SR L

Note : SR ‐ staff recommendations, D ‐ display, L – lists of foster employees, I‐ intranet. The main external recruitment methods used by organizations subject research are:

350

Nicoleta Valentina Florea and Mihaela Badea Table 5: The most used methods of external recruitment Organization Erdemir Oțel Inox Mechel UPET ELSID ARCTIC

1‐very important SA, PAU I I PAU PAU I, EI

2‐ important RTV, HH RTV RTV, EI RTV SA RTV, SA

3‐ insignifiant EI, I EI PAU SA, EI RTV AR

Note: I ‐ Internet, EI‐ educational institutions, RTV‐ radio and TV, PAU‐ placement agencies for unemployed, RA‐ recruitment agencies, SA‐ spontaneous applications, HH‐ head‐ hunting. As external recruitment sources organizations are using placement agencies for unemployed, second place RTV sources and spontaneous applications, and on 3rd place educational institutions and Internet, head‐ hunting and recruitment agencies are the least used. Reasons why organizations turn to external recruitment: the most important reason is experience, new knowledge, skills and abilities, reduced training costs and reduce negative feelings for colleagues. The most used sources of recruitment are the cheapest, but also to achieve a low level of knowledge, experience and skills, such as CEA, posters at the organization gate, sometimes recommendation of employees (without giving any bonus to those who make these recommendations). Internet is used only by four of six organizations and recruitment agencies make use only by Arctic, and that only by two years, the remaining organizations are realizing recruitment by its own forces. Discussing with the HR specialists and viewing the organization’s web‐sites for recruiting we observed:

that organizations are not investing considerable resources in website development because they believe:

that the attractiveness, quality, and ease of use of the website will not influence applicants’ attraction to the organization,

Research has found that individuals prefer organizations with high‐quality websites and that website designs influence applicants’ impressions of the organization. As a result, the organizations argue that vacancy characteristics are much more important determinants of applicants’ attraction than the attractiveness of the sites.

6. Conclusions It is clear that many job applicants are now using online systems to search for jobs and gather information about employment opportunities in organizations. They often use online systems to attract passive job seekers who are currently employed, but secretly searching for new or better employment opportunities. Although e‐ Recruitment is widely used, there is still a great deal of uncertainty about its acceptance among job applicants. Our research shows that most applicants continue to prefer newspaper advertising, placement agencies for unemployed, radio and TV advertising, and spontaneous applications to e‐Recruitment. Our survey indicates that the Internet is not the number one source to look for a job for most candidates, maybe due to the low number of Internet users‐ 9,642,383 in 2012 at a population of 21,848,504 (rate of penetration being 44.1% in June 2012) (www.internetworldstats.com). Many applicants still rate employee referrals and personal recruitment more favourably than the Internet because they can gather realistic information about the organization from the current employees. E‐Recruitment is something that cannot be ignored but the process should conform to all the criteria of more traditional approaches and should be regularly monitored to ensure that it is delivering the required results. It should be fully integrated with other recruitment methods. It is also important to ensure that websites are kept up to date and that there is a contact number for those having technical difficulties.

References Armstrong M.(2006), Managementul resurselor umane, Editura CODECS, Bucuresti; Armstrong M.(2007), A handbook of human resources management practice, Kogan Page, USA; Bach S.(2005), Managing HR: personnel management in transition, Blackwell Pub., USA;

351

Nicoleta Valentina Florea and Mihaela Badea Benchemam F. and Galindo G.(2009), Gestion des resources humaines, Gualino ed., Paris; Bodea C. et al.(2003), Human resources management in the Internet Age: e‐Recruitment and e‐selection methods, Economy Informatics, pp.5‐7; Bondarouk T.(2009), Handbook of research on e‐transformation and human resources management, Idea Group Inc.; Bournois F.et al.(2007), RH –les meilleures pratiques, Eyrolles Ed. D’organisation, Paris; th CCH Wolters Kluwer business(2012), Australian master HR Guide, 8 Edition, McPherson’s Printing Group, Australia; Cohen D.(2001), Web‐based recruiting and staffing, Towers Perrin, McGraw‐Hill, USA; Cooper D. et al.(2003), Recruitment and selection. A framework for success, Thomson, UK; th Cushway B.(2012), The employer’s handbook, 9 Edition, Kogan Page Ltd., UK; Czerney A.(2004), Not so qiuck and easy, People management, 26 febr.; Doran A.(2001), E‐work arhitect‐how HR leads the way using the Internet, IHRIM Press Book, Austin, Texas, USA; Epstein R. and Singh G.(2003), Internet recruitment effectiveness: evidence from a biomedical device firm, In.Journal of HRD&M, 3(3), 216‐225; Florea N.V.(2010), „E‐Recruitment: using Internet and Intranet to improve the organization performance”, 11th European Conference on E‐COMM‐LINE 2010, IPA and ASE, 27‐28 sept.2010, CNCSIS, ISBN 9789731704180, secțiunea 3, p.1‐ 10; Guental H.G. and Stone D.L.(2005), The brave new world of e‐HR. HRM in the digital age, Jossey‐Bass a Wiley Print, CA, USA; Galanaki E.(2002), The decision to recruit online, Career development international, 243‐250; e Guillot‐Soulez C.(2009), La gestion des ressources humaines, 2 Edition, Gualiano, Paris; Heneman R and Greenberger D.(2002), Human resources management in virtual organizations, Inf. Age Pub.Inc.; Hopkins B. and Markham J.(2003), E‐HR: using intranets to improve the effectiveness of your people,Gower Pub.Ltd., England; Lancaster J.(2003), Village kiosks bridge India’s digital divide, Wa.Post, oct.12th, A01; Pell A.R., Managementul resurselor umane, Editura Curtea Veche, Bucuresti, 2008, Price A.(2000), Priciples of human resources management:an active learning approach, Blackwell Business, Oxford, UK; Reynolds D.H. and Weiner J.A.(2009), Online recruiting and selection innovations in talent acquisiyion, Wiley‐Blackwell, UK; Segretain E.(1998), Le marketing des resources humaines, Maxima, Paris; Stredwick J.(2000), An introduction to human resource management, Butterworth‐Heinemann, Great Britain; Torres‐Coronas T. and Arias‐Oliva M.(2005), E‐HRM. Managing knowledge people, Idea Group Pub., USA; Torrington D., Hall L. and Taylor S.(2005), Human resources management, Pearson Edu.Ltd., Harlow, England; Waddill D.D and Marquand M.J.(2011), The e‐HR advantage. The complete handbook for technology enabled HR, Boston, USA; Wuttke F.(2008), E‐Recruitment vs. traditional recruitment. A descriptive analysis, GRIN Verlag, Germany; www.internetworldstats.com.

352

Identifying Entrepreneurial Competencies Which Lead to Innovative Performance in Malaysian Academic Libraries Haziah Sa’ari, Rusnah Johare, Jafalizan Md. Jali, Asmad Rizal Umar and Hafiza Adenan Faculty of Information Management, Universiti Teknologi MARA (UiTM), Malaysia azie.crystal@gmail.com r.johare@yahoo.com jeffchaosz@gmail.com asmadrizal@yahoo.com hafizaadenan@ymail.com Abstract: This paper identifies entrepreneurial competencies and the outcomes measured as innovative performance demonstrated by academic librarians. By using a literature review, purposive sampling, multiple case studies and interviews we discovered that the antecedents of entrepreneurial competencies include recognizing opportunities, initiating innovation and strategic thinking. These antecedents influence academic librarians’ innovative performance which was evaluated based on generativity or idea generation, championing and formative investigation. This study explores entrepreneurial competencies and innovative performance in Malaysian research universities. Keywords: entrepreneurial competencies, innovative performance, academic librarian

1. Introduction The effort to put a stronger focus on wealth creation and economic activities based on knowledge and innovation proves that Malaysia is determined to become an international hub for education and a source of highly skilled knowledge workers. In order to enhance the reputation of Malaysian universities, five publicly funded higher institutions have been granted research university (RU) status namely Universiti Malaya, Universiti Kebangsaan Malaysia, Universiti Sains Malaysia, Universiti Putra alaysia and Universiti Teknologi Malaysia based on their outstanding achievements as knowledge activity clusters (Nordin, 2011). Among the criteria for determining research status are the support facilities and professional services offered by academic libraries in those universities selected. In addition to the recognition of their knowledge activities, librarians have many opportunities to expand their current roles and demonstrate the skills they have honed to meet organizational objectives especially managing information which has emerged as a central focus of the current knowledge economy (Hayes, 2004). However, librarians have not taken advantage of the changing information landscapes and their competencies are being questioned. specially since the great economic recession has had a negative impact on library expenditure and on the standing of librarians. As well, the abnormal increase in the costs of resources has put pressure on librarians to maintain existing subscription lists as required by users even though current costs make this impossible (Bosch et. al. 2011). Similarly in Malaysia, the issue of academic librarians’ competency is being debated. For example, the study conducted by Kaur (2009) has revealed that academic librarians lack the capabilities to fully utilise library websites as a marketing tool to reach out to the community and as a source of information. Hence, strategies to market information resources for optimum utilisation are challenging tasks for librarians in RU (USM, 2010). To overcome these challenges librarians should demonstrate their entrepreneurial competencies as a crucial component of the university’s research capacity (Feret & Marcinek, 1999; Carpenter, 2012; Noh, 2012). By reviewing the literature on entrepreneurial competencies and innovative performance and through interviews, this paper is intended to explore the entrepreneurial competencies of academic librarians and to measure the impact of these competencies on innovative performance among academic librarians in Malaysian research universities. To achieve its objectives, the following research questions were used to guide the study: (a) how do academic librarians demonstrate entrepreneurial competencies? (b) how is innovative performance measured by the top management of academic libraries? The paper is structured as follows. In the next section, we examine three antecedents of entrepreneurial competency and explain the outcomes of innovative performance which can be measured. In section 3, we discuss the methodology and research design used to measure the variables. We report our empirical results in section 4. In section 5, we provide a conclusion and outline directions for future research.

353

Haziah Sa’ari et al.

2. Literature review Entrepreneurial competencies are essential for stimulating entrepreneurial activity and are recognized as a driving force and source of innovation and competitive advantage (Rae, 2007). Entrepreneurial competencies can be defined as a cluster of related entrepreneurial knowledge, skills and personality traits which can lead to the production and accomplishment of outstanding performance or maximizing profits (Man et. al. 2002). Bird (1995) outlines a definition of entrepreneurial competences as the underlying characteristics of an individual which include both generic and specific prior knowledge, intention, personality traits, self branding, social role and skills which results in new ventures, survival and/or growth. These competencies are further demonstrated by individuals as entrepreneurs who are able to manipulate resources and opportunities which can increase an organizations’ profitability. To further understand the behaviours that are associated with entrepreneurial competencies, the following discussion of each key characteristic is provided.

2.1 Recognizing opportunity Most entrepreneurial literature discusses the same concepts and themes in defining ‘recognizing opportunity’ as forming new methods and/or results, to improve or introduce new products, services, raw materials and organizing methods rather than utilizing an existing framework. Shane and Venkataraman (2000) outlined their definition as a process to identify, recognise, and discover potential opportunities for new business, ventures, market and technology creation. The requirements for recognizing opportunities could be categorized into three different perspectives as follows: (i) a cognitive psychology perspective as in Pech and Cameron (2006) proposed that the ability for recognizing opportunity is embedded in the development of an individual’s cognition including prior experience, self‐efficacy, beliefs and values which impact and provide insights in the decision process. In addition, cognitive ability is also important to one’s ability to exploit specific bits of information in order to create opportunities for innovative services/products (Busenitz et. al. 2003). The other cognitive elements such as self‐alertness (Kirzner, 1979), prior knowledge (Shane, 2000) and self‐efficacy (Ozgen, 2003) are also parallel pre‐requirements for recognizing opportunity; (ii) the social network perspective explains an individual’s interpersonal networks, including weak‐tie networks and strong‐tie networks contribution to information flow about the task activity of individuals in organisation. Strong‐tie is important in promoting information flow within an organisational whereas weak‐tie is useful for encouraging information flow outside an organisational structure (Friedkin, 1982). In this light, weak‐tie networks and strong‐tie networks facilitate access to diversity in people, information and benefits in the learning and information dissemination process which leads to the discovery of further opportunities and new contacts (Ardichvili & Cardozo, 2000); and (iii) the environmental perspective describes the external environment such as new technology and social values which would motivate individuals to recognise opportunities (Ferreira, 2002).

2.2 Initiating innovation Innovation is defined as the initiation of a new product, or an iterative process to create new products, processes, knowledge or services by using new or existing scientific knowledge (Kusiak, 2007). Damanpour (1996) also included improvements of administrative procedures as innovation. West and Farr (1990) mentioned that innovation is the purposeful formation of creativity and application within a role, group or organisation of ideas, processes, products or procedures, new to the relevant unit of adoption. These are designed to significantly benefit the individual, the group, the organisation or wider society. Basically, the literature demonstrates that the method of innovation is conducted in an incremental mode which builds upon previous work and a radical manner which constitutes a totally new innovation. The extant literature suggests a number of individual traits and characteristics which influence an innovation propensity within organisations. In 1999, Sternberg and Lubart proposed an ‘investment theory’, suggesting that the propensity to innovate requires a confluence of six distinct resources including intellectual abilities, knowledge, styles of thinking, personality, motivation and environment. Other integrative approaches include the “geneplore” model (Finke et. al. 1996) and the “componential model” (Amabile, 1996) which focuses at the person level associates to innovative behaviour. Among the pre‐requirements for initiating innovation are: (i) cognitive abilities and mental processes associated to (a) general intelligence (Lubart, 2003); (b) genius and; (c) observer judgments of intelligence (Feist & Barron, 2003); (ii) awareness of specific knowledge where it reflects how much an individual knows about a required discipline in doing a specific task (Mascitelli, 2000); (iii) self‐ motivation (Sauermann, 2008); (iv) proactive including personal initiative (Frese et. al., 2007); and (v) voice

354

Haziah Sa’ari et al. behaviour which means the ability to speak up or verbalise ideas and share them with others (LePine & Dyne, 1998).

2.3 Strategic thinking Strategic thinking is the process of determining, shaping and achieving an inspired or desired future. The process involves a creative and provocative engagement of people in the organisation that might effect a business’s future direction. According to Stacey (1992), strategic thinking is about constructing creative new ideas and designing actions on the basis of new learning. It also involves discovering novel, imaginative strategies for gaining competitive advantage and for envisioning potential futures significantly different from the present (Heracleous, 1998). Mintzberg (1994) points out that strategic thinking will produce strategic planning which involves the systematic programming of pre‐identified strategies. Bonn (2001) also agrees with this view and states that the strategic planning process takes place after strategic thinking. Based on the work of others, Liedtka (1998) developed a model which defines strategic thinking as a particular way of thinking, with very specific and clearly identifiable characteristics. The model consists of five key elements which are: (i) systems perspective/system thinking which refers to the ability to clarify one’s roles and the impact of one’s behaviour within the larger system; (ii) intent‐focused refers to the ability to leverage energy, focus attention and concentrate in order to achieve a goal; (iii) intelligent opportunism refers to the openness to new experiences which allows one to take advantage of alternative strategies; (iv) thinking in time refers to the ability to connect the past, present, and future as critical inputs; and (v) hypothesis‐driven refers to the ability to think and judge critically. On the other hand, Webster (1988) suggests that market‐oriented thinking has considerable impact on strategic thinking and on its usefulness in enhancing a firms' marketing performance. The study by Kumar et. al. (2011) supported embracing a market orientation as one way to respond to competitive intensity in the marketplace. With the strategic market management, managers are able to scan the environment and develop proactive strategies to improve performance in competitive situations.

2.4 Innovative performance Innovative performance is rooted in innovative behaviour demonstrated by employees in organizations in order to create new process, products or improve administration processes and vital contributor to the effective functioning of organizations (Amo & Kolvereid, 2005). Innovative behaviour is a vital contributor to the effective functioning of organizations (Woodman et. al., 1993). The measurement of innovative performance among employees in organisations is recommended. Immediate supervisors should rate the employees innovative work behaviours in the workplace. This is the most common approach used in the measurement of individual innovative performance and can be based on: (i) generativity which entails behaviours including generating ideas and solutions to opportunities (Howell & Higgins 1990); (ii) championing including persuading and influencing (Kanter, 1988), pushing and negotiating (Van De Ven & Rogers 1988); (iii) formative investigation including formulation of ideas and solutions, experimenting with ideas and proposing solutions (Amabile, 1988).

3. Research methodology Understanding from the academic librarian’s point of view, experience and interpretation. The methodology may provide insights into competencies from a process perspective that traditional quantitative methods may not bring to light (Hindle & Yencken, 2004). Furthermore, Smith and Morse (2005) affirm that due to the broad concept of competencies, it is a challenge to measure something which is elusive and slippery and which manifests itself differently in different contexts, changes over time and is not directly observable. In using multiple case studies, academic libraries working in three public‐funded RU located in the Klang Valley of Peninsula Malaysia were evaluated in this study. We scrutinised entrepreneurial competencies and innovative performance among academic librarians. We note that many of these elements are among the competencies required to fulfil the client’s needs in a changing information environment (Partridge et. al., 2010; Jantz, 2012). Hence, we sought through interviews the interpretation and viewpoints of the academic librarians which we consider vital data when exploring entrepreneurial competencies and innovative performance in the three academic libraries. We undertook four interview sessions (three sessions were conducted with representatives of the departments of Reference, Information Technology and Acquisition and another one session was conducted with chief librarians) in each RU. The sample was selected by using purposive sampling to select information‐rich cases that can be studied in‐depth (Perry, 2001) and to identify a set of key informants who constituted a source of multiple perspectives (Strauss & Corbin, 1990) in different library departments. In total,

355

Haziah Sa’ari et al. forty‐five academic librarians and three chief librarians were invited to participate in the study. Permission to conduct the research was obtained from the top management of the academic libraries involved. We sent a letter to each of the academic librarians selected to inform them about the research and the date that the interview would take place. Each letter clearly explained the objectives of the research including the confidentiality of the responses. The data collection was based on a structured questionnaire and semi‐ structured interviews. The questions posed were focused on how academic librarians demonstrate entrepreneurial competencies and how they were measured and evaluated by superiors/top management. Secondary data such as job descriptions, performance measurement tools and written information on the libraries’ innovative projects were also included in our study in order to gather relevant contextual knowledge about library management practices in those RU. All the interviews were recorded for subsequent coding with the interviewees’ agreement. The recorded data were transcribed verbatim to form texts and were analysed by coding.

4. Research findings Through coding analysis, the study found three dimensions of entrepreneurial competencies demonstrated by academic librarians namely recognizing opportunity, initiating innovation and strategic thinking. The antecedents related to the competency domain and frequency is shown in Table 1. Table 1: Entrepreneurial competencies of academic librarians Competency domain Recognising opportunity

Initiating innovation

Strategic thinking

Antecedent Knowledge and experience Self‐alertness Social network Environmental Motivation Specific knowledge Proactive Voice Cognitive ability System thinking Intent‐focus Intelligent opportunism Thinking in time Hypothesis‐driven

Frequency 83 50 40 45 80 67 73 76 45 60 58 40 45 37

4.1 Recognizing opportunity By combining the interviews, we analysed the entrepreneurial competencies as followed and discover the ways and methods to improve or develop new library services, library management processes and library resources. The ability of librarians to identify opportunity relates to all kinds of unmet needs around them and is identified primarily through their prior knowledge and experience. For instance, the increase of market demand for traditional medicine alerted librarians to prepare lists of the scientific names of herbs and plants as required by companies as that information must be submitted to the Ministry of Health in order to get product approval. Since the library does not have the complete list, in responding to the demand, librarians set up a library link with the Bio Science Institute of the same university in order to supply the information to the users. Knowledge in computer network infrastructure enabled librarians to develop the link between the computer system and the library’s computer server and was essential to developing the service. In addition, prior knowledge in botany strengthens the librarians’ ability to organise the collective information and as such the library was successful in marketing the university’s resources effectively. One of the informants mentioned, “We have been selected for the Excellence Library Award 2011 because of the recognition of the value of our user guide and the agricultural portal as an important service. The development of new products like this, enable widespread dissemination of knowledge in Malaysian research agriculture which is beneficial for users all over the world”. Consistent with Ko and Butler’s (2006) identification of prior knowledge and experience as influencing entrepreneurial opportunity identification, we discovered that most librarians evaluated entrepreneurial opportunities based on their prior knowledge. This prior knowledge gained from industry experience in various types of libraries was a major factor in opportunity identification compared with those who are new to librarianship. Therefore, as a mirror of the university, the library should always recognise

356

Haziah Sa’ari et al. opportunities to provide high quality responses and to take on the challenge of fulfilling its customer’s demands.

4.2 Initiating innovation The problems and obstacles arising in the work place have motivated librarians to create and improve library processes and services. At the same time, top management urges and pressures librarians to use their initiative and be self‐starting in leveraging ideas to overcome problems or improve performance. For example, one of the librarians soberly said, “For seven years in this department, I experienced three changes in library acquisition circulars. These circulars have impacts, and the biggest was in 2009, when controls on library purchases were implemented and approval of the appointed committee was required.” Through the interviews, we find that initiating innovation in academic libraries is pursued in an incremental manner. On the other hand, the radical method is not fully accepted as libraries are limited by government policies and established management procedures. Significantly with Sanger and Levin (1992), the public sector is typically evolutionary and does not innovate from scratch, thus there are relatively minor changes and adaptations to existing services or processes. Initiating innovation is brought about by intrinsic motivation which links work to task (Amabile, 1988). It is also proactive in problem solving (Frese et. al. 2007) which involves an individual taking charge of an idea for improving the workplace, either by voicing the idea to others or by self‐ implementing the idea as a condition of innovation. For example, one interviewee said, “I do love this job and always thought towards efficiency. On the first day I was attached to the Acquisition department, I discovered there was a problem in monitoring the work process from subscribed materials to making it ready for users. When there is no systematic monitoring, the flow of material cannot be tracked and complaints arise from users due to delays in getting the materials. Therefore, in one of the staff meetings, I proposed an idea to develop a portal that we call an e‐click to monitor and speed up the preparation of material for customer usage”.

4.3 Strategic thinking We sum up strategic thinking competency as the ability to address issues, make decisions, and foresee the possible futures. For example, one of the librarians soberly said, “I do believe that the library is a part of the parent organisation. We should demonstrate our value and importance to the university and that we fit in with current university missions as well as looking to the future as to what will be have value next, so that we can begin to prepare an appropriate strategy on how to align with the future”. Strategic thinking facilitates the maximization of organisational resources, adding value to businesses and evoking positive change (Grant, 1991). The strategic thinking behaviour among librarians includes focusing on finding and developing unique opportunities to create value such as working on unexploited existing information and knowledge that the organization already possesses, for innovations to improve organizational performance or customer value. For instance, one interviewee said, “We put our concentration on reaching out resources and research reports which are scattered everywhere in faculties and put them together in the library database. With the trend to pushing those items out into the digital sphere, we would do the digitalisation in stages as it would be expensive to them all at once; consume a lot of time and be labour intensive. But it is done enthusiastically”. The finding is consistent with Skrzeszewski (2006) who stated that librarians have the ability to manipulate the intellectual assets by identifying and realizing an opportunity, rather than exploiting it. In the next section, we provided the analysis of innovative performance measurement.

4.4 Innovative performance In the analysis of the interview texts, we find that innovative performance is the ability to generate, introduce and apply new process, products and services or improve administration processes to increase performance and enhance work efficiency. Management encourages librarians to share their ideas in solving problems or finding resolutions of any issues/conflicts. The idea would be potentially accepted and implemented into innovative endeavours after taking into consideration the novelty, practical, economic and relevancy elements. Further, the ideas generated will be measured based on the librarian's ability to win support from others through presentations, meetings, paperwork and by vigorously convincing others of the importance of realizing these ideas. To promote the ideas, the librarian participates in innovative competition before ideas/product are patterned and commercialised. The rewards and recognition given by the top management to those who initiated or generated ideas is regarded as extrinsic motivation which affects individual effort and

357

Haziah Sa’ari et al. the overall quality of the innovative endeavours and thus, increase the propensity for innovation (Danish & Usman, 2010).

5. Conclusion The entrepreneurial competencies identified by academic librarians are encouraging and they are agreed that the competencies should be embraced for sustainability. Overall, the findings are also significant indicating that librarians should adopt an entrepreneurial approach to be more innovative and creative about promoting resources and services as well as developing library products for revenue generating opportunities and maximising working competitiveness. We recognize this study is subject to some important limitations. The study is limited due to the lack of standards and appropriate measurements for innovative performance for Malaysian academic libraries in the public sector. According to Siggins (1992) reward and performance assessment is crucial to stimulate behaviour and encourage organizational commitment. To be successful, such a system must be perceived to be equitable, realistic in its expectations, and conforming to a set of clearly understood and accepted standards. The study is also limited to the public sector; future research in the non‐ profit and the private sectors is recommended. In addition, the next studies should compare public and private sector organisations with similar entrepreneurial competencies to determine if there are differences in innovative performance measurements between the sectors.

Acknowledgements A special note of gratitude goes to Stan Skrzeszewski for sharing wisdom, knowledge and encouragement.

References Amabile, T.M. (1988) "A model of creativity and innovation in organizations”, Research in Organizational Behavior, Vol. 10, pp 123‐167 Amabile, T. M. (1996) Creativity in context: Update to the social psychology of creativity. Westview Press, Boulder, CO. Amo, B. and Kolvereid, L. (2005) “Organizational strategy, individual personality and innovation behaviour”, Journal of Enterprising Culture, Vol. 17, No. 1, pp 7‐19 Ardichvili, A. and Cardozo, R. (2000) “A Model of the Entrepreneurial Opportunity Recognition Process”, Journal of Enterprising Culture, Vol. 8, No. 2, pp 103‐119 Bird, B. (1995) “Towards a Theory of Entrepreneurial Competency”, in Katz, J.A. and Brockhaus, R.H. (Eds), Advances in Entrepreneurship, Advances in Entrepreneurship, Firm Emergence and Growth, Vol. 2, JAI Press, Greenwich, CT, pp 51‐72 Bonn, I. (2001) "Developing strategic thinking as a core competency", Management Decision, Vol. 39, No. 1, pp 63 – 71 Bosch, S., Henderson, K. and Klusendork, H. (2011) “Under Pressure, Times are Changing”, Library Journal, Vol. 136, No. 8, pp 30‐34 Busenitz, L., West, G., Shepherd, D., Nelson, T., Chandler, G. and Zacharakis, A. (2003) “Entrepreneurship research in emergence: Past trends and future directions”, Journal of Management, Vol. 29, No. 3, pp 285–308 Carpenter, M. (2012) “Cheerleader, Opportunity Seeker, and Master Strategist: ARL Directors as Entrepreneurial Leaders”, College & Research Libraries, Vol. 73, No. 1, pp 11‐32 Damenpour, F. (1992) “Organizational Size and Innovation”, Organization Studies, Vol. 13, No. 3, pp 375‐402 Damanpour, F. (1996) “Organizational Complexity and Innovation: Developing and Testing Multiple Contingency Models”, Management Science, Vol. 42, No. 5, pp 693‐716 Danish, R. and Usman, A. (2010) “Impact of Reward and Recognition on Job Satisfaction and Motivation: An Empirical Study from Pakistan”, International Journal of Business and Management, Vol. 5, No. 2, pp 159‐167 Feist, G. and Barron, F. (2003) “Predicting creativity from early to late adulthood: Intellect, potential, and personality”, Journal of Research in Personality, Vol. 37, No. 2, pp 62–88 Ferreira, J. (2002) “Corporate entrepreneurship: A strategic and structural perspective”, Paper read at 47th Proceedings of the ICSB, Puerto Rico, June Finke, R. A., Ward, T. B., & Smith, S. M. (1992) Creative cognition: Theory, research, and applications. MIT Press, Cambridge Frese, M., Garst, H., and Fay, D. (2007) “Making things happen: Reciprocal relationships between work characteristics and personal initiative in a four‐wave longitudinal structural equation model”, Journal of Applied Psychology, Vol. 92, No. 4, pp 1084‐1102 Friedkin, N. (1982) “Information Flow Through Strong and Weak Ties in Intraorganizational Social Networks”, Social Network, Vol. 3, No. 4, pp 273‐285 Grant, R. (1991) “The Resource‐Based Theory of Competitive Advantage: Implications for Strategy Formulation”, California Management Review, Vol. 33, No. 3, pp 114‐135 Hayes, H. (2004) “The Role of Libraries in the Knowledge Economy”, Serials, Vol. 17, No. 3, pp 231‐238 Heracleous, G. (1998) “Strategic thinking or strategic planning”, Long Range Planning, Vol. 31, No. 3, pp 481‐487 Hindle, K. and Yencken, J. (2004) “Public research commercialization, entrepreneurship and new technology‐based firms: an integrated model”, Technovation, Vol. 24, No. 10, pp 793‐803

358

Haziah Sa’ari et al. Hirst, G., Knippenberg, D., Chin‐Hui, C., and Sacramento, C. (2011) “How Does Bureaucracy Impact Individual Creativity? A Cross‐Level Investigation of Team Contextual Influences on Goal Orientation‐Creativity Relations”, Academy of Management Journal, Vol. 54, No. 3, pp 624–641 Howell, J. and Higgins, C. (1990) “Champions of change: identifying, understanding, and supporting champions of technological change”, Organizational Dynamics, Vol. 19, No. 1, pp 40‐55 Jantz, Ronald C. (2012) "Innovation in Academic Libraries: An Analysis of University Librarians' Perspectives", Library and Information Science Research, Vol. 34, No. 1, pp 3‐12 Kohli, A., Jaworski, B. and Kumar, A. (1993) “MARKOR: a measure of market orientation”, Journal of Marketing Research, Vol. 30, No. 4, pp 467‐477 Kanter, R. (1988) When a thousand flowers bloom: Structural, collective and social conditions for innovation in organizations. In B. M. Straw & L. L. Cummings (Eds.), Research in Organizational Behavior, Vol. 10, pp 123‐167 Kaur, K. (2009) “Marketing the academic library on the web”, Library Management, Vol. 30, No. 6, pp 454‐468 Kirzner, I. (1979) Perception, Opportunity, and Profit: Studies in the Theory of Entrepreneurship, University of Chicago Press, Chicago Kleysen, R. F., and Street, C. T. (2001) “Towards a multi‐dimensional measure of individual innovative behaviour”, Journal of Intellectual Capital, Vol. 2, No. 3, pp 284‐296 Ko, S. and Butler, J. (2006) “Prior knowledge, bisociative mode of thinking and entrepreneurial opportunity identification”, Int. J. Entrepreneurship and Small Business, Vol. 3 No. 1, pp 3‐16. Kumar, V., Jones, E., Venkatesan, R. and Leone, R. (2011) “Is Market Orientation a Source of Sustainable Competitive Advantage or Simply the Cost of Competing?”, American Marketing Association, Vol. 75, pp 1‐31. Kusiak, A. (2007) “Innovation Science: A Primer”, Int. J. Computer Application in Technology, Vol. 28, No. 2, pp 140‐149 LePine, J.A. and Dyne, L.V. (1988) “Predicting Voice Behavior in Work Groups”, Journal of Applied Psychology, Vol. 83, No. 6, pp 853‐868 Liedtka, J. (1998) “Strategic thinking; can it be taught?”, Long Range Planning, Vol. 31, No. 1, pp 120‐129 Lubart, T. (2003). Psychologie de la créativité. Armand Colin, Paris Man, T., Lau, T. and Chan, K. (2002) “The competitiveness of small and medium enterprises ‐ A conceptualization with focus on entrepreneurial competences. Journal of Business Venturing, Vol. 17, No. 2, 123‐142 Mascitelli, R. (2000) “From Experience: Harnessing Tacit Knowledge to Achieve Breakthrough Innovation”, Journal of Product Innovation Management, Vol. 17, No. 3, 179–193 Mintzberg, H. (1994) “The rise and fall of strategic planning”, The Free Press, New York Noh, Y. (2012) "The Impact of University Library Resources on University Research Achievement Outputs", ASLIB Proceedings, Vol. 64, No. 2, pp 109‐133 Nordin, K.M. (2011) “Higher Education in the New Economy: Roadmap of Prospects and Challenges”, Speech presented at Sunway Resort Hotel and Spa, Selangor, Malaysia Ozgen, E. (2003) Entrepreneurial Opportunity Recognition: Information Flow, Social and Cognitive Perspective, PhD Dissertation, Rensselaer Polytechnic Institute, New York Partridge, H., Lee, J. and Munro, C. (2010) “Becoming “Librarian 2.0”:the skills, knowledge, and attributes required by library and information science professionals in a Web 2.0 world (and beyond)”, Library Trends, Vol. 59, No. 1/2, pp 315‐335 Pech, R. and Cameron, A. (2006) “An Entrepreneurial Decision Process Model Describing Opportunity Recognition”, European Journal of Innovation Management, Vol. 9, No. 1, pp 61‐78 Perry, C. (2001) "Case research in marketing", The Marketing Review, Vol. 1, No. 3, pp 303–323 Rae, D. (2007) Entrepreneurship: From Opportunity to Action, Palgrave Macmillan, New York. Sanger, M. and Levin, M. (1992) “Using Old Stuff in New Ways: Innovation as a Case of Evolutionary Tinkering”, Journal of Policy Analysis and Management, Vol. 11, No. 1, pp 88‐115 Sauermann, H. (2008) Individual incentives as drivers of innovative processes and performance, PhD Dissertation, Duke University Shane, S. (2000) “Prior Knowledge and the Discovery of Entrepreneurial Opportunities”, Organization Science, Vol. 11, No. 4, pp 448‐469 Shane, S. and Venkataraman, S. (2000) “The Promise of Entrepreneurship in the Field of Research”, Academy of Management Review, Vol. 25, No. 1, pp 217‐226 Siggins, J. (1992) “Job Satisfaction and Performance in a Changing Environment”, Library Trends, Vol. 41, No. 2, pp 299‐315 Skrzeszewski, S. (2006) The Knowledge Entrepreneur, Scarecrow Press, Maryland. Smith, B. and Morse, E. (2005) Entrepreneurial Competencies: Literature Review and Best Practises, Small Business Policy Branch Industry Canada, Ottawa Stacey, R. (1992) Managing the Unknowable. Jossey‐Bass, San Francisco. Strauss, A. and Corbin, J. (1990) Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications, Newbury Park, CA USM Annual Report. (2010) Universiti Sains Malaysia (USM), Penang Van De Ven, A. and Rogers, E. (1988) "Innovation and organisations: critical perspectives", Communication Research, Vol. 15, pp 632‐51 Webster, F. (1988) “The Rediscovery of the Marketing Concept,” Business Horizons, Vol. 31, No. 3, pp 29‐39

359

Haziah Sa’ari et al. West, M. and Farr, J. (1990) Innovation at work. In M.A. West and J.L. Farr (eds.): Innovation and creativity at work: Psychological and organizational strategies. Wiley, Chichester Woodman, R.W., Sawyer, J.E., and Griffin, R.W. (1993) "Toward a theory of organizational creativity," The Academy of Management Review, Vol. 18, No. 2, pp 293‐321 Zhou, J. and Shalley, C. (2003) “Research on employee creativity: A critical review and directions for future research”, Research in Personnel and Human Resource Management, Vol. 22, pp 165‐217

360

Information Sharing Problems and Construction Efficiency Huan Vo‐Tran1, Brian Corbitt2, Sittimont Kanjanabootra3 and Peter Macauley1 1 School of Business IT & Logistics, RMIT University, Melbourne, Australia 2 Graduate School of Business, RMIT University, Melbourne, Australia 3 School of Architecture and Built Environment, University of Newcastle, Callaghan, Australia huan.vo‐tran@rmit.edu.au Abstract: Information can be seen as one of the most important assets that an organisation can possess. Utilised correctly, information may allow an organisation, irrespective of its industry or domain, to communicate, operate, plan, and make decisions that would ultimately be beneficial to itself, clients and any other entities that interact with them. This paper will report upon the differences in information management practices for a team of architects and builders. The study reports how they cycle through the inspection stage of a complex construction project involving a multi‐million dollar state‐of‐the‐ art purpose‐built educational building for a prominent Australian university. The study explores how specific domain knowledge and experience plays its part in the way information is documented, maintained and disseminated. The paper articulates the major differences (and any similarities) between the processes and techniques used by the respective architects and builders. It also explores the reliance of tacit knowledge in comparison to that of explicit knowledge. The data was collected over an intensive fourteen‐week period in which the investigators used of a combination of techniques, which included: document analysis, observations, and one‐on‐one interviews with builders and architects. Academic rigour was maintained through the triangulation of the data collection process in which data went through a verification stage by third parties via a novel method of experienced researchers ‘observing the observing’. This provided a useful check and balance through the data collection stages. While this novel method will be discussed, the paper will focus on the following major findings: The multiple methods of information recording, coding and storage amongst builders and

architects The relationship between tacit knowledge and experience in this specific knowledge domain. Information gathering happing in an ‘ad‐hoc’ manner The differences in the level of detail collected, stored and disseminated The numerous amounts of lists created and the duplication of data. Finally, the paper will discuss the implications for future on‐the‐job collaboration between architects and builders and also the available technologies, which may facilitate more cohesive and comprehensive information management practices within the construction industry. Keywords: information management, construction industry, knowledge management, information sharing

1. Introduction Information can be seen as an integral part of any organisation and one of the most valuable assets that it may possesses. Irrespective of an organisation’s industry or domain, information has the ability to facilitate operations, communication, planning, and decision‐making that would ultimately be beneficial to itself, clients and any other entities that may interact with them. As organisations continue to create, disseminate, store and dispose of their information, it is evident that the management of this becomes imperative. It is not just the traditional paper‐based documents that these organisations have to deal with, but instead, information that manifests itself via a multitude of forms and formats. In addition to this, studies (Blumberg & Atre 2003; Jenkins, Glazier & Schaper 2004,) have indicated that there is an increasing amount of unstructured data being created by organisations, The following paper reports the differences in information management practices between a team of architects and builders as they cycle through the inspection stage of a complex construction project involving a multi‐million dollar state‐of‐the‐art purpose‐built educational building for a prominent Australian university. The report is done through the lenses of information sharing and information audits.

2. Research contexts 2.1 Information management within the construction industry In all sorts of projects, including construction, the way people categorise documents and manage their information is very personalised. This most often linked to their experiences and the specific domain knowledge they possess. The combination of these two factors influences the way they make use of

361

Huan Vo‐Tran et al. information and studies have demonstrated that this changes over time as well as from project to project (Arnorsson, 2012; Benedict & David, 2012; Peansupap & Walker, 2006). The concept of information management within the construction industry is not new and has been reported in various previous studies (Al‐Sudairi, 2007; Al Nahyan, Sohal, Fildes, & Hawas, 2012; Chen, Zeng, & Xie, 2008; Haiyan, Tramel, Hwang et al 2009 & Wei, 2011). The studies have involved the exploration of information management within the different stages of a construction project; however there have been limited studies that have dealt directly with the inspection stage.

2.2 Information sharing Information sharing can be seen as the process of exchanging information between two or more people (Ford & Staples 2010; Ling, Sandhu & Jain 2008). The information exchanged, may come in the form of tacit and explicit information (Ford & Staples 2010), which includes a mixture of both existing and newly generated information. A majority of the time, information sharing happens whilst employees or experts with specific domain knowledge collaborate together (Lileoere & Hansen 2011). Previous research has indicated the importance of information sharing as it can be leveraged to create competitive advantages for the organisation (Bryant 2005; Porter 1993). However, the information shared amongst individuals does not always happen in an efficient manner and there are many factors that may affect this. An individual’s absorptive capacity as described by Reilly and Sharkey (2010) is an example of such a factor. It involves the individual’s ability to interpret received information, utilise it, and turn it into meaningful action (Lilleoere & Hansen 2011). Without an understanding of the information presented the individual would be rendered less effective in the information sharing process. In addition to the individual’s absorptive capacity, Nonaka (1994) mentions that the ability to share tacit knowledge/information in an organisation requires social interactions between individuals through human activities. It is through these social interactions that individuals form groups in which they are able to pursue new problems and solutions. These interaction groups can be drawn from within an organisation or amongst individuals formed outside the organisation. Nonaka then goes on to suggest that in order for information sharing to be a success, the following factors need to be addressed: trust among individuals in the group, and the existence of a common perspective that each individual has towards the group and towards dialog or individual communications (Nonaka 1994). Holste and Fields (2010) compiled a study into trust in information sharing within an international organization. The research findings seem to indicate that warm relationships and respect are most likely to develop through face‐to‐face interaction among workers. This relationship among workers or trust has affected their willingness to share tacit information. A study by Van den Hooff and Huysman (2009) also found that organizational culture has an effect in organizational information sharing, the more interaction between employees within an organisation, the higher the trust (Van den Hooff & Huysman 2009).

2.3 Information audits According to Dubois (1995) Information Audits can be seen as a useful tool in information management in which the results can be used to identify the contribution made by information to the work of an organisation, and in particular, its importance to decision making. Used properly, an information audit allows the user the ability to: identify resources, services and information flows; verifying the existence of appropriate resources; rationalising resources; controlling resources; exploiting the resulting improvements. (Dubois 1995). The same settlements are echoed by Orna (1999) in her second edition of her book “Practical Information Policies” in which she describes information audits as: ‘A systematic evaluation of information use, resources and flows, with verification by both reference to both people and existing documents, in order to establish the extent to which they are contributing to an organisation’s objectives’ Henczel (2000) built in the previous works in information audits and described them as a process that can effectively determine and define the current information environment by identifying what information is

362

Huan Vo‐Tran et al. required to meet the needs of the organisation/process. The information audit has the ability to establish what information is being supplied and can identify any gaps, inconsistencies and duplications (Henczel 2000). Henczel then also goes on and mentions that, through its use, information audits allow organisations to map its information flows both internally and to its external environment and identify any bottlenecks or inefficiencies (Henczel 2000).

3. The project case study Although from the initial conception to the practical completion of this multi‐million dollar innovative academic building took a little over four years, this particular research project focused on the inspection phase of the construction process, which ran over a period of fourteen weeks. This case study was of interest as a large amount of defects were both generated and handled during the inspection phase. In total, there were over fifteen thousand defects spanning eleven storeys and covering a Gross Floor Area (GFA) of 35,000 square metres was identified and rectified within a short amount of time with the building handed over to the clients over six months ahead of schedule. To add to the complexity and enormity of the task, various stakeholders were involved in completing this critical stage. These included: project managers, engineers, tradespeople, architects, suppliers and the builders themselves. The case study investigated how these various stakeholders, and, in particular architects (in which there were three) and builders (in which there were two) managed the information that was being created during the inspection phase of this construction project. Through the lens of information sharing and information audits, it examined the information management practices of the organisations in which the stakeholders were employed. In addition to the organisation’s practices, stakeholders’ individual information management practices as well as the tools and technologies involved in completing this task we also studied.

4. Research method The research undertaken took on an interpretive framework as the investigators primarily made use of observations of the stakeholders involved in the inspection phase of the single case study of a construction project being studied. Denzin (1994) describes of interpretive research as one where the “fieldworkers can neither make sense or understand what has been learnt until they sit down and write interpretive text, telling the story first to themselves then to significant others and then to the public” (Denzin 1994, p 502). In addition to Denzin, Walsham (1995) defines interpretive research as a research method in which human actors construct our knowledge of reality. Walsham mentions that value‐free data cannot be obtained, since the inquirer makes use of their preconceptions in order to guide the process of inquiry, and furthermore the interactions between the researcher and human subjects of the enquiry change the perceptions of both parties (Walsham 1995). According to Yin (1994), case study research is one in which it ‘investigates a contemporary phenomena within its real‐life context, especially when the boundaries between phenomena and context are not clearly evident’ (Yin 1994, p. 13). Works from both Yin (1994) and Eisenhardt (1989) also indicates that a case study is a good technique to use whilst studying social phenomena within a single setting. It can also assist the researchers in answering the ‘how’ and ‘why’ questions in situations that involves social behaviour through exploratory, descriptive or explanatory research. The research involved the investigators working closely the stakeholders, in particular architects and builders as they inspected a purposely‐built academic building for defects. Over an intensive fourteen‐week period the primary investigator was involved in iterative set of interviews, discussions and on‐site observations. Each iteration involved feedback, re‐discussion of observations and re‐questioning of the participants to check observations and conclusions of the researchers throughout the process. The data collected was interpreted through a hermeneutic lens of feedback, interpretation, observation, interpretation and re‐interpretation, leading to sets of conclusions. Academic rigour was maintained through the triangulation of the data collection process in which data went through a verification stage by third parties via a novel method of experienced investigators ‘observing the observing’. At the conclusion of each observation session, investigators would compare notes to see if what

363

Huan Vo‐Tran et al. had been observed concurred with one another. It also provided a useful check and balance through the data collection stages.

5. Findings Data was collected both onsite and offsite over an intensive fourteen‐week period. During this time, investigators made use of a combination of techniques, these techniques included: document analysis, observations, shadowing, and one‐on‐one interviews with the key stakeholders in the process, namely the builders and architects. The findings highlighted a number of conclusions that have contributed to the knowledge surrounding information use, sharing and storage within the inspection phase of a major construction project. The findings seem to indicate issues relating to process, domain knowledge and issues that could obstruct the efficiency and effectiveness during this phase of the construction process.

5.1 Tacit knowledge and experience in this specific knowledge domain It was observed that during the inspections senior architects and builders would rely more upon their tacit knowledge in their respected domains rather than constantly referring back to other forms of explicit knowledge such as the drawings or any other formal documentation. Subsequent follow‐up interviews with senior architects confirmed this with one stating: ‘When I first started in this business I wouldn’t have known what, you know … you got an idea of what’s acceptable and what’s not acceptable. But you just build up that knowledge over a period of time’. These findings were also crossed referenced against the builder’s observations of the architects involved in the inspection stage. The builders noted there were distinct differences between the architects, especially, in terms of tacit knowledge and experience. When the builder was questioned in relation to the senior architect’s reliance on tacit knowledge she responded by stating: ‘I think has been doing it for a lot longer, he goes a lot more intuition, visual, you know, so he didn’t go I don’t have a plan, he’d be like it doesn’t look right therefore it mustn’t be right’. Whereas, when the same question was asked of that about the less senior and experienced architect the same builder responded by stating: ‘He’d be like, I just don’t have a plan, I don’t know, I need a copy of the drawings’ In order to explore this finding even further, the investigators interviewed the less senior and experienced architect in relation his use and preference of tacit and explicit knowledge. His response was as followed: ‘It’s very important to always carry the drawings and schedules for specification with you and to be able to answer any questions otherwise you cannot answer any questions that come at you’ This statement illustrates the architect’s reliance on explicit knowledge through the use of the documented drawing and schedules. It also revealed the lower levels of specific domain knowledge as compared to that of the more experience senior architect.

5.2 Multiple methods of information recording, coding and storage Although all the participants within the “inspection stage” of the construction project were working towards the same goal, it was observed that the methods of information recording, coding and storage varied significantly between amongst the builders and architects. In addition to the differences between the two professions, it was also noted that there were differences amongst those that worked in same role for the same company. Listed in the tables below is a summary of the different methods in which the participants recorded, coded and stored their information.

364

Huan Vo‐Tran et al. Table 1: Architect A Role

Recording of information Coding of information

Storage of information

Architect A Architect A is one of the directors of the architectural practice involved within the investigation. He has had over 20 years of experience within the industry and has been working on this project from its conception. During the inspections he would be onsite once or twice a week to solve any major design questions. Architect A did not take any notes during the “inspection stage”. He primarily made use of his intimate knowledge of the building to make any design decisions that may arise. The recording of the information and decisions was left up to either the other architects present or the builders. Although Architect A did not record any information whilst onsite, it was observed that, whilst back in the office, Architect A would start a new page in his notepad, where he would write the date on the top right hand corner of the page and provide a title. If there were any references to any other entries then he would also note that down in the current entry. Architect A made use of a notepad to record any notes that may have arisen during the inspections. These notes would then be brought back to the office where they would be used as references in future conversations, meetings, inspections or any other communications.

Table 2: Architect B Role

Recording of information

Coding of information

Storage of information

Architect B Architect B is a very experienced “site architect” who has been in the industry for over 30 years. His primary role in this project was to be onsite and work through all the defects with the builders. He also needed to make sure that the information from the drawings and other architects were relayed correctly back to the builders and that finished product met the initial specifications. Architect B At his previous places of employment Architect B made use of a Dictaphone to record the information during his “defect inspection walkthroughs” The recordings were then sent back to the office for the secretary to transcribe. The transcriptions are then sent back to Architect B to verify brought along with him the next time the inspect the building. With this particular project and architectural firm, Architect B records all his notes on either in a notepad or on printed drawings he brings along with him on inspections Within this particular project Architect B made use of a notepad in which he had a ‘traffic light’ colour‐coding system. Any items that were urgent and required immediate attention would be record in red, any on‐going issues that required further clarification would be written in black and then highlighted in yellow, and finally any items that have been rectified would be written in green. As for the mark‐up of the drawings Architect B made use a highlighter to highlight the area and then add his own comments beside it. Architect B is similar to that of A in which he also carries around his notepad during inspections. However, one of the major differences between Architect A and B is that Architect B is required to head back into the office, scan his notes and upload into an online knowledge management system which is shared amongst the builders and architects.

Table 3: Architect C Role

Recording of information

Coding of information

Storage of information

Architect C Architect C could be classified as a “documentation architect” his primary role was to document the specifications for the building. Architect C was trained in Germany and has been working in the industry for over 10 years. During this project Architect C assumed Architect B’s role for 6 weeks during the inspections” whilst Architect B was overseas on leave. Architect C made use of a combination of methods to record the information during the “defects inspections”. Similar to that of Architect B, Architect C would make use of a notepad and a set of drawings. However, Architect C would also bring along a camera to take photos of items that he was unsure of and bring them back to the office for further analysis. Architect C recorded and coded his notes in an ad‐hoc manner, sometimes he would note the information in his notepad under the date of inspections whilst others would be organised under a description of the room. All notes would be recorded in black pen, as there was no use of colour. As for the coding of information on the drawings, a simple arrow pointing to the area of the defect and a short explanation was provided on the diagram Architect C, although carried a notepad along with him during the inspections, he would then head back to the office and type up meticulous notes and email them back through to the builders. Any photos that were taken would be printed out and attached to the notes as well as being and uploaded to an online knowledge management system.

365

Huan Vo‐Tran et al. Table 4: Builder A Role

Recording of information

Coding of information

Storage of information

Builder A Builder A is an experienced site manager with over 10 years experience within the industry. Builder A specialises in the inspection and handover stages of a construction project. He has defected over 240,000m2 of office space to date and on this particular project leads the building team. Whilst onsite Builder A rarely took notes or recorded any information. Builder A was reliant on Builder B (who was a junior member of the team) to record notes on his behalf. When Builder A did take notes, it was generally through the notes section of his mobile phone. It was observed that Builder A spent the majority of the inspections on the phone organising tradespeople to rectify the defects. Builder A rarely took notes while conducting the inspections. Hence, the coding of information was kept to a bare medium. Notes that he did take were often added as a Builder A new note in his mobile phone, which automatically attributes the time and date in which the note was taken. Builder A tended to make use of verbal forms of communication. Builder A made use of his mobile phone to store his information. Whilst back in the office he would often make use of his emails as a storage and retrieval system for all his files and documents.

Table 5: Builder B Role

Recording of information

Coding of information Storage of information

Builder B Builder B can be classified as a Generation Y graduate with the building firm. She holds a degree in architecture as well as a Masters in building and construction. Her role on this project is to solely work on defecting the building, which involves record all the defects identified by the builders and architects then forward these lists onto the tradespeople for rectification. Builder B was allocated a tablet in which all defects could be recorded. The tablet contained software that had the ability to sync with her workstation back at the site office. However, even though the technology was available she often discarded the use of the tablet and opted to make use of a pen, notepad in which she took details notes. These notes would then be typed up back at the office and emailed to the architects and tradesmen. When Builder B made use of the tablet system the codes were already pre‐defined. However as she prominently made use of the pen and notepad. Her notes were organised in terms of room number, nature of defect and direction (i.e. east wall). Builder B made use of a variety of methods to store the data. Data relating to the inspections was stored in multiple locations including the tablet, her workstation back at site office and on her notepad.

5.3 Information gathering happens in an ‘ad‐hoc’ manner Throughout the course of the inspection phase, investigators observed that the information gathering techniques of the parties involved was invariably not systematic and happened on a ‘needs basis’ or in an ‘ad‐ hoc’ manner. During a follow‐up interview with a senior architect it was noted that: ‘In a perfect world when you do defects the builder’s supposed to say I’ve got to the point where I’ve virtually finished the job, I want you to come and check and verify that it’s all been done correctly’. However, on this project, like many others, the inspection phase occurred in parallel with the fit‐off (finalisation) of the building, which, in turn, made it difficult for the parties involved to work in a systematic manner and perform their primary duties of ‘defecting’ the building. Due to the construction works happing in parallel because the design and build procurement system was used in this project, the builders and architects would often switch from one space to another without adhering to the originally agreed plans. This was mainly due to the constant distractions from construction workers who were working on other aspects of the building. The construction workers often require instant clarification in order to continue on with their tasks. It was these small but constant distractions, which lead to additional inspection times being scheduled around the architects and builders already busy agendas.

366

Huan Vo‐Tran et al. In addition to the distractions, another example of how information was collected in an ‘ad‐hoc’ manner involved the way in which the architects and builders collected and recorded their data. The investigators observed on many occasions the builders switching from recording the defects via the tablet, to the next moment making use of a pen and notepad. Subsequent interviews and further clarification with both parties revealed that this was attributed to the amount of time available to them. Both the architects and builders stated that there was ‘too much time spent wasted recording notes on the tablet versus taken hand written notes’

5.4 The differences in the level of detail collected, stored and disseminated Although the builders and architects were working on the same project and attempting to achieve the same goals, there were a number of differences in the level of detail collected, stored and disseminated. The investigators noted that the level of detail collected by the builders varied from session to session depending on which information management tool was utilised. Whilst making use of the tablet, the data collated was structured, coded and detailed. In contrast, when the pen and notepad was adopted, the level of detail was not as comprehensive, which lead to information sharing problems such as tradespeople receiving unclear information. Not receiving detailed information made it difficult tradespeople to pinpoint the location of the defects for required rectifications. This often slowed the construction process down as clarification needed to be sorted. It was also observed during the inspections the differences between the levels of detail the builders collected in contrast to that of the architects. The architects tended to take less detailed notes and relied on the mark‐ ups on the drawings as well as any photos that were taken. The architects were also reliant on the builders to take detailed notes as the builders were seen to take the lead in this process.

5.5 The numerous amounts of lists being created and the duplication of data Throughout the inspection phase, both the builders and architects were reliant on the creation and examination of lists in order to inspect over fifteen thousand defects. Different lists were created and updated frequently, sometimes on an hourly basis. These lists were often stored in multiple formats and locations. During follow‐up interviews with Builder B, she stated that: ‘You end up generating about four different lists for a single space that you defect and so in this process we invite all these parties to attend but usually it’s only just me and the architect’ Much of the information from these lists was duplicated with minor changes being made for the respected parties, which included: the project manager, the tradespeople, the architects and the builders themselves. To add to the complexity of managing multiple lists, the investigators also noted that individuals were creating their own lists for personal reference. These personalised lists often contained items in which the individual would seek further clarification from either the documentation or supervisors when they returned back to their offices. Listed below is Table 6: ‘The different lists created during the inspection phase’ provides an overview of the different types of list that are created within the inspection phase. Table 6: The different lists created during the inspection phase List Initial defects list

Owner/Creator Builders

Defects list sent to the tradespeople

Builders

Rectified list

Tradespeople

Rectified list – confirmation

Builders

Description The builders created this list as they conducted their preliminary walkthroughs. Its main purpose is to list all the defects that a space may have in order for the tradespeople to rectify before the final inspections with the architects. Individual lists are sent out to the corresponding tradespeople including: electricians, plumbers, plasterers, painters, carpenters, heating ventilation and air‐condition experts outlining the defects that require rectification. It is derived from the initial defects list and the tradespeople are given three days to rectify the defects. The tradespeople send a list of the rectified items back to the builder stating what works has been completed. The builders receive the rectified list back from the tradespeople. Once again they conduct walkthroughs to confirm if the items have been rectified.

367

Huan Vo‐Tran et al. List Defects list sent to the architects prior to the final inspections.

Owner/Creator Builders

Additional defects list (whilst on inspections) Rectified items list

Architects/ Builders

Personalised lists

Architects/ Builders

Design issues list

Architects/ Builders

Builders

Description This list is sent to the architects and the project management manager up to an hour prior to their final inspections. It itemises the defects in which the builders have picked up during their walkthroughs and is used as the basis for the inspections with the architects and any other invited parties. This is an additional list that is created by the builders and architects as they are on their inspections. This list contains any new items that they or the architects might have picked up along the way. Once the defect items are rectified by the tradespeople the builder generates a list for signoff by the architects and project managers. These are lists created by individuals for personal reference. The lists may contain items in which need to be clarified once the builders/architects get back into their respective offices. This is an additional list that has been created by the builders and architects as they inspect the building for defects. The list contains items that are not classified as defects, instead items in which the architects might have missed when designing the building or the built form was not what the architects had in mind. The architects and builders to see if anything could be done then would review this list.

6. Discussion After the examination of the data collected through observations, follow‐up interviews and document analysis over an intensive fourteen week period, the figure below outlines the information sharing problems that the stakeholders faced whilst attempting to conduct the final inspections for a construction project.

Figure 1: Information sharing problems causes during the inspection process The research demonstrated how stakeholders, namely the architects and builders made decisions, managed the information, and undertook their roles within the inspection phase of a construction process. It highlighted how the differences in experiences within the stakeholder’s knowledge domain impacted the level of detail collected, stored and disseminated. The findings from this study also seem to indicate that there was correlation between the level of experience and the use of knowledge. It was observed that the more experience an individual possessed, the less reliant they were on explicit knowledge. Typically, the interaction of differentiated knowledge domains result in poor decision‐making and often leads to ineffective project management and sometimes project failure. In this particular construction project the potential for

368

Huan Vo‐Tran et al. incompleteness was compounded by the multitude of methods and techniques used throughout the inspection process. The use of different personal information management techniques, incompatible use of technologies, and the lack of attention to a project information management system created conditions for significant failure. However, this particular construction project was deemed a success as it was completed six months ahead of schedule. This could be attributed to a number of factors, which include: Experienced stakeholders: It was observed that the senior or more experienced stakeholders (especially the architects) played a vital part in the success within the inspection phase. They were able to make use of their tacit knowledge to solve difficult problems as they arise. Within this project, the senior architect (Architect B) was able to act as an intermediary offering sense making to each of the knowledge domain and communicate information effectively to the other stakeholders. Information flows and information sharing centred on the pivotal role the architect played. He became both the conduit and interpreter of the existing differential knowledge and enable project success. Same goals: The research data indicated that stakeholders involved in the inspection phase made use of the information presented to them differently. Each made sense of the situation through their own experiences, their organisation’s restrictions and polices, the tools available as well as their own personal preferences. However, this came secondary to the overriding desire to fulfil their respected roles and complete the project as outlined. Information management problems within the construction stages have been reported in various literature (Al‐Sudairi, 2007; Al Nahyan, Sohal, Fildes, & Hawas, 2012; Chen, Zeng, & Xie, 2008; Haiyan, Tramel, Hwang et al 2009). However, this study differs from those that have come before it in the way that it deals specially with the inspection stage. It provides new insights through an information‐auditing framework and highlights the sometimes chaotic and serendipitous ways that this phase of construction embodies.

7. Conclusion In conclusion, the findings of this study seem to indicate that there needs to be a greater awareness of effective information management practices within the construction industry and in particular the inspection phase. It also highlights the importance of experienced stakeholders and their possession of tacit knowledge to successfully guide a construction project to completion. Having a greater understanding of these factors may facilitate a decrease in the number of defects and re‐work that often occur in every construction project. If the amount of re‐work in construction projects can be reduced or eliminated, then the construction process effectiveness then can then be improved.

References Al‐Sudairi, A. A. (2007). Evaluating the effect of construction process characteristics to the applicability of lean principles. Construction Innovation: Information, Process, Management, 7(1), 99‐121. Al Nahyan, M. T., Sohal, A. S., Fildes, B. N., & Hawas, Y. E. (2012). Transportation infrastructure development in the UAE: Stakeholder perspectives on management practice. Construction Innovation: Information, Process, Management, 12(4), 492‐514. Arnorsson, H. (2012). Optimizing the Information Flow on the Construction Site. Master, Aalborg University. Benedict, D. I., & David, J. K. (2012). Building Information Modeling and Integrated Project Delivery in the Commercial Construction Industry: A Conceptual Study. Journal of Engineering, Project, and Production Management, 2(1), 23‐36. Blumberg, R., Atre, S. (2003). “The problem with unstructured data” [online]. DM Review http://soquelgroup.com/Articles/dmreview_0203_problem.pdf Bryant, S. E. (2005). The Impact of Peer Mentoring on Organizational Knowledge Creation and Sharing. Group & Organization Management, 30(3), 319‐338. Chen, H., Zeng, L., & Xie, J. (2008). Highway construction management information systems based on workflow. Paper presented at the Proceedings ‐ International Conference on Intelligent Computation Technology and Automation, ICICTA 2008. Denzin, N. (1994). The art and politics of interpretation. In N. Denzin & Y. Lincoln Eds.), Handbook of qualitative research (pp. 500–515). Thousand Oaks, CA: Sage.

369

Huan Vo‐Tran et al. Dubois, C. P. R. (1995). "The information audit: its contribution to decision making." Library Management, Vol 16, No. 7, pp 20 ‐ 24. Eisenhardt, K. M. (1989). Building Theories from Case Study Research. The Academy of Management Review, 14(4), 532‐ 550. Ford, D. P., & Staples, S. (2010). Are full and partial knowledge sharing the same? Journal of Knowledge Management, 14(3), 394‐409. Henczel S (2000), “The Information Audit as a First Step Towards Effective Knowledge Management: An Opportunity for the Special Librarian”, Inspel, Vol. 34, No. 3/4, pp. 210‐226. Haiyan, X., Tramel, J. M., & Wei, S. (2011, 10‐12 June). Building Information Modeling and simulation for the mechanical, electrical, and plumbing systems. Paper presented at the Computer Science and Automation Engineering (CSAE), 2011. Holste, J. S., & Fields, D. (2010). Trust and tacit knowledge sharing and use. Journal of Knowledge Management, 14(1), 128‐ 140. Hwang, B., Thomas, S., Haas, C., & Caldas, C. (2009). Measuring the Impact of Rework on Construction Cost Performance. Journal of Construction Engineering and Management, 135(3), 187‐198. doi: doi:10.1061/(ASCE)0733‐ 9364(2009)135:3(187) Jenkins, T., Glazier, D., Schaper, H. (2004) Enterprise Content Management Technology: What you need to know, Open Text Corporation, Ontario. Lilleoere, A.‐M., & Hansen, E. H. (2011). Knowledge‐sharing enablers and barriers in pharmaceutical research and development. Journal of Knowledge Management, 15(1), 53‐70. Ling, C. W., Sandhu, M. S., & Jain, K. K. (2008). Knowledge sharing in an American multinational company based in Malaysia. Journal of Workplace Learning, 21(2), 125‐142. Nonaka, I. (1994). A Dynamic Theory of Organizational Knowledge Creation. Organisation Science, 5(1), 14‐37. Orna, E. (1999). Practical Information Policies. Hampshire, Aldershot. Peansupap, V., & Walker, D. H. T. (2006). Information communication technology (ICT) implementation constraints: A construction industry perspective. Engineering, Construction and Architectural Management, 13(4), 364‐379. Porter, M. E. (1993). Competitive strategy. Measuring Business Excellence, 1(2), 12‐17. Reilly, M., & Sharkey, S., P. (2010). Dynamic Capabilities, Absorptive Capacity and Knowledge Sharing: A Research Agenda into Explicating the Antecedent Factors Conducive to Subsidiary Bargaining Power. Paper presented at the Academy of International Business (UKI Chapter). Van den Hooff, B., & Huysman, M. (2009). Managing knowledge sharing: Emergent and engineering approaches. Information & Management, 46(1), 1‐8. Walsham, G 1995, 'The Emergence of Interpretivism in IS Research', INFORMATION SYSTEMS RESEARCH, vol. 6, no. 4, pp. 376‐94. Yin, R. K. (1994). Case Study Research: SAGE publication.

370

Work in Progress Papers

371

372

Modeling Practices of Competitive Intelligence in Moroccan Firms Fatima‐zzahra Cheffah and Mostafa Hanoune Faculté des Sciences Ben M'Sik Université Hassan II Mohammedia Casablanca, Casablanca, Morocco cheffah.fz@gmail.com mhanoune@gmail.com Abstract: In an increasingly complex and constantly changing environment, organizations are facing many constraints caused by stiff competition as a result of increasing globalization of markets which affects all sectors. Production economy, as we know it nowadays, evolves into knowledge economy, technologic economy and information economy. Management of information has become therefore an imperative for any organization wishing to ensure its sustainability. Competitive Intelligence is an important tool for information management for organizations. It allows controlling the information about the external environment and therefore adapting his behavior in advance. Competitive Intelligence identifies opportunities and determinants of success, anticipate threats and prevent risks. It allows organizations to secure, to act and to influence their outside world in a perspective of international competitiveness. However, its practices are not unified, they are often imperfect and minor, where the interest to develop these practices to meet the needs of organizations. It is in this context that our empirical study focuses on Competitive Intelligence practices, especially practices adopted by the Moroccan market, where the concept of Competitive Intelligence itself is misunderstood by the various economic actors. The goal of our research is to develop a unified model of Competitive Intelligence practices adapted to SMEs / SMIs in Morocco. We chose to conduct our research by studying first the existing practices and procedures followed by the Moroccan SMEs / SMIs. Then, we will evaluate these practices via a tool that measures their success. Finally, we will propose a unified model adapted to SMEs / SMIs in Morocco. Our research is thus divided into three parts: Survey of Competitive Intelligence practices in Moroccan SMEs / SMIs. Tool for assessing the degree of success of Competitive Intelligence practices. Summary of the survey, evaluation and proposal of a new model of Competitive Intelligence practices. Keywords: competitive intelligence (CI), strategy, information system, modeling

1. Introduction The CI’s environment is perceived uncertain and managers of SMEs / SMIs do not currently have reliable methods and tools to establish an adequate CI system. We have no real information and statistics on the state of competitive intelligence in Moroccan firms. CI is often confused with technology watching, while watching is only a component of the CI. Hence our interest in Moroccan field that we consider blank. Our thesis tries to answer the question: How can we help Moroccan SMEs / SMIs implement and practice the CI, and thus make progress, taking into account their specificities, in order to improve decision‐making and strategy? Our research is a work in progress. We are now advanced enough on the first part of the study of existing CI practices. The second and last parts are currently in theoretical phase.

2. Survey of competitive intelligence practices in Moroccan SMEs / SMIs Taking inspiration of M. Audigier, G. Coulon and P. Rassat, "the first thing that comes to mind to show the usefulness of a new concept is to rely on concrete examples and experiences, and to show their lines of strength", we have focused our research methodology to a field study. The objective of our survey is to identify existing practices within the Competitive Intelligence of Moroccan firms. Each business has its own characteristics, its strengths and weaknesses in the application of Competitive Intelligence. Each tries to get closer to the ideal structure and to develop innovative practices sometimes. This is why we chose to conduct our investigation on two studies: qualitative and quantitative. On the one hand, the qualitative study will treat the diversity of companies; see their practices as adaptation to the industry. This will help us later to derive a general model, unified and adapted to all. On the other hand, the quantitative

373

Fatima‐zzahra Cheffah and Mostafa Hanoune study will allow us to reach a greater number of enterprises, which will be useful to our study because the practices of CI are a minority. The main purpose of the survey is to answer the questions:

Are companies aware of the importance of CI?

Is there a CI cell in companies?

How to practice CI within companies?

2.1 Development of the questionnaire Since our study is both qualitative and quantitative, we have chosen to separate it into two inquiries: Qualitative inquiry: We chose to do interviews with company representatives, and design a questionnaire with open‐ended questions to allow the recipient to answer without constraint and to have full, meaningful answer using the recipient's own knowledge, and thus collect more relevant information about the CI practices. Quantitative inquiry: Based on the experience of the first sample, we extended the study to reach a larger number of companies, by sending online a form with open‐ended, semi‐open and closed questions. Our aim here is to get a maximum of information. The questionnaire is divided into three parts: The first part consists of the identity card of the company (size, industry, type of products, competitors) and the interlocutor’s role. The second part concerns the identification of CI practices within the company: On the one hand, the CI system presentation: its origin, its creation, its objectives, its staff, its budget, and its organizational structure. On the other hand, the description of CI practices: detection of needs, information gathering, analysis and data processing, data dissemination, data storage, tools... And finally, the third part consists in evaluating the effectiveness of CI practices: Results of CI implementation, satisfaction, strengths, and areas for improvement. In short, our aim is to know whether the CI application does meet the company's needs.

3. Tool for assessing the degree of success of competitive intelligence practices We consider that the more successful a CI System is, the more effectively he performs its functions and the more able he is to achieve the objectives that have been assigned to it. Therefore, we believe that the identification of key success factors of a CI System is essential for its evaluation. Indeed, the success of a CI System is determined by the compliance with all the aspects of its characteristics. Compliance with the key success factors of a CI System guarantees a successful CI system, which leads to an effective strategic management and therefore allows the company to keep its long‐term competitiveness. The study of evaluating the success of a CI System has been addressed by several authors. We include J.P. Herring (1996) who conducted a study leading to the proposal of a checklist guide, to evaluate the value of the CI in organizations. F. Jakobiak (1998), meanwhile, proposed a functional test and an assessment of the effectiveness of CI from operation indicators (input stream, output, and information use) and results. We consider these estimates as partial because they do not take into account all key success factors of a CI system. We identify company's indicators to assess by: its culture, its strategy, its organization, its people, its information (relevance, completeness, accuracy, smoothness, accuracy, timeliness, reliability, form, accessibility) and tools (technologies ... etc.). A successful CI System is a system that takes into account the success of all of these indicators, and the cross‐ correlation between them, as shown in the figure below:

374

Fatima‐zzahra Cheffah and Mostafa Hanoune

Culture

Tools & technology

Strategy

Organisation

Information

Staff

Figure 1: The correlation between the company's indicators to assess C. DHAOUI's study entitled "Success criteria for a competitive Intelligence System for a better strategic steering" is close to our expectations. Indeed C. DHAOUI has made a list of key factors that a CI System must satisfy to ensure its success. During her study, she proposes a model for assessing the success of a CI System called MERSIE, which is based on the key success factors of a CI System. The model has more than 70 factors divided into six classes, which are cultural factors, strategic factors, individual factors, informational factors, organizational factors and technological factors. In addition, the MERSIE model aims to propose to the responsible for the CI System a diagnostic tool of achievement of key success factors of his system. We will therefore, first, build on this model a tool to evaluate the success of Moroccan SMEs / SMIs CI practices identified in the survey. We will improve this model, then, to suit the Moroccan market conditions and Moroccan SMEs / SMIs specificities. This model will serve as a tool for evaluating the CI practices quality and their impact on policy and decision process.

4. Summary of the survey, evaluation and proposal of a new model of competitive intelligence practices On the basis of information gathered during the survey and taking into account the evaluation tool, we will establish a unified CI System, adapted to Moroccan companies and which take into account the specificities of Moroccan market. We intend subsequently to experiment it and to improve it. We have already contacted several companies to achieve this aim. Competitive Intelligence has become indispensable for solving strategic problems. We intend to set up a model of CI's practices which includes a set of phases from information requirements definition to useful information dissemination for decision making.

5. Conclusion and prospect The aim of our research is to implement a CI practices model, unified for all Moroccan companies but also suitable to the specificities of each. To achieve our goal, we started by studying existing practices of CI in the Moroccan market via a survey. Then taking into account the different characteristics of a business, we intend to assess these practices, based on an assessment tool for the success of a CI System. Afterwards, we will exploit the results gathered to implement a model of CI practices. And finally, to complete the whole process, we intend to test and refine this model in Moroccan SMEs to measure its effectiveness and impact on decision making.

375

Fatima‐zzahra Cheffah and Mostafa Hanoune

References Catherine De La Robertie, Norbert Lebrument (August 2012), Personal networks management and competitive intelligence in French SMEs, Chedia Dhaoui (2008), Thèse de doctorat : Les critères de réussite d’un système d’intelligence économique pour un meilleur pilotage. Université Nancy 2, France. François Jakobiak (2004), L'intelligence économique : la comprendre, l'implanter, l'utiliser; Éditions d'Organisation. Jan P. Herring (1996), Measuring the Effectiveness of Competitive Intelligence: Assessing & Communicating CI's Value to Your Organization. Society of Competitve Intelligence Professionals. Jérôme Bondu (2001), Benchmarking des pratiques d’intelligence économique, mémoire du master intelligence scientifique, technique et économique de l’ESIEE, France. Kaiquan Xua, Stephen Shaoyi Liaoa, Jiexun Lib, Yuxia Songa (March 2011), Mining comparative opinions from customer reviews for Competitive Intelligence, Elsevier. Marc Audigier, Gérard Coulon, Patrick Rassat (2003), L'intelligence économique : Un nouvel outil de gestion, Maxima. Phani Tej Adidam, Madhumita Banerjee, Paurav Shukla (April 2011), Competitive intelligence and firm's performance in emerging markets: an exploratory study in India, Emerald Group Publishing Limited. Philipe Clerc (2006), L’intelligence économique au Maroc : innover dans le développement, Les Cahiers de l’Orient, N 83, P 119. Rida Chafik, (2007) Mémoire pour l’obtention du Diplôme des Etudes Supérieures Approfondies : Etude Empirique sur les Pratiques des Entreprises Marocaines en Matière d’Intelligence Economique, Université Hassan II, Casablanca, Morocco.

376

Towards an e‐Health Strategy for Vietnam ‐ Issues and Approaches Anna Shillabeer RMIT University, Ho Chi Minh City, Vietnam anna.shillabeer@rmit.edu.vn Abstract: Vietnam has a population of 91.5 million with 70% living in rural areas, many of which are not well serviced by infrastructure of any form that would support e‐health. There is no defined e‐health infrastructure or management strategy in Vietnam and hence it could be argued that therefore there are no issues. However Vietnam does have a number of population health issues that could be somewhat addressed by e‐health initiatives as has occurred in so many other similar geographies. The key problem is how to develop and manage a sustainable e‐health strategy and implementation framework. E‐health is a prominent part of the landscape in a number of similar countries and there is evidence that significant funding is available for e‐health projects, especially in less developed areas such as Vietnam however little progress has been made to date. A number of significant barriers have been identified including a lack of infrastructure to enable e‐health delivery, poor strategy development and co‐ordinated efforts towards defined goals, skills shortages in critical areas including health information management, mobile technologies and security and a lack of cultural sensitivity by current research groups and funding bodies. Breaking down these barriers is critical to any future progress towards an e‐health platform. This paper discusses the Vietnamese health environment, the e‐health barriers, drivers and opportunities in Vietnam and outlines factors that will influence future progress towards a time where e‐health is an integrated part of the healthcare system. Keywords: e‐health infrastructure management strategy issues

1. Background The trend in modern healthcare is towards making health ubiquitous to enable equity in accessibility and delivery of quality healthcare services. By providing sustainable, standardised primary healthcare options across whole populations there are considerable cost and time savings to be realised through the bulk purchasing power, centralised training, single patient record systems, unified patient transfer policies and preventative initiatives. There are also security improvements and error reduction to be achieved through standardisation of patient records, transfer protocols, treatments, nationally co‐ordinated pharmaceutical recalls, pharmaceutical use education and other clinical considerations. This trend is also seen as a driver for e‐health initiatives as it facilitates a national approach to healthcare and provides an environment in which technology can provide greatest value. The benefits of applying e‐health solutions include:

Providing expert input and professional development programs to rural practices as required to enable continuous improvement and professional development,

Providing health information to a broad population to reduce avoidable health issues,

Assessing and monitoring patients as part of case management to support rural professionals, and

Assisting with rural based diagnosis and treatment monitoring to reduce the burden on urban clinics.

The very nature of e‐health initiatives call for a ubiquitous solution and there has been huge success in such technology applications around the world. The healthcare environment in Vietnam however is highly fragmented and presents a number of unique issues that have to date prevented any such solution from being implemented. Given the advancements in technology training and investment in healthcare by the Government and health insurance providers the time is now ripe to start developing such solutions. A current project by the Health Research Group (HRG) within RMIT University Vietnam is investigating the potential for e‐health solutions in Vietnam and is working towards a strategic framework for e‐health implementation in the country. The first milestone for this work was in identifying if there is a feasible mobile technology and transmission foundation upon which to launch e‐health solutions. The ability to transmit data over some communications channel is a critical requirement for most e‐health systems and without this capacity all further work would be futile. This paper presents the current state of understanding in this area and presents early findings that will inform future work.

2. The Vietnamese health environment Vietnam has a long history of traditional medicine. 70% of the population live in rural areas and represent a number of separate cultures each with their own beliefs, some of which are very isolated such as the Hmong

377

Anna Shillabeer people in the northern hills area. Many have little or no access to modern healthcare facilities or clinicians so the traditional ways are retained for those people. Healthcare provision and knowledge across Vietnam is frequently received through community elders and family. This results in fragmented populations and practices. A significant proportion of these people are also very poor (Vietnam’s per capita income was just $1,270 last year (The World Bank 2012)) and cannot afford high quality treatments or insurance policies or most significantly for this project, technology. A comparative overview of the Vietnamese context against other locations is provided in Table 1. Table 1: Comparison of healthcare environments Context Country/urbanisation % Health workers per 1000 people Maternal deaths per 100k births Child mortality per 1000 births Health spend as % of GDP Hospital beds per 1000 people Avg years of education/literacy

Vietnam 30 1.2

China 47 1.4

Aust. 89 3

U.S. 82 2.7

Norway 79 4.1

69*

25* 7.2 2.87 10/94

15.62 4.6 4.06 12/92

4.55 8.5 3.82 21/99

5.98 16.2 3.1 16/99

3.5 9.7 3.52 17/100

Statistics from (CIA 2012) except * from (Vietnam News 2012) Table 1 identifies a number of foci for healthcare improvement in Vietnam. The most obvious is maternal and child health where according to the CIA (CIA 2012) Vietnam is well behind the rest of the world. Viet Nam News on July 11th, 2012 reported that while the statistics had improved dramatically over the past 12 years there was still a significant difference between urban and rural rates of maternal and newborn deaths. This is believed to be due to limited access to healthcare and far lower rates of trained obstetricians in rural areas compared to cities (Vietnam News 2012). This is identified as an area for greater investment by government and presents a viable area of focus for research in this country. The formal healthcare system in Vietnam has a four tiered healthcare model as shown in Figure 1. Patients are usually diagnosed within a commune level healthcare institution and are then referred up through the tiers until they reach a National Hospital which specialises in the treatment of a particular serious or chronic illness such as late stage renal failure, AIDS or cancer. These National Hospitals also provide some palliative care support but this is not a common practice. Some patients may move several times from their rural homeland until they are admitted for treatment. This can take months and cover great distances resulting in dislocation and isolation. Vietnamese culture is very strongly family oriented and hence the potential for dislocation in particular is a significant deterrent to accessing healthcare services even if available.

Figure 1: The 4 tiered Vietnamese healthcare structure

378

Anna Shillabeer Unfortunately even when a patient is diagnosed and admitted for treatment their situation does not improve. Hospitals are very overcrowded and under staffed across the country in both rural and urban locations. There are several reports that suggest patients cannot even assume basic rights as there are not enough beds to allow for one for each patient and doctors see on average 100 patients every day and hence often do not have time to even inform patients of their diagnosis or treatment options. (DeLengocky 2011) Recent work by the HRG with the Ho Chi Minh Cancer Hospital revealed that there were up to 300 new diagnoses per day, an average of greater than two patients per bed, 10,000 outpatients to be managed, long queues waiting in hot buildings or sitting on floors outside or in stairwells and most alarmingly, less than 60% of patients are told their diagnosis. These observations are common around the country. The healthcare system only has at best a tenuous ability to adequately meet the needs of the people it is designed to serve. This presents a very complex and unconstrained environment in which to introduce any new population health initiative.

3. The Vietnamese technology environment Industry Vietnam has approximately 1,000 software outsourcing and IT businesses with most being small‐sized businesses of 10‐30 employees. There were 120,000 employees working in software and IT services in 2011 which was a twenty‐fold increase compared to 2002 (Business Times, 2012). “In developed economies like the U.S. and European nations, IT accounts for some 7 per cent of gross domestic product (GDP), while the figure in Vietnam is less than 2 per cent” (Business Times, 2012b). Although behind the world figures for GDP the annual growth rate was reportedly 25‐35% over the past 10 years for this industry segment (Business Times, 2012). Due to this growth, the demand for IT specialists by outsourcers in particular has far exceeded the supply. As a result, local enterprises are experiencing 20% of their workforce constantly job hopping as they are lured away by promises of working with new technologies, high wages and promotions by international companies (Business Times, 2012). Education To meet the need for skilled professionals the number of universities and colleges offering a computing focused program has grown over the past 10 years. There are currently 277 institutions with a total enrolment of 169,000 students, with 56,000 fresh students enrolling annually (Business Times, 2012). Whilst skills are becoming available, especially in the area of mobile technologies, there is currently no identified opportunity to specialise in health systems development or informatics. This is an obvious issue that needs to be addressed if Vietnam is to take and maintain control of this important public service sector. Technology adoption Research suggests that mobile phones are the most widely adopted form of technology in the world, including developing countries such as Africa where more than half of the population are connected. (Kahn et al, 2010, Heeks, 2011). Data from Vietnam showing that there are 143 mobile phones per 100 people clearly supports this claim (The World Bank 2011). Of those using mobile phones 30% use their phone to access the internet and 35% use it for social networking (Ashwill 2012). In contrast, data for fixed line internet access shows that only 8/100 people across the world are connected and in Vietnam the level is much lower at 4.3/100 (The World Bank 2011). Interestingly there is a significant skew towards younger users with a reported 95% of those aged 15‐24 having internet access of some form (Ashwill 2012). This is an important statistic in the context of the research presented here as it has already been identified that most health information comes from older members of the community and family members, especially parents. The data on technology adoption suggests that these are the people least likely to have access to current, clinically accurate (if the correct sources are accessed) and appropriate information to counsel others with. Table 2 provides an overview of mobile technology adoption in Vietnam and Table 3 shows the reliability of mobile networks as tested in 2010.

379

Anna Shillabeer Table 2: Mobile technology adoption (BMI 2012, VNPT 2011, MIC 2012) Metric

Value

Total domestic connection bandwidth

425538 Mbps

Users per capita

35.58%

Users

31304211

International connection bandwidth

346997 Mbps

Domestic connection bandwidth

460374 Mbps

Total VNIX Network Traffic

134850152 Gbytes

Dot VN active domain names

229815

Allocated IPv4 addresses

15551232 address

Allocated IPv6 addresses

73015820288 /64

Total Broadband Subscribers

4325995

3G Phone Subscribers/100 people

8.5

3G service coverage

30 cities and provinces

Number of 2G/3G cell sites in the south west

7,100.

Table 3: Reliability of Vietnamese mobile networks (VNPT 2011)

There are a number of Government initiatives that aim to strengthen the mobile/Internet technology context in Vietnam by 2015 including (Vietnam Telecomp 2013):

The licensing of 4G services

Ensuring 40‐45% of households have a telephone and Internet access

Providing mobile coverage to 90% of the population

Since becoming officially connected to the global internet network in 1997 the industry has grown significantly to currently support 19 Internet service providers, 1,064 licensed websites and 335 social networking sites. 3G Internet users account for 18% of the population (VNPT 2012).

380

Anna Shillabeer Vietnam is a good fit for the application of mobile E‐health technologies given that it has reasonably good literacy rates as shown in Table 1, the number of University technology graduates in the workplace is growing and there is a drive towards technology adoption and social equity with other countries.

4. Barriers to e‐health Given that early findings suggest that the foundations in terms of skills, technology and connectivity are all either well developed or being developed, the question arises regarding why we are not seeing e‐health implementations in Vietnam. There are a number of barriers beyond the stated three key requirements and these include Vietnamese culture, international interest and investment and the physical geography of Vietnam. Vietnamese culture As discussed in the section on the Vietnamese healthcare environment, there are significant cultural differences to be considered when looking at any new healthcare initiative especially one as innovative as e‐ health. Given the reliance on traditional medicine and the low level of technology adoption by older Vietnamese there is little opportunity for successful implementation of any public health solution using tools and methods that are significantly outside of the usual method of health management. Many Vietnamese have little or no contact with the healthcare system or technology during their lives so expecting a high level of buy‐ in is unrealistic. There is a similar story with clinicians who have little in terms of formal IT training or experience (Mai et.al. 2011). Clinician input is critical to the development of any health system, technology driven included. Informed input is the only way to have any confidence in a successful outcome. Untrained or novice users cannot hope to fully engage or be able to influence technology design decisions. The Vietnamese culture, political system and history have led to a people who do not feel comfortable questioning anyone in a perceived position of power, doctors included. This means that whatever a doctor says is accepted and anything a doctor does not say is deemed irrelevant. This leads to some significant barriers to the use of e‐health solutions in an educational capacity which has worked so well in locations such as Africa. The following quote by a doctor in the cancer hospital is both a reason for concern and a clear demonstration of the lack of empowerment patients accept. “If the patient is diagnosed with cancer, the doctor generally keeps it a secret from the patient. The family is told first, and it is the family’s decision whether or not to tell the patient.” “Patients sometimes undergo treatment for prolonged periods of time without knowing the real cause of their illness.” (Khuu 1999) If the patient never knows what their diagnosis is, how do they know what questions to ask? How do they find information to inform themselves? How do they comply when they have no knowledge of what is being done and why? If they are not made aware of the seriousness then expecting them to comply with an at home solution for example will not succeed. Their first instinct will be to return to their family and care for them in their usual manner, either nurturing or working. The extra burden of monitoring or learning new technologies will not be integrated into their already busy lifestyle. International interest and investment There has been little investment in e‐health in Vietnam. The RICE project is definitely the exception. (Mai et. Al. 2011) Most projects have been concerned with preventing disease, vaccinating children and providing basic necessities such as clean drinking water. Vietnam is a country that is evolving and in some respects is still third world and raising the status of the country has taken precedence over other initiatives. Until the early 1980’s there was little open access to Vietnam and there are still some perception problems from foreigners regarding access, business operations, investment processes and research potential. There are also barriers with the Vietnamese who are a proud people who want to maintain control over their country and also do not understand how to attract overseas investment or work with western attitudes and processes. This presents a critical barrier to progress in many fields, health included.

381

Anna Shillabeer Geography Many large areas of Vietnam are inaccessible due to terrain and have insufficient transport infrastructure. Installation and management of mobile technologies and relay stations is an almost insurmountable problem where the only access is by foot. The key barrier to e‐health implementation in this context is therefore infrastructure enablement. There is also an issue regarding availability of skill sets to design and manage such large programs. Most are managed and financed by external investors used to working in far different geographies. Little local expertise is used or developed thus removing control from the Vietnamese people, increasing expenses and lowering sustainability.

5. E‐health drivers and opportunities The potential for e‐health initiatives in Vietnam is not all bad news. There are major opportunities available and a number of important drivers that will inform any future national e‐health strategy. Drivers

The current healthcare model is clearly failing and some method of providing broad scale health improvement programs, chronic disease management outside of the hospital system where possible and public health monitoring is critically important.

There are not sufficient medical staff and very few with any IT skills. Training is necessary before any progress can be made and there are already proven health education programs available that could be tested.

Internal migration from rural to urban areas means that there are condensed populations that have open access to technology and broadband connectivity. It also means extra pressure on urban health services leading to a pull for innovative solutions given that real estate is limited in terms of an ability to expand but greater service provision is demanded.

There is a defined Government policy direction towards equity in healthcare but the current situation is one of great disparity.

Opportunities

Whilst there is a growing technology maturity there is not yet sufficient capacity to build a ubiquitous health platform. Input in a way that develops skilled capacity in the workforce and empowers Vietnamese people to be able to control their future is key to the implementation of e‐health in any form. Vietnam needs help to realise a higher state of maturity in health management and technologies and this will require significant international expertise on the ground in the medium to long term.

The Government is ready to invest and there is sufficient demand to ensure that action will be taken, but it needs to be guided, cost effective, culturally sensitive and sustainable.

There is a Government awareness of mobile solutions and technology driven initiatives and they are realising the potential for technology solutions to several of the biggest health issues facing Vietnam now and into the future. They now need assistance to design and deliver measurable solutions.

External investment and interest in the region and Vietnam in particular is increasing. There are millions of dollars being invested in technology and health projects. There is however a need for an integrated, unified approach to prevent fragmented solutions and repetition as has been seen in the past. There is also a need to have a defined focus on e‐health strategies and technologies as they offer a feasible solution to a number of the endemic issues presented in this paper.

6. Factors influencing future progress Work by the HRG has identified a number of success factors that will influence and guide future progress in Vietnamese health research. These are the foundation upon which all future work by the HRG will be built.

Develop and promote a centralised, co‐ordinated effort to identify research opportunities rather than follow a fragmented, narrow focussed approach. There should be a research register developed which can be accessed by any interested party to see where the best place to direct research effort may be, and where there may be collaborative opportunities to save duplication.

382

Anna Shillabeer

Implement and manage a structured, strategic approach to solutions development instead of a trial and error, isolated attempt by those who enter the country, test an idea and leave or do not gain the support they require for continued research and hence have their work curtailed

Apply and monitor internal control and external collaboration. The Vietnamese must be an integral part of any project team at all levels and must be empowered to ensure projects are implemented, championed and sustained in a culturally sensitive way. External collaborators are required to ensure rigor and sufficient expertise is applied.

There should be a ubiquitous solution to health research. Too much work only solves part of a problem or benefits a very small subsection of the population and little measurable value is delivered. This creates reluctance for further investment and leaves projects with an unfinished or low impact outcome

There are two areas identified for further e‐health research by the HRG:

Technology for health information and knowledge management including:

Rural medical practitioner professional development and knowledge transfer

Population healthcare awareness and education programs

Potential for social media and gaming as health awareness tools.

Technology infrastructure and social development including:

Implementation and assessment of technologies to provide measured improvements in healthcare outcomes for Vietnam

Healthcare monitoring and service planning through data analytics

Development of a long term, sustainable strategic direction for e‐health in Vietnam

Health policy development for rural Vietnam.

7. Conclusion Vietnam is a country experiencing a period of economic and social transition between a time of prolonged unrest and stagnation and its embrace of the technology age where its people are learning to live with new found freedom, peace and comparative wealth. It is a country offering many opportunities both to its own people and overseas interests. It has a complex cultural context that informs everything that occurs, including healthcare. There is a trend is towards greater investment in Vietnamese Health initiatives and technology but little expertise in actually delivering innovative, technology driven solutions. Collaboration between internal and external stakeholders is seen as a critical factor for the future of e‐health as it is not a well understood concept in Vietnam. This complex environment presents both barriers and opportunities and significant investment has been made in both research and infrastructure projects although there has been little directed towards e‐health research. A major limitation to progress is that there are no national solutions, no consensus on direction and little local expertise, strength or strategy. It would appear from the early findings presented here, that the building blocks required for successful e‐health implementation are available. It is now time to connect the pieces together and plan to build and test broad scale solutions.

References Mark Ashwill. (2012). Internet Penetration, Social Media & Student Recruitment. Available: http://markashwill.com/2012/12/. Last accessed 7th February 2013. Business Monitor International (BMI). (2012). Vietnam Telecommunications Report Q2 2012. Available: http://www.researchandmarkets.com/reports/2137416/vietnam_telecommunications_report_q2_2012. Last accessed 5th February 2013. Business Times. (2012). IT industry posts annual growth rate of 25‐35%. Available: http://businesstimes.com.vn/it‐industry‐ posts‐annual‐growth‐rate‐of‐25‐35/. Last accessed 4th February 2013. Business Times. (2012b). IT attractive to foreign investors. Available: http://businesstimes.com.vn/it‐attractive‐to‐foreign‐ investors/. Last accessed 4th February 2013. Central Intelligence Agency (CIA). (2012). East & Southeast Asia :: Vietnam . Available: https://www.cia.gov/library/publications/the‐world‐factbook/geos/vm.html. Last accessed 7th February 2013. Dr. DeLengocky. (2011). Deplorable Conditions of Vietnamese Hospitals. Available: http://eyedrd.org/2011/08/deplorable‐ conditions‐of‐vietnamese‐hospitals.html. Last accessed 4th February 2013.

383

Anna Shillabeer Richard Heeks. (2011). Mobile services in poor countries Not just talk. Available: http://www.economist.com/node/18008202. Last accessed 5th February 2013. Duke Trinh Khuu. (1999). Vietnamese Health Care. Available: http://med.stanford.edu/medicalreview/smrvietnam.pdf. Last accessed 7th February 2013. James G. Kahn, Joshua S. Yang and James S. Kahn, 'Mobile' Health Needs And Opportunities In Developing Countries. Health Affairs, 29, no.2 (2010):252‐258 Bui Thi Thanh Mai, Tran Hung Minh, Dang Thi Hong Linh, Nguyen Tuan Quang,. (2011). THUOCNET: EHEALTH INFORMATION PORTAL. Available: http://healthmarketinnovations.org/sites/healthmarketinnovations.org/files/FINAL_ThuocNET02011_0.pdf. Last accessed 7th February 2013. Ministry of Information and Communications (MIC). (2012). Report on Internet statistics of Vietnam. Available: http://www.vnnic.vn/en/stat/report‐internet‐statistics?lang=en. Last accessed 5th February 2013. The Vietnam Post & Telecommunications Group (VNPT). (2011). Vietnam Telecommunications Report . Available: http://www.vnpt.com.vn/Portals/0/users/host/052011/05/EBCVT_web.pdf. Last accessed 4th February 2013. The World Bank. (2012). Vietnam. Available: http://data.worldbank.org/country/vietnam. Last accessed 4th February 2013. Vietnam News. (2012). Maternal mortality shows rapid decline. Available: http://vietnamnews.vn/social‐ issues/227287/maternal‐mortality‐shows‐rapid‐decline.html. Last accessed 5th February 2013. Vietnam Telecomp. (2013). Market Updates. Available: http://vietnam‐telecomp.com/VNC13/Main/lang‐ eng/ShowUpdate.aspx. Last accessed 7th February 2013.

384

Proposed Mobile Technology Acceptance Model of the Information Services in a Library Context Sununthar Vonjaturapat and Singha Chaveesuk Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand sununthar@buu.ac.th singha@it.kmitl.ac.th Abstract: According to the Education Center for Applied Research findings, it was found that students expect to access their institutes’ services from their handheld devices. While libraries have become an integral part of educational institutions, research findings indicate that library services have only 14.8% usage. This paper aims to investigate the factors that predict a patron’s intentions to accept mobile technology and to develop the proposed model that identify the potential factors on the mobile technology acceptance of information services in a library context. The work extends the extending in Unified Theory of Acceptance and Use of Technology model (UTAUT2), which is an extension of the Technology Acceptance Model (TAM) and the UTATU model and add technology characteristics and task characteristics variables. The research includes both exploratory and quantitative techniques. There are two stages in the preliminary study: 1) Reviewing and investigating the potential factors from previous research that is relevant to this work.2) Using semi‐structured interviews data by applying check list methods to collect data from potential patrons and patrons of the mobile technology acceptance in the library service context. The contribution of the model is a design guideline for the mobile technology function that consistent of a service function in the library context. In addition, this model will provide a useful tool to help understand and predict patron’s intentions to accept mobile technology of information service in a library context. Other outcomes of this study are the development and enhancement of library services which also provide more choices of the library service process. Keywords: unified theory of acceptance and use of technology model (UTAUT), UTAUT2, task technology fit model, library services, information services, mobile technology

1. Introduction Information Communication Technology ICTs include communication devices or application covering radio, television cellular phones, computer and network hardware and software, wireless technology and internet as well as various services and application associated with them, such as videoconferencing and distance learning (UNDP 2005). ICTs are usually used in a specific context, such as ICTs in health care, education, or libraries. (Techtarget.com 2011) These contexts use many different tools, depending on the capability of existing communication devices. Gartner (2012) also estimates that mobile technology application development will outnumber personal computer development by a ratio of 4 to 1, smartphone and tablet usage will increase to 90% by 2015. The growth of students own mobile devices that they use to access the internet on a regular basis is increasing (Smith 2009). Students can also use their mobile technology to listen to music, watch videos, play games and connect to social networking sites. Even though mobile technology brings many advantages, there are differences from library context where patrons have to interact with IT for long period of time for gaining in utilitarian benefits, efficiency, effectiveness and self‐improvement. Research finding indicates that the digital information service are not widely used all the time due to the existing habits of users in reading through traditional methods or they are uncomfortable due to reading from a screen (Yonghua, Xinhao, Xingbo 2011; Kosturki and Skornia 2010). Therefore, success from a technical perspective alone does not mean acceptance of the system. The key question of the service providers is how to transfer the potential of mobile technology into optimization services. Thus, this research is important to investigate the effecting factors of the intention to use mobile technology with information services in a library context. TAM (Davis 1989) has been widely used to study the acceptance in variety of information system (IS) contexts (Chin, Marcolin, and Newsted 2003; Riemenschneider, Harrison, and Mykytyn 2003). TAM focuses on user perception, attitudes and beliefs. This is the most common concept with the UTAUT2 (Venketesh, Thong, and Xu 2012) as an extension of the TAM and UTATU model (Venketesh et al. 2003) However, the TAM model ignores social impact, gender, age and other factors that can influence user acceptance of information technology (IT). The UTAUT model does not take technology characteristics and task characteristics into

385

Sununthar Vonjaturapat, and Singha Chaveesuk consideration (Zhang, Huang, and Chen 2010). Therefore, an improved IT acceptance model could require technology characteristics and task characteristics to be included in the UTAUT2 model.

2. Literature review 2.1 Mobile technology in library context Mobile technologies include laptops, netbooks, e‐readers, tablets, mobile phones, smart phone MP3/ MP4 players and internet capable handheld devices (Seeler 2011; Youghue, Xinhao, and Xingbo 2011). In addition, wireless networks, such as 3G, are providing the network infrastructure for users. However, tablets and smart phones are widely used. Tablets are also available for browsing online, downloading, streaming and lending digital content from libraries. This means that searching a database, downloading articles, and seeking catalogs can be conducted via tablets. In addition, the benefits of mobile technology can support users to access documents, including e‐books, audio‐visual objects and website. A pilot program in various libraries has established that tablets provide convenience in accessing the library’s resources and tablets are a new innovation that attracts students. Specifically, the design of tablets with lighter weights, slimmer shapes and multi‐touch screen interfaces can eliminate the need for peripheral devices (Kosturki and Skornia 2010). This means that patrons will likely use mobile technology to surf the internet, listen to music, read documents, and so on, are easier.

2.2 TAM and UTAUT2 TAM (Davis 1989) has been broadly used to examine the adoption of advanced mobile services (Alain, 2012). The UTAUT2 (Venketesh, Thong, and Xu 2012) was original based on the TAM and UTAUT model. The UTAUT is a well ‐ established model that broadly predicts and explains human behavior in various domains (Venketesh et al. 2003). The original TAM focuses on technological aspects, based on 3 basis relationships effecting behavioral intension (BI): (1) perceived usefulness (PU) (2) perceived ease of use (PEOU) and (3) PU leading to BI. These relationships include the actual use of technology directly influenced by an individual’s behavioral intention to use a system, and in turn, the behavioral intention to use it determined by PU and attitude toward the technology. However, other streams of research may increase our understanding, such as findings on the UTAUT2. The UTAUT2 has become the latest and most powerful theory with 73 percent of the variance in intention 2 (adjusted R ), to predict and explain information systems usage intention (Venketesh, Thong, and Xu 2012). This theoretical model comprises seven main determinants of intention and use: (1) performance expectancy (PE), (2) effort expectancy (EE), (3) social influence (SI), (4) facilitating conditions (FC), (5) hedonic motivation (HM), (6) price value (PV) and (7) habit (HA) as well as three moderating variables: (1) gender, (2) age and (3) experience (4) voluntariness. All of these models make important and unique contributions to technology acceptance.

3. Research model With regard to the original relationship in the UTAUT2, we believe that the direct affecting factors of PE, EE, SI, FC, HM, PV and HA will be hold in the mobile technology acceptance of information services in a library context. Patrons will use IT if it is useful (i.e, relevance to their needs) and easy to use. We also retain the moderation effects of age, gender, experience, as controlled variables in the model. However, the acceptance mobile technology of information services in a library context is voluntary, the moderating variables, voluntariness, were not applicable and so were not included in the model. Figure1 depicts the theoretical model on this research.. Task characteristics/assignments are defined as the action carried out by individuals in turning input to output (Goodhue and Thompson 1995) and may influence the patron’s perception of performance expectancy of using mobile technology of information services in a library context. Task characteristics/assignment in retrieving information from a library via mobile technology may lead to lower performance expectancy when patrons perceive that the systems as not useful and unable to satisfy or accomplish their job (Rahman, Jamaludin, and Mahmud 2011).

386

Sununthar Vonjaturapat, and Singha Chaveesuk Technology characteristics/form‐factors (including cost) are viewed as tools used by individuals in carrying out their task (Goodhue and Thompson 1995) and may influence the patrons’ perception of the effort expectancy of using mobile technology in a library context. Technology characteristic such as lighter weights, slimmer shapes and multi‐touch screen interfaces may lead to feelings of ease and correspond with the information services and therefore lead to increased use of the system (Zhang, Huang, and Chen 2010). More specifically for patrons, a number of items that consists of performance expectancy and facilitation conditions were identified from the literature. For performance expectancy, utilitarian outcomes (Thong et al. 2011), precisely located information (Zhang, Huang, and Chen 2010), and comfort in retrieving information (Rahman, Jamaludin, and Mahmud 2011) were proposed as operationalized items that may influence the patrons’ perception of the effectiveness of mobile technology in a library context. For facilitation conditions, accessibility, design user screens, and convenience in delivering data (Mckinney, Yoon, and Zahedi. 2003) were identified as operationalized items that might influence the patrons’ use of mobile technology in a library context. One further construct that may influence the patrons’ use of mobile technology of information services in library context, the library service type, was identified. The library service type corresponding with patrons’ devices will help more patrons in less time to receive, respond and communicate with the library efficiency through existing patron’s devices. This may influence the patron’s perception of behavioral intention to use a system. Therefore, this factor will be added to examine whether the library service type will influence the use of mobile technology.

Figure 1: Theoretical model used in this research

4. Conclusions The contribution of the model is a design guideline for the mobile technology function that consistent of a service function in the library context. In addition, this model will provide a useful tool to help understand and predict a patron’s intentions to accept mobile technology of information service in a library context. Therefore, understanding the ways in which these students relate to and use their own mobile technology is an important goal for both service providers and researchers. If patrons feel that they are supported in the use of own mobile technology, this tool is an easy and reliable alternative, they will accept it and hence the university and application vender or service providers will benefit.

387

Sununthar Vonjaturapat, and Singha Chaveesuk

References Alain, Y.L.C., Felix, T.S.C. and Keng, B. O. (2012) “Predicting consumer decisions to adopt mobile commerce: Cross country empirical examination between China and Malaysia”, Decision support system, Vol.53, pp 34‐43. Chin, W.W., Marcolin, B.L. and Newsted, P.R. (2003) “A partial least squares latent variable modeling approach for measuring interaction effects: results from a Monte Carlo simulation study and an electronic mail emotion/adoption study”, Information Systems Research, Vol.14, No.2, pp 189‐217. Davis, F.D. (1989) “Perceived usefulness, perceived ease of use, and user acceptance of information technologies”, MIS Quarterly, Vol.13, No.3, pp 319‐340. Gartner (2012) Gartner: Top Predictions for IT Organizations 2012 and Beyond [Online], www.gartner.com/it/page.jsp?id=13281132010. Goodhue, D.L. and Thompson, R.L.(1995) “Task‐technology fit and individual performance”, MIS Quarterly, Vol.19, No.2, pp 213‐236. Kosturki, K. and Skornia, F. (2010) “Handheld Libraries 101Using Mobile Technologies in the Academic Library”, [online], Information today inc,www.infotoday.com/cilmag/ jul11/Kosturski_Skornia.shtml. Mckinney, V., Yoon, K. and Zahedi, F. M. (2003) “The measurement of web‐customer satisfaction: An expectation and disconfirmation approach”, Information system research, Vol.13, No. 3, pp 296‐315. Rahman, A.L., Jamaludin, A. and Mahmud, Z. (2011) “Intention to use digital library based on modified UTUTA Model: Perspectives of Malaysian postgraduate Students”, World academy of science, Engineering and technology, Vol.75, pp 116‐122. Riemenschneider, C.K., Harrison, D.A. and Mykytyn, P.P. (2003) “Understanding IT adoption decisions in small business: integrating current theories”, Information Management, Vol. 40, No.4, pp 269–287. Seeler, K. (2011) “Moving Forward: Opportunities for Mobile Technology in the Academic Library”, [online], University of Maryland, www.slideshare.net/KatieSeeler/mobile‐technology‐and‐the‐academic‐library. Smith, S.D.(2009) The ECAR study of understanding student and information technology. Vol.6 pp 1‐132. [online], www.educause.edu/ecar. Techtarget.com (2011) “Definitions of ICTs”, [Online], www.techtarget.com. Thong, J.Y.L., Venkatesh,V., Xu, X., Hong, S.J. and Tam, K.Y. (2011) “Consumer acceptance for personal information and communication technology services”, IEEE transaction on engineering management, Vol.58, No.4, pp 613‐625. UNDP (2005) “Regional Human Development Report. Promoting ICT for Human Development in Asia”, Realizing the Millennium Development Goals, Elsevier. Venketesh, V., Morris, M.G., Davis, G.B. and Davis, F.D. (2003) “User acceptance of information technology: toward a unified view”, MIS Quarterly, Vol. 27, No. 3, pp 245‐478. Venketesh, V., Thong J.Y.L. and Xu, X. (2012) “Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology”, Forthcoming MIS Quarterly, Vol. 36, No.1, pp157‐178. Youghue, F.,Xinhao, Z. and Xingbo, H.(2011) “Realization of Electronic paper‐based mobile digital library”,IEEE, pp 6719‐ 6722. Zhang,J., Huang, J. and Chen,J. (2010) “Empirical research on use acceptance of mobile search”, Tsinghua science and technology, Vol.15, No. 2, pp 235‐245.

388