Evaluation of agent-based modeling and simulation tools based on ISO 25010

Document Type : Original Article

Authors

1 PhD Student, Modeling Software Engineering Research Group, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran

2 Associate Professor, Model Driven Software Engineering Research Group, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran

3 Assistant Professor, Modeling Software Engineering Research Group, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran

Abstract

Due to the key role and importance of using agent-based modeling and simulation tools in various applications, such as urban management and planning, social networks, financial markets, traffic flows, and crisis management, it is worthwhile to evaluate and quantitatively compare these tools based on a well-defined standard framework. So far, several studies have reviewed and compared agent-based modeling and simulation tools from different aspects. However, none of the studies are quantitatively performed based on a well-known standard framework. To fill this research gap, in this paper, five of the most widely used agent-based modeling and simulation tools including AnyLogic, NetLogo, Repast, GAMA, and MASON were compared and evaluated based on the well-known ISO 25010 standard, regarding the quality in use, by Goal-Question-Metric (GQM) method. The evaluation results show that AnyLogic tools are of better quality than others, and this is due to the attractiveness of the tool from the user's point of view and less effort to develop an agent-based application compared to other tools.

Keywords


[1]    A. Borshchev, The big book of simulation modeling: multimethod modeling with AnyLogic 6. AnyLogic North America, 2013.
[2]    D. J. Barnes and D. Chu, “Agent Based Modeling,” in Guide to Simulation and Modeling for Biosciences, Springer London, 2015, pp. 15–78.
[3]    M. Wooldridgey and P. Ciancarini, “Agent-oriented software engineering: The state of the art,” in International workshop on agent-oriented software engineering, 2000, pp. 1–28.
[4]    E. Bonabeau, “Agent-based modeling: Methods and techniques for simulating human systems,” Proceedings of the national academy of sciences, vol. 99, no. suppl 3, pp. 7280–7287, 2002.
[5]    M. Niazi and A. Hussain, “Agent-based computing from multi-agent systems to agent-based models: a visual survey,” Scientometrics, vol. 89, no. 2, pp. 479–499, 2011.
[6]    E. Abolfathi, A. Toloie Eshlaghy, and M. reza Hamidizadeh, “An Operating Anatomy for Agent-Based Modeling Stand on the Categorization of Research Done In the Humanities: The Diffusion of Innovation in Iran,” Modern Research in Decision Making, vol. 3, no. 2, pp. 1–25, 2018.
[7]    M. Khani,  abbas saghaei, and K. Heidarzadeh Hanzaee, “Modeling New Product Launch Strategies within agent-based simulation,” Modern Research in Decision Making, vol. 4, no. 4, pp. 176–196, 2019.
[8]    F. jahanseir khararoudi, A. Azar, and T. Karimi, “Decision- making on dimensions of ambidexterity using an agent-based modeling approach,” Modern Research in Decision Making, vol. 5, no. 4, pp. 1–18, 2020.
[9]    D. Helbing, “Agent Based Modeling,” in Social Self-Organization, D. Helbing, Ed. Springer Berlin Heidelberg, 2012, pp. 25–70.
[10] C. Macal and M. North, “Introductory Tutorial: Agent-Based Modeling And Simulation,” in Proceedings of the 2014 Winter Simulation Conference, 2014, pp. 3650–3661.
[11] C. W. Loh, S. Nissom, W. S. Cheah, and N. Jali, “Agent oriented methodology for malaria transmission modelling and simulation,” in Proceedings of the 6th International Conference of Computing & Informatics, 2017, pp. 160–166.
[12] E. Shakshuki and Y. Jun, “Multi-agent development toolkits: An evaluation,” in International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 2004, pp. 209–218.
[13] C. Nikolai and G. Madey, “Tools of the trade: A survey of various agent based modeling platforms,” Journal of Artificial Societies and Social Simulation, vol. 12, no. 2, p. 2, 2009.
[14] A. Singh, D. Juneja, and A. K. Sharma, “Agent development toolkits,” International Journal of Advancements in Technology, vol. 2, no. 1, pp. 158–164, 2011.
[15] K. Kravari and N. Bassiliades, “A Survey of Agent Platforms,” Journal of Artificial Societies and Social Simulation, vol. 18, no. 1, p. 11, 2015.
[16] S. Abar, G. K. Theodoropoulos, P. Lemarinier, and G. M. P. O’Hare, “Agent Based Modelling and Simulation tools: A review of the state-of-art software,” Computer Science Review, vol. 24, pp. 13–33, 2017.
[17] I. S. O. Iso, “Iec25010: 2011 systems and software engineering--systems and software quality requirements and evaluation (square)--system and software quality models,” International Organization for Standardization, vol. 34, p. 2910, 2011.
[18] V. R. B. G. Caldiera and H. D. Rombach, “The goal question metric approach,” Encyclopedia of software engineering, pp. 528–532, 1994.
[19] R. Allan, “Survey of Agent Based Modelling and Simulation Tools,” 2010.
[20] L. Braubach, A. Pokahr, and W. Lamersdorf, “A universal criteria catalog for evaluation of heterogeneous agent development artifacts,” From Agent Theory to Agent Implementation (AT2AI-6), pp. 19–28, 2008.
[21] Wikipedia, “AnyLogic.” [Online]. Available: https://en.wikipedia.org/wiki/AnyLogic. [Accessed: 09-Mar-2020].
[22] “NetLogo.” [Online]. Available: http://ccl.northwestern.edu/netlogo/. [Accessed: 10-Mar-2020].
[23] “The Repast Suite Documentation.” [Online]. Available: https://repast.github.io/index.html. [Accessed: 10-Mar-2020].
[24] “GAMA-Platform.” [Online]. Available: https://gama-platform.github.io/wiki/Home. [Accessed: 09-Mar-2020].
[25] “MASON Multiagent Simulation Toolkit.” [Online]. Available: https://cs.gmu.edu/~eclab/projects/mason/. [Accessed: 10-Mar-2020].
[26] A. Hussain, N. L. Hashim, N. Nordin, and H. M. Tahir, “A metric-based evaluation model for applications on mobile phones,” Journal of Information and Communication Technology, vol. 12, pp. 55–71, 2013.
[27] F. McCown, “Schelling’s Model of Segregation.” [Online]. Available: http://nifty.stanford.edu/2014/mccown-schelling-model-segregation/. [Accessed: 09-Mar-2020].