Applying Evidence Theory to Aggregate Feedbacks in 360 Degree Feedback Model

Document Type : Original Article

Authors

1 PhD. student, Department of Industrial Engineering, Faculty of Engineering, Payam Noor university, Tehran, Iran

2 Associate Professor, School of Industrial Engineering, College of Engineering, University of Tehran, Tehran, Iran

Abstract

In 360 degree feedback model several groups of raters with different perspectives provide assessments typically in Likert scale format. Due to the qualitative and subjective nature of the assessments, these data are highly uncertain and divergent. This problem causes that aggregation of the data in 360 degrre model, with traditional average based methods be inaccurate and invalid. Regarding the importance of the aggregation problem in 360 degrre feedback process and lack of a suitable solution to cope with it, in this paper a new model based on Evidence Theory has been proposed to aggregate its assessments and model the uncertainty contained in it. In the proposed model, first the evaluation data obtained from each rating group has been aggregated and its uncertainty has been modeled using a basic belief assignment. In the next step, the evidences obtained from rating groups have been aggregated using evidence combination rules.
To design the model, various methods to transform feedbacks to basic belief assignents, various evidence combination rules and different criteria in extracting the final results of the model, have been investigated through a simulation study. In the simulation process the model's performance in 27 different states, defined based on different combination of model's parameters, and using ten thousand simulated records, has been examined. Results reveal that the proposed model, compared to the traditional average based aggregation, significantly reduces error and increase the accuracy of the results in 360 degree feedback model. In addition, other benefits of the proposed model have been explained in the text.

Keywords

References

[1]      Valle M, Bozeman DP. Interrater agreement on employees’job performance: review and directions. Psychol Rep. 2002; 90(3):975-985.
[2]      Espinilla M, de Andrés R, Martínez FJ, Martínez L. A 360-degree performance appraisal model dealing with heterogeneous information and dependent criteria. Inf Sci (Ny). 2013; 222:459-471. doi:http://dx.doi.org/10.1016/ j.ins.2012.08.015.
[3]      Golman R, Bhatia S. Performance evaluation inflation and compression. Accounting, Organ Soc. 2012; 37(8):534-543.
[4]      Brutus SS. Words versus numbers: A theoretical exploration of giving and receiving narrative comments in performance appraisal. Hum Resour Manag Rev. 2010; 20(2):144-157. doi:10.1016/j.hrmr.2009.06.003.
[5]      Gibbs MJ, Merchant KA, der Stede WA, Vargus ME. The benefits of evaluating performance subjectively. Perform Improv. 2005; 44(5):26.
[6]      Markham SE, Smith JW, Markham IS, et al. A new approach to analyzing the Achilles’ heel of multisource feedback programs: Can we really trust ratings of leaders at the group level of analysis? Leadersh Q. 2014; 25(6):1120-1142. doi:10.1016/j.leaqua.2014.10.003.
[7]      de Andrés R, García-Lapresta JL, González-Pachón J. Performance appraisal based on distance function methods. Eur J Oper Res. 2010; 207(3):1599-1607. doi:10.1016/j.ejor.2010.06.012.
[8]      Xu D-L. An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis. Ann Oper Res. 2012; 195(1):163-187. doi:10.1007/s10479-011-0945-9.
[9]      Sentz K, Ferson S. Combination of Evidence in Dempster-Shafer Theory. Vol 4015. Sandia National Laboratories Albuquerque; 2002.
[10]   Tang D, Wong TC, Chin KS, Kwong CK. Evaluation of user satisfaction using evidential reasoning-based methodology. Neurocomputing. 2014; 142:86-94. doi:http://dx.doi.org/10.1016/j.neucom.2014.01.055.
[11]   Deng Y., Sadiq R., Jiang W., Tesfamariam S. Risk analysis in a linguistic environment: A fuzzy evidential reasoning-based approach. Expert Syst Appl. 2011;38(12):15438-15446. doi:10.1016/j.eswa.2011.06.018.
[12]   lotfi demirchi M., Mirfakhradini SH., naser sadrabadi A. Strategy Evaluation Based on D Numbers and BSC Framework (Case Study: Electrical Industry). Mod Res Decis Mak. 2017;2(3):77-97. http://journal.saim.ir/article_28676.html.
[13]   Moon C, Lee J, Lim S. A performance appraisal and promotion ranking system based on fuzzy logic: An implementation case in military organizations. Appl Soft Comput. 2010; 10(2):512-519. doi:http://dx.doi.org/10.1016/j.asoc. 2009.08.035.
[14]   Sepehrirad R, Azar A, Sadeghi A. Developing a Hybrid Mathematical Model for 360-Degree Performance Appraisal: A Case Study. Procedia - Soc Behav Sci. 2012;62(June 2016):844-848. doi:10.1016/j.sbspro.2012.09.142.
[15]   Avazpour R, Ebrahimi E, Fathi MR. A 360 Degree Feedback Model for Performance Appraisal Based on Fuzzy AHP and TOPSIS. Int J Econ Manag Soc Sci. 2013;2(11):969-976.
[16]   Ozkan C, Keskin GA, Omurca SI. A variant perspective to performance appraisal system: fuzzy c–means algorithm. Int J Ind Eng. 2014;21(3).
[17]   Nahid titkanlu H, Keramati A. Potential capabilities of Evidence Theory in development of 360 degree appraisal model. In: 2nd Management Tools and Techniques Conference. ; 2015. https://www.civilica.com/Paper-MANAGTOOLS02-MANAGTOOLS02_120.html.
[18]   Hamidi N, Akbari Shemiran R, Shirdel GH, Taleshi B. Proposing a non-additive, fuzzy-hybrid, multi-criteria decision making model for prioritizing and assessing suppliers(Azin Tane Company). Manag Res Iran. 2012;16(3)(59-81).
[19]   Arbaiy N, Suradi Z. Staff performance appraisal using fuzzy evaluation. Artif Intell Innov 2007 from Theory to Appl. 2007:195-203. doi:10.1007/978-0-387-74161-1_21.
[20]   Min-peng X, Xiao-hu Z, Xin D. Modeling of Engineering R&D Staff Performance Appraisal Model Based on Fuzzy Comprehensive Evaluation. Syst Eng Procedia. 2012; 4:236-242. doi:http://dx.doi.org/10.1016/ j.sepro.2011.11.071.
[21]   Afsar A, , Rahmat Houshdar Mahjoub BM. Customer Credit Clustering for Presenting Appropriate Facilities. Manag Res Iran. 2013;17(4):1-24. http://mri.modares.ac.ir/article_2645.html.
[22]   Leung Y, Ji N-N, Ma J-H. An integrated information fusion approach based on the theory of evidence and group decision-making. Inf Fusion. 2013;14(4):410-422. doi:10.1016/j.inffus.2012.08.002.
[23]   Cobb BR, Shenoy PP. On the plausibility transformation method for translating belief function models to probability models. Int J Approx Reason. 2006;41(3):314-330. doi:10.1016/j.ijar.2005.06.008.
[24]   Smets P. Decision making in the TBM: The necessity of the pignistic transformation. Int J Approx Reason. 2005;38(2):133-147. doi:10.1016/j.ijar.2004.05.003.
[25]   Florea MC, Jousselme AL, Bossé É, Grenier D. Robust combination rules for evidence theory. Inf Fusion. 2009; 10(2):183-197. doi:10.1016/j.inffus. 2008.08.007.
[26]   Ha-Duong M. Hierarchical fusion of expert opinions in the Transferable Belief Model, application to climate sensitivity. Int J Approx Reason. 2008; 49(3):555-574. doi:http://dx.doi.org/10.1016/j.ijar.2008.05.003.
[27]   Lefèvre E, Elouedi Z. How to preserve the conflict as an alarm in the combination of belief functions? Decis Support Syst. 2013;56(1):326-333. doi:10.1016/j.dss.2013.06.012.

Files

Share

How to cite

Statistics