TY  - JOUR
AU  - Radovanović, Sandro
AU  - Bohanec, Marko
AU  - Delibašić, Boris
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2447
AB  - Creating decision models for risk assessment of ski injuries is a challenging task. Ski injuries are rare events, but they carry a high cost, that is, can cause working or movement disabilities. Usually, ski risk assessment is performed on small-scale, case-controlled studies where the effect of a single factor is evaluated. Recently, data mining and machine learning algorithms are being employed for ski risk assessment and injury prediction. However, these models do not generally satisfy the need for interpretation of the decision model, do not provide explanations for the predictions, and in general do not ensure the completeness and consistency of decision rules. To make data mining and machine learning models useful, one needs to implement the aforementioned properties. Decision support systems are expected to have these properties; however, the process of building such decision support systems is still tedious: it has to consider human biases, assumptions, and subjective values, as well as focus on the decision problem being solved. We propose a method for extraction of decision models from data at hand. Our method DIDEX, Data Induced DEcision eXpert, builds models that have desirable properties for inclusion in decision support systems. The proposed method is used to build a decision model for ski injury prediction based on data from Mt. Kopaonik ski resort, Serbia. The results show that DIDEX generates up to a five times simpler model compared to the existing domain expert DEX models while having a 6% better predictive accuracy. Additionally, its predictive accuracy is comparable to similar machine learning algorithms, such as decision tree classifiers, random forest, and logistic regression.
PB  - Wiley, Hoboken
T2  - International Transactions in Operational Research
T1  - Extracting decision models for ski injury prediction from data
DO  - 10.1111/itor.13246
UR  - conv_2820
ER  - 

@article{
author = "Radovanović, Sandro and Bohanec, Marko and Delibašić, Boris",
year = "2023",
abstract = "Creating decision models for risk assessment of ski injuries is a challenging task. Ski injuries are rare events, but they carry a high cost, that is, can cause working or movement disabilities. Usually, ski risk assessment is performed on small-scale, case-controlled studies where the effect of a single factor is evaluated. Recently, data mining and machine learning algorithms are being employed for ski risk assessment and injury prediction. However, these models do not generally satisfy the need for interpretation of the decision model, do not provide explanations for the predictions, and in general do not ensure the completeness and consistency of decision rules. To make data mining and machine learning models useful, one needs to implement the aforementioned properties. Decision support systems are expected to have these properties; however, the process of building such decision support systems is still tedious: it has to consider human biases, assumptions, and subjective values, as well as focus on the decision problem being solved. We propose a method for extraction of decision models from data at hand. Our method DIDEX, Data Induced DEcision eXpert, builds models that have desirable properties for inclusion in decision support systems. The proposed method is used to build a decision model for ski injury prediction based on data from Mt. Kopaonik ski resort, Serbia. The results show that DIDEX generates up to a five times simpler model compared to the existing domain expert DEX models while having a 6% better predictive accuracy. Additionally, its predictive accuracy is comparable to similar machine learning algorithms, such as decision tree classifiers, random forest, and logistic regression.",
publisher = "Wiley, Hoboken",
journal = "International Transactions in Operational Research",
title = "Extracting decision models for ski injury prediction from data",
doi = "10.1111/itor.13246",
url = "conv_2820"
}

Radovanović, S., Bohanec, M.,& Delibašić, B.. (2023). Extracting decision models for ski injury prediction from data. in International Transactions in Operational Research
Wiley, Hoboken..
https://doi.org/10.1111/itor.13246
conv_2820

Radovanović S, Bohanec M, Delibašić B. Extracting decision models for ski injury prediction from data. in International Transactions in Operational Research. 2023;.
doi:10.1111/itor.13246
conv_2820 .

Radovanović, Sandro, Bohanec, Marko, Delibašić, Boris, "Extracting decision models for ski injury prediction from data" in International Transactions in Operational Research (2023),
https://doi.org/10.1111/itor.13246 .,
conv_2820 .

TY  - JOUR
AU  - Milosavljević, Miloš
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2528
AB  - An effective, efficient, fair, and trusted tax administration is a top priority for every country in the world; however, tax administration faces many issues, such as corruption, tax avoidance, or lack of flexibility. Some countries perform better in this process, and this paper aims to identify the main drivers of tax administration performance. We analyzed 35 European tax administrations by 12 performance dimensions in 2 consecutive years (2018-2019) and created a comprehensive performance measurement indicator using a data-driven neutral-aggregation approach. The findings indicate that (a) digitalization of tax administrations is the most influential driver of the overall tax administration performance, (b) Nordic countries and Switzerland can serve as role models for tax administration performance, and
PB  - Elsevier, Amsterdam
T2  - Economic Modelling
T1  - What drives the performance of tax administrations? Evidence from selected european countries
VL  - 121
DO  - 10.1016/j.econmod.2023.106217
UR  - conv_2896
ER  - 

@article{
author = "Milosavljević, Miloš and Radovanović, Sandro and Delibašić, Boris",
year = "2023",
abstract = "An effective, efficient, fair, and trusted tax administration is a top priority for every country in the world; however, tax administration faces many issues, such as corruption, tax avoidance, or lack of flexibility. Some countries perform better in this process, and this paper aims to identify the main drivers of tax administration performance. We analyzed 35 European tax administrations by 12 performance dimensions in 2 consecutive years (2018-2019) and created a comprehensive performance measurement indicator using a data-driven neutral-aggregation approach. The findings indicate that (a) digitalization of tax administrations is the most influential driver of the overall tax administration performance, (b) Nordic countries and Switzerland can serve as role models for tax administration performance, and",
publisher = "Elsevier, Amsterdam",
journal = "Economic Modelling",
title = "What drives the performance of tax administrations? Evidence from selected european countries",
volume = "121",
doi = "10.1016/j.econmod.2023.106217",
url = "conv_2896"
}

Milosavljević, M., Radovanović, S.,& Delibašić, B.. (2023). What drives the performance of tax administrations? Evidence from selected european countries. in Economic Modelling
Elsevier, Amsterdam., 121.
https://doi.org/10.1016/j.econmod.2023.106217
conv_2896

Milosavljević M, Radovanović S, Delibašić B. What drives the performance of tax administrations? Evidence from selected european countries. in Economic Modelling. 2023;121.
doi:10.1016/j.econmod.2023.106217
conv_2896 .

Milosavljević, Miloš, Radovanović, Sandro, Delibašić, Boris, "What drives the performance of tax administrations? Evidence from selected european countries" in Economic Modelling, 121 (2023),
https://doi.org/10.1016/j.econmod.2023.106217 .,
conv_2896 .

TY  - JOUR
AU  - Petrović, Andrija
AU  - Radovanović, Sandro
AU  - Nikolić, Mladen
AU  - Delibašić, Boris
AU  - Jovanović, M.
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2526
AB  - Currently, one of the biggest challenges in modern traffic engineering is related to traffic state estimation (TSE). Although many machine learning and domain models can be used for TSE, they do not consider the sparsity and spatial dependence of traffic state variables. In this paper, we propose a hybrid soft computing model of two Gaussian conditional random field (GCRF) models for the inference of traffic speed, which is a relevant variable for TSE and travel information systems. The proposed model can infer the traffic state variables in large-scale networks whose nodes are geographically dispersed. Moreover, by combining a Gaussian conditional random field binary classification model (GCRFBC), which classifies traffic regimes as free-flow or potentially congested, and a regression GCRF model for the prediction of traffic speed in potentially congested traffic regimes, the model addresses two specifics of the problem: sparsity in traffic data, and the fact that observations are not independent. The proposed model was tested on two large-scale real-world networks in Serbia, namely an arterial E70-E75 335 km long highway stretch and the major ski resort Kopaonik with 55 km of ski slopes. In addition, the proposed model showed better prediction performance than several other unstructured and structured models.
PB  - Elsevier Ltd
T2  - Applied Soft Computing
T1  - Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks
VL  - 133
DO  - 10.1016/j.asoc.2022.109893
UR  - conv_3750
ER  - 

@article{
author = "Petrović, Andrija and Radovanović, Sandro and Nikolić, Mladen and Delibašić, Boris and Jovanović, M.",
year = "2023",
abstract = "Currently, one of the biggest challenges in modern traffic engineering is related to traffic state estimation (TSE). Although many machine learning and domain models can be used for TSE, they do not consider the sparsity and spatial dependence of traffic state variables. In this paper, we propose a hybrid soft computing model of two Gaussian conditional random field (GCRF) models for the inference of traffic speed, which is a relevant variable for TSE and travel information systems. The proposed model can infer the traffic state variables in large-scale networks whose nodes are geographically dispersed. Moreover, by combining a Gaussian conditional random field binary classification model (GCRFBC), which classifies traffic regimes as free-flow or potentially congested, and a regression GCRF model for the prediction of traffic speed in potentially congested traffic regimes, the model addresses two specifics of the problem: sparsity in traffic data, and the fact that observations are not independent. The proposed model was tested on two large-scale real-world networks in Serbia, namely an arterial E70-E75 335 km long highway stretch and the major ski resort Kopaonik with 55 km of ski slopes. In addition, the proposed model showed better prediction performance than several other unstructured and structured models.",
publisher = "Elsevier Ltd",
journal = "Applied Soft Computing",
title = "Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks",
volume = "133",
doi = "10.1016/j.asoc.2022.109893",
url = "conv_3750"
}

Petrović, A., Radovanović, S., Nikolić, M., Delibašić, B.,& Jovanović, M.. (2023). Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks. in Applied Soft Computing
Elsevier Ltd., 133.
https://doi.org/10.1016/j.asoc.2022.109893
conv_3750

Petrović A, Radovanović S, Nikolić M, Delibašić B, Jovanović M. Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks. in Applied Soft Computing. 2023;133.
doi:10.1016/j.asoc.2022.109893
conv_3750 .

Petrović, Andrija, Radovanović, Sandro, Nikolić, Mladen, Delibašić, Boris, Jovanović, M., "Structured prediction of sparse dependent variables for traffic state estimation in large-scale networks" in Applied Soft Computing, 133 (2023),
https://doi.org/10.1016/j.asoc.2022.109893 .,
conv_3750 .

TY  - JOUR
AU  - Dodevska, Zorica
AU  - Petrović, Andrija
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2488
AB  - Ranking is a prerequisite for making decisions, and therefore it is a very responsible and frequently applied activity. This study considers fairness issues in a multi-criteria decision-making (MCDM) method called VIKOR (in Serbian language-VIsekriterijumska optimizacija i KOmpromisno Resenje, which means Multiple Criteria Optimization and Compromise Solution). The method is specific because of its original property to search for the first-ranked compromise solutions based on the parameter v. The VIKOR method was modified in this paper to rank all the alternatives and find compromise solutions for each rank. Then, the obtained ranks were used to satisfy fairness constraints (i.e., the desired level of disparate impact) by criteria weights optimization. We built three types of mathematical models depending on decision makers' (DMs') preferences regarding the definition of the compromise parameter v. Metaheuristic optimization algorithms were explored in order to minimize the differences in VIKOR ranking prior to and after optimization. The proposed postprocessing reranking approach ensures fair ranking (i.e., the ranking without discrimination). The conducted experiments involve three real-life datasets of different sizes, well-known in the literature. The comparisons of the results with popular fair ranking algorithms include a comparative examination of several rank-based metrics intended to measure accuracy and fairness that indicate a high-quality competence of the suggested approach. The most significant contributions include developing automated and adaptive optimization procedures with the possibility of further adjustments following DMs' preferences and matching fairness metrics with traditional MCDM goals in a comprehensive full VIKOR ranking.
PB  - Springer, Dordrecht
T2  - Autonomous Agents and Multi-Agent Systems
T1  - Changing criteria weights to achieve fair VIKOR ranking: a postprocessing reranking approach
IS  - 1
VL  - 37
DO  - 10.1007/s10458-022-09591-5
UR  - conv_2809
ER  - 

@article{
author = "Dodevska, Zorica and Petrović, Andrija and Radovanović, Sandro and Delibašić, Boris",
year = "2023",
abstract = "Ranking is a prerequisite for making decisions, and therefore it is a very responsible and frequently applied activity. This study considers fairness issues in a multi-criteria decision-making (MCDM) method called VIKOR (in Serbian language-VIsekriterijumska optimizacija i KOmpromisno Resenje, which means Multiple Criteria Optimization and Compromise Solution). The method is specific because of its original property to search for the first-ranked compromise solutions based on the parameter v. The VIKOR method was modified in this paper to rank all the alternatives and find compromise solutions for each rank. Then, the obtained ranks were used to satisfy fairness constraints (i.e., the desired level of disparate impact) by criteria weights optimization. We built three types of mathematical models depending on decision makers' (DMs') preferences regarding the definition of the compromise parameter v. Metaheuristic optimization algorithms were explored in order to minimize the differences in VIKOR ranking prior to and after optimization. The proposed postprocessing reranking approach ensures fair ranking (i.e., the ranking without discrimination). The conducted experiments involve three real-life datasets of different sizes, well-known in the literature. The comparisons of the results with popular fair ranking algorithms include a comparative examination of several rank-based metrics intended to measure accuracy and fairness that indicate a high-quality competence of the suggested approach. The most significant contributions include developing automated and adaptive optimization procedures with the possibility of further adjustments following DMs' preferences and matching fairness metrics with traditional MCDM goals in a comprehensive full VIKOR ranking.",
publisher = "Springer, Dordrecht",
journal = "Autonomous Agents and Multi-Agent Systems",
title = "Changing criteria weights to achieve fair VIKOR ranking: a postprocessing reranking approach",
number = "1",
volume = "37",
doi = "10.1007/s10458-022-09591-5",
url = "conv_2809"
}

Dodevska, Z., Petrović, A., Radovanović, S.,& Delibašić, B.. (2023). Changing criteria weights to achieve fair VIKOR ranking: a postprocessing reranking approach. in Autonomous Agents and Multi-Agent Systems
Springer, Dordrecht., 37(1).
https://doi.org/10.1007/s10458-022-09591-5
conv_2809

Dodevska Z, Petrović A, Radovanović S, Delibašić B. Changing criteria weights to achieve fair VIKOR ranking: a postprocessing reranking approach. in Autonomous Agents and Multi-Agent Systems. 2023;37(1).
doi:10.1007/s10458-022-09591-5
conv_2809 .

Dodevska, Zorica, Petrović, Andrija, Radovanović, Sandro, Delibašić, Boris, "Changing criteria weights to achieve fair VIKOR ranking: a postprocessing reranking approach" in Autonomous Agents and Multi-Agent Systems, 37, no. 1 (2023),
https://doi.org/10.1007/s10458-022-09591-5 .,
conv_2809 .

TY  - JOUR
AU  - Radovanović, Sandro
AU  - Petrović, Andrija
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2465
AB  - Recently, the usage of machine learning algorithms is subject to discussion from a legal and ethical point of view. Unwanted discrimination regarding gender or race of a prediction model can lead to legal consequences. Therefore, during predictive model learning, one needs to be aware of possible bias and adjust the model to be fair. However, in bank marketing applications, one customer can receive multiple offers instead of just one. Because of their correlation between, a multi-label classification approach is the most suitable one. This paper proposes a fair classifier chain machine learning model for multi-label classification. Our algorithm solves the multi-label classification problem in an efficient manner, and it is suitable for real-life application employment. The proposed approach allows for controlling fairness constraints during the process of machine learning. It is based on the logistic regression model, thus enabling high efficiency and understandability. We apply our model to a real-life model from bank marketing campaign response prediction. The obtained results are promising. More specifically, our model achieves high fairness measures having an increase from 7% to 17%. However, fairness has a price of a decrease in predictive performance, up to 9% of AUC. To the best of our knowledge, this is the first algorithm that introduce fairness constraints in multi-label classification problems.
PB  - Wiley, Hoboken
T2  - International Transactions in Operational Research
T1  - A fair classifier chain for multi-label bank marketing strategy classification
EP  - 1339
IS  - 3
SP  - 1320
VL  - 30
DO  - 10.1111/itor.13059
UR  - conv_2555
ER  - 

@article{
author = "Radovanović, Sandro and Petrović, Andrija and Delibašić, Boris and Suknović, Milija",
year = "2023",
abstract = "Recently, the usage of machine learning algorithms is subject to discussion from a legal and ethical point of view. Unwanted discrimination regarding gender or race of a prediction model can lead to legal consequences. Therefore, during predictive model learning, one needs to be aware of possible bias and adjust the model to be fair. However, in bank marketing applications, one customer can receive multiple offers instead of just one. Because of their correlation between, a multi-label classification approach is the most suitable one. This paper proposes a fair classifier chain machine learning model for multi-label classification. Our algorithm solves the multi-label classification problem in an efficient manner, and it is suitable for real-life application employment. The proposed approach allows for controlling fairness constraints during the process of machine learning. It is based on the logistic regression model, thus enabling high efficiency and understandability. We apply our model to a real-life model from bank marketing campaign response prediction. The obtained results are promising. More specifically, our model achieves high fairness measures having an increase from 7% to 17%. However, fairness has a price of a decrease in predictive performance, up to 9% of AUC. To the best of our knowledge, this is the first algorithm that introduce fairness constraints in multi-label classification problems.",
publisher = "Wiley, Hoboken",
journal = "International Transactions in Operational Research",
title = "A fair classifier chain for multi-label bank marketing strategy classification",
pages = "1339-1320",
number = "3",
volume = "30",
doi = "10.1111/itor.13059",
url = "conv_2555"
}

Radovanović, S., Petrović, A., Delibašić, B.,& Suknović, M.. (2023). A fair classifier chain for multi-label bank marketing strategy classification. in International Transactions in Operational Research
Wiley, Hoboken., 30(3), 1320-1339.
https://doi.org/10.1111/itor.13059
conv_2555

Radovanović S, Petrović A, Delibašić B, Suknović M. A fair classifier chain for multi-label bank marketing strategy classification. in International Transactions in Operational Research. 2023;30(3):1320-1339.
doi:10.1111/itor.13059
conv_2555 .

Radovanović, Sandro, Petrović, Andrija, Delibašić, Boris, Suknović, Milija, "A fair classifier chain for multi-label bank marketing strategy classification" in International Transactions in Operational Research, 30, no. 3 (2023):1320-1339,
https://doi.org/10.1111/itor.13059 .,
conv_2555 .

TY  - JOUR
AU  - Dodevska, Zorica
AU  - Radovanović, Sandro
AU  - Petrović, Andrija
AU  - Delibašić, Boris
PY  - 2023
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2458
AB  - We propose introducing fairness constraints to one of the most famous multi-criteria decision-making methods, the analytic hierarchy process (AHP). We offer a solution that guarantees consistency while respecting legally binding fairness constraints in AHP pairwise comparison matrices. Through a synthetic experiment, we generate the comparison matrices of different sizes and ranges/levels of the initial parameters (i.e., consistency ratio and disparate impact). We optimize disparate impact for various combinations of these initial parameters and observed matrix sizes while respecting an acceptable level of consistency and minimizing deviations of pairwise comparison matrices (or their upper triangles) before and after the optimization. We use a metaheuristic genetic algorithm to set the dually motivating problem and operate a discrete optimization procedure (in connection with Saaty's 9-point scale). The results confirm the initial hypothesis (with 99.5% validity concerning 2800 optimization runs) that achieving fair ranking while respecting consistency in AHP pairwise comparison matrices (when comparing alternatives regarding given criterium) is possible, thus meeting two challenging goals simultaneously. This research contributes to the initiatives directed toward unbiased decision-making, either automated or algorithm-assisted (which is the case covered by this research).
PB  - MDPI, Basel
T2  - Mathematics
T1  - When Fairness Meets Consistency in AHP Pairwise Comparisons
IS  - 3
VL  - 11
DO  - 10.3390/math11030604
UR  - conv_2841
ER  - 

@article{
author = "Dodevska, Zorica and Radovanović, Sandro and Petrović, Andrija and Delibašić, Boris",
year = "2023",
abstract = "We propose introducing fairness constraints to one of the most famous multi-criteria decision-making methods, the analytic hierarchy process (AHP). We offer a solution that guarantees consistency while respecting legally binding fairness constraints in AHP pairwise comparison matrices. Through a synthetic experiment, we generate the comparison matrices of different sizes and ranges/levels of the initial parameters (i.e., consistency ratio and disparate impact). We optimize disparate impact for various combinations of these initial parameters and observed matrix sizes while respecting an acceptable level of consistency and minimizing deviations of pairwise comparison matrices (or their upper triangles) before and after the optimization. We use a metaheuristic genetic algorithm to set the dually motivating problem and operate a discrete optimization procedure (in connection with Saaty's 9-point scale). The results confirm the initial hypothesis (with 99.5% validity concerning 2800 optimization runs) that achieving fair ranking while respecting consistency in AHP pairwise comparison matrices (when comparing alternatives regarding given criterium) is possible, thus meeting two challenging goals simultaneously. This research contributes to the initiatives directed toward unbiased decision-making, either automated or algorithm-assisted (which is the case covered by this research).",
publisher = "MDPI, Basel",
journal = "Mathematics",
title = "When Fairness Meets Consistency in AHP Pairwise Comparisons",
number = "3",
volume = "11",
doi = "10.3390/math11030604",
url = "conv_2841"
}

Dodevska, Z., Radovanović, S., Petrović, A.,& Delibašić, B.. (2023). When Fairness Meets Consistency in AHP Pairwise Comparisons. in Mathematics
MDPI, Basel., 11(3).
https://doi.org/10.3390/math11030604
conv_2841

Dodevska Z, Radovanović S, Petrović A, Delibašić B. When Fairness Meets Consistency in AHP Pairwise Comparisons. in Mathematics. 2023;11(3).
doi:10.3390/math11030604
conv_2841 .

Dodevska, Zorica, Radovanović, Sandro, Petrović, Andrija, Delibašić, Boris, "When Fairness Meets Consistency in AHP Pairwise Comparisons" in Mathematics, 11, no. 3 (2023),
https://doi.org/10.3390/math11030604 .,
conv_2841 .

TY  - JOUR
AU  - Sikimić, Vlasta
AU  - Radovanović, Sandro
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2271
AB  - As more objections have been raised against grant peer-review for being costly and time-consuming, the legitimate question arises whether machine learning algorithms could help assess the epistemic efficiency of the proposed projects. As a case study, we investigated whether project efficiency in high energy physics (HEP) can be algorithmically predicted based on the data from the proposal. To analyze the potential of algorithmic prediction in HEP, we conducted a study on data about the structure (project duration, team number, and team size) and outcomes (citations per paper) of HEP experiments with the goal of predicting their efficiency. In the first step, we assessed the project efficiency using Data Envelopment Analysis (DEA) of 67 experiments conducted in the HEP laboratory Fermilab. In the second step, we employed predictive algorithms to detect which team structures maximize the epistemic performance of an expert group. For this purpose, we used the efficiency scores obtained by DEA and applied predictive algorithms - lasso and ridge linear regression, neural network, and gradient boosted trees - on them. The results of the predictive analyses show moderately high accuracy (mean absolute error equal to 0.123), indicating that they can be beneficial as one of the steps in grant review. Still, their applicability in practice should be approached with caution. Some of the limitations of the algorithmic approach are the unreliability of citation patterns, unobservable variables that influence scientific success, and the potential predictability of the model.
PB  - Springer, Dordrecht
T2  - European Journal for Philosophy of Science
T1  - Machine learning in scientific grant review: algorithmically predicting project efficiency in high energy physics
IS  - 3
VL  - 12
DO  - 10.1007/s13194-022-00478-6
UR  - conv_2718
ER  - 

@article{
author = "Sikimić, Vlasta and Radovanović, Sandro",
year = "2022",
abstract = "As more objections have been raised against grant peer-review for being costly and time-consuming, the legitimate question arises whether machine learning algorithms could help assess the epistemic efficiency of the proposed projects. As a case study, we investigated whether project efficiency in high energy physics (HEP) can be algorithmically predicted based on the data from the proposal. To analyze the potential of algorithmic prediction in HEP, we conducted a study on data about the structure (project duration, team number, and team size) and outcomes (citations per paper) of HEP experiments with the goal of predicting their efficiency. In the first step, we assessed the project efficiency using Data Envelopment Analysis (DEA) of 67 experiments conducted in the HEP laboratory Fermilab. In the second step, we employed predictive algorithms to detect which team structures maximize the epistemic performance of an expert group. For this purpose, we used the efficiency scores obtained by DEA and applied predictive algorithms - lasso and ridge linear regression, neural network, and gradient boosted trees - on them. The results of the predictive analyses show moderately high accuracy (mean absolute error equal to 0.123), indicating that they can be beneficial as one of the steps in grant review. Still, their applicability in practice should be approached with caution. Some of the limitations of the algorithmic approach are the unreliability of citation patterns, unobservable variables that influence scientific success, and the potential predictability of the model.",
publisher = "Springer, Dordrecht",
journal = "European Journal for Philosophy of Science",
title = "Machine learning in scientific grant review: algorithmically predicting project efficiency in high energy physics",
number = "3",
volume = "12",
doi = "10.1007/s13194-022-00478-6",
url = "conv_2718"
}

Sikimić, V.,& Radovanović, S.. (2022). Machine learning in scientific grant review: algorithmically predicting project efficiency in high energy physics. in European Journal for Philosophy of Science
Springer, Dordrecht., 12(3).
https://doi.org/10.1007/s13194-022-00478-6
conv_2718

Sikimić V, Radovanović S. Machine learning in scientific grant review: algorithmically predicting project efficiency in high energy physics. in European Journal for Philosophy of Science. 2022;12(3).
doi:10.1007/s13194-022-00478-6
conv_2718 .

Sikimić, Vlasta, Radovanović, Sandro, "Machine learning in scientific grant review: algorithmically predicting project efficiency in high energy physics" in European Journal for Philosophy of Science, 12, no. 3 (2022),
https://doi.org/10.1007/s13194-022-00478-6 .,
conv_2718 .

TY  - JOUR
AU  - Delibašić, Boris
AU  - Radovanović, Sandro
AU  - Jovanović, Miloš
AU  - Obradović, Zoran
AU  - Suknović, Milija
AU  - Lojić, Ranko
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2305
AB  - This paper addresses a novel topic in ski injury research - how a ski group size indicates the risk of ski injury. There is evidence in research literature that people ski in groups. However, the relationship between group size and the risk of injury has remained unexplored. Based on ski lift entrance data, we use the density-based clustering algorithm OPTICS to identify groups of skiers and discuss the advantages of using this algorithm. We show that the ski group size can be used to improve the identification of skiers who experience ski injury. The results of the identification of ski groups at Mt. Kopaonik Ski Resort in Serbia show that skiing alone is most susceptible to ski injury, while skiing in couples or in bigger groups reduces the risk of injury by 46%. In addition, it is confirmed that ski injuries are an early failure event phenomena. Based on the CHAID decision tree analysis, spending a small amount of time at the ski resort and skiing alone are associated with the: 6 times greater ski injury risk than the average ski injury risk.
PB  - Sage Publications Ltd, London
T2  - Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology
T1  - A study on ski groups size and their relationship to the risk of injury
DO  - 10.1177/17543371221118193
UR  - conv_2748
ER  - 

@article{
author = "Delibašić, Boris and Radovanović, Sandro and Jovanović, Miloš and Obradović, Zoran and Suknović, Milija and Lojić, Ranko",
year = "2022",
abstract = "This paper addresses a novel topic in ski injury research - how a ski group size indicates the risk of ski injury. There is evidence in research literature that people ski in groups. However, the relationship between group size and the risk of injury has remained unexplored. Based on ski lift entrance data, we use the density-based clustering algorithm OPTICS to identify groups of skiers and discuss the advantages of using this algorithm. We show that the ski group size can be used to improve the identification of skiers who experience ski injury. The results of the identification of ski groups at Mt. Kopaonik Ski Resort in Serbia show that skiing alone is most susceptible to ski injury, while skiing in couples or in bigger groups reduces the risk of injury by 46%. In addition, it is confirmed that ski injuries are an early failure event phenomena. Based on the CHAID decision tree analysis, spending a small amount of time at the ski resort and skiing alone are associated with the: 6 times greater ski injury risk than the average ski injury risk.",
publisher = "Sage Publications Ltd, London",
journal = "Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology",
title = "A study on ski groups size and their relationship to the risk of injury",
doi = "10.1177/17543371221118193",
url = "conv_2748"
}

Delibašić, B., Radovanović, S., Jovanović, M., Obradović, Z., Suknović, M.,& Lojić, R.. (2022). A study on ski groups size and their relationship to the risk of injury. in Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology
Sage Publications Ltd, London..
https://doi.org/10.1177/17543371221118193
conv_2748

Delibašić B, Radovanović S, Jovanović M, Obradović Z, Suknović M, Lojić R. A study on ski groups size and their relationship to the risk of injury. in Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology. 2022;.
doi:10.1177/17543371221118193
conv_2748 .

Delibašić, Boris, Radovanović, Sandro, Jovanović, Miloš, Obradović, Zoran, Suknović, Milija, Lojić, Ranko, "A study on ski groups size and their relationship to the risk of injury" in Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology (2022),
https://doi.org/10.1177/17543371221118193 .,
conv_2748 .

TY  - JOUR
AU  - Radovanović, Sandro
AU  - Savić, Gordana
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2295
AB  - Achieving fairness in algorithmic decision-making tools is an issue constantly gaining in need and popularity. Today, unfair decisions made by such tools can even be subject to legal consequences. We propose a new constraint that integrates fairness into data envelopment analysis (DEA). This allows the calculation of relative efficiency scores of decision-making units (DMUs) with fairness included. The proposed fairness constraint restricts disparate impact to occur in efficiency scores, and enables the creation of a single data envelopment analysis for both privileged and unprivileged groups of DMUs simultaneously. We show that the proposed method FairDEA produces an interpretable model that was tested on a synthetic dataset and two real-world examples, namely the ranking between hybrid and conventional car designs, and the Latin American and Caribbean economies. We provide the interpretation of the FairDEA method by comparing it to the basic DEA and the balanced fairness and efficiency method (BFE DEA). Along with calculating the disparate impact of the model, we performed a Wilcoxon rank-sum test to inspect for fairness in rankings. The results show that the FairDEA method achieves similar efficiency scores as other methods, but without disparate impact. Statistical analysis indicates that the differences in ranking between the groups are not statistically different, which means that the ranking is fair. This method contributes both to the development of data envelopment analysis, and the inclusion of fairness in efficiency analysis.
PB  - Elsevier, Amsterdam
T2  - European Journal of Operational Research
T1  - FairDEA-Removing disparate impact from efficiency scores
EP  - 1098
IS  - 3
SP  - 1088
VL  - 301
DO  - 10.1016/j.ejor.2021.12.001
UR  - conv_2678
ER  - 

@article{
author = "Radovanović, Sandro and Savić, Gordana and Delibašić, Boris and Suknović, Milija",
year = "2022",
abstract = "Achieving fairness in algorithmic decision-making tools is an issue constantly gaining in need and popularity. Today, unfair decisions made by such tools can even be subject to legal consequences. We propose a new constraint that integrates fairness into data envelopment analysis (DEA). This allows the calculation of relative efficiency scores of decision-making units (DMUs) with fairness included. The proposed fairness constraint restricts disparate impact to occur in efficiency scores, and enables the creation of a single data envelopment analysis for both privileged and unprivileged groups of DMUs simultaneously. We show that the proposed method FairDEA produces an interpretable model that was tested on a synthetic dataset and two real-world examples, namely the ranking between hybrid and conventional car designs, and the Latin American and Caribbean economies. We provide the interpretation of the FairDEA method by comparing it to the basic DEA and the balanced fairness and efficiency method (BFE DEA). Along with calculating the disparate impact of the model, we performed a Wilcoxon rank-sum test to inspect for fairness in rankings. The results show that the FairDEA method achieves similar efficiency scores as other methods, but without disparate impact. Statistical analysis indicates that the differences in ranking between the groups are not statistically different, which means that the ranking is fair. This method contributes both to the development of data envelopment analysis, and the inclusion of fairness in efficiency analysis.",
publisher = "Elsevier, Amsterdam",
journal = "European Journal of Operational Research",
title = "FairDEA-Removing disparate impact from efficiency scores",
pages = "1098-1088",
number = "3",
volume = "301",
doi = "10.1016/j.ejor.2021.12.001",
url = "conv_2678"
}

Radovanović, S., Savić, G., Delibašić, B.,& Suknović, M.. (2022). FairDEA-Removing disparate impact from efficiency scores. in European Journal of Operational Research
Elsevier, Amsterdam., 301(3), 1088-1098.
https://doi.org/10.1016/j.ejor.2021.12.001
conv_2678

Radovanović S, Savić G, Delibašić B, Suknović M. FairDEA-Removing disparate impact from efficiency scores. in European Journal of Operational Research. 2022;301(3):1088-1098.
doi:10.1016/j.ejor.2021.12.001
conv_2678 .

Radovanović, Sandro, Savić, Gordana, Delibašić, Boris, Suknović, Milija, "FairDEA-Removing disparate impact from efficiency scores" in European Journal of Operational Research, 301, no. 3 (2022):1088-1098,
https://doi.org/10.1016/j.ejor.2021.12.001 .,
conv_2678 .

TY  - JOUR
AU  - Vukicević, Ana
AU  - Vukićević, Milan
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2292
AB  - Crowdsourcing and crowd voting systems are being increasingly used in societal, industry, and academic problems (labeling, recommendations, social choice, etc.) due to their possibility to exploit "wisdom of crowd" and obtain good quality solutions, and/or voter satisfaction, with high cost-efficiency. However, the decisions based on crowd vote aggregation do not guarantee high-quality results due to crowd voter data quality. Additionally, such decisions often do not satisfy the majority of voters due to data heterogeneity (multimodal or uniform vote distributions) and/or outliers, which cause traditional aggregation procedures (e.g., central tendency measures) to propose decisions with low voter satisfaction. In this research, we propose a system for the integration of crowd and expert knowledge in a crowdsourcing setting with limited resources. The system addresses the problem of sparse voting data by using machine learning models (matrix factorization and regression) for the estimation of crowd and expert votes/grades. The problem of vote aggregation under multimodal or uniform vote distributions is addressed by the inclusion of expert votes and aggregation of crowd and expert votes based on optimization and bargaining models (Kalai-Smorodinsky and Nash) usually used in game theory. Experimental evaluation on real world and artificial problems showed that the bargaining-based aggregation outperforms the traditional methods in terms of cumulative satisfaction of experts and crowd. Additionally, the machine learning models showed satisfactory predictive performance and enabled cost reduction in the process of vote collection.
PB  - Springer, Dordrecht
T2  - Group Decision and Negotiation
T1  - BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing
EP  - 818
IS  - 4
SP  - 789
VL  - 31
DO  - 10.1007/s10726-022-09783-0
UR  - conv_2685
ER  - 

@article{
author = "Vukicević, Ana and Vukićević, Milan and Radovanović, Sandro and Delibašić, Boris",
year = "2022",
abstract = "Crowdsourcing and crowd voting systems are being increasingly used in societal, industry, and academic problems (labeling, recommendations, social choice, etc.) due to their possibility to exploit "wisdom of crowd" and obtain good quality solutions, and/or voter satisfaction, with high cost-efficiency. However, the decisions based on crowd vote aggregation do not guarantee high-quality results due to crowd voter data quality. Additionally, such decisions often do not satisfy the majority of voters due to data heterogeneity (multimodal or uniform vote distributions) and/or outliers, which cause traditional aggregation procedures (e.g., central tendency measures) to propose decisions with low voter satisfaction. In this research, we propose a system for the integration of crowd and expert knowledge in a crowdsourcing setting with limited resources. The system addresses the problem of sparse voting data by using machine learning models (matrix factorization and regression) for the estimation of crowd and expert votes/grades. The problem of vote aggregation under multimodal or uniform vote distributions is addressed by the inclusion of expert votes and aggregation of crowd and expert votes based on optimization and bargaining models (Kalai-Smorodinsky and Nash) usually used in game theory. Experimental evaluation on real world and artificial problems showed that the bargaining-based aggregation outperforms the traditional methods in terms of cumulative satisfaction of experts and crowd. Additionally, the machine learning models showed satisfactory predictive performance and enabled cost reduction in the process of vote collection.",
publisher = "Springer, Dordrecht",
journal = "Group Decision and Negotiation",
title = "BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing",
pages = "818-789",
number = "4",
volume = "31",
doi = "10.1007/s10726-022-09783-0",
url = "conv_2685"
}

Vukicević, A., Vukićević, M., Radovanović, S.,& Delibašić, B.. (2022). BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing. in Group Decision and Negotiation
Springer, Dordrecht., 31(4), 789-818.
https://doi.org/10.1007/s10726-022-09783-0
conv_2685

Vukicević A, Vukićević M, Radovanović S, Delibašić B. BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing. in Group Decision and Negotiation. 2022;31(4):789-818.
doi:10.1007/s10726-022-09783-0
conv_2685 .

Vukicević, Ana, Vukićević, Milan, Radovanović, Sandro, Delibašić, Boris, "BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing" in Group Decision and Negotiation, 31, no. 4 (2022):789-818,
https://doi.org/10.1007/s10726-022-09783-0 .,
conv_2685 .

TY  - JOUR
AU  - Petrović, Andrija
AU  - Nikolić, Mladen
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
AU  - Jovanović, Miloš
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2370
AB  - With growing awareness of societal impact of artificial intelligence, fairness has become an important aspect of machine learning algorithms. The issue is that human biases towards certain groups of popu-lation, defined by sensitive features like race and gender, are introduced to the training data through data collection and labeling. Two important directions of fairness ensuring research have focused on (i) instance weighting in order to decrease the impact of more biased instances and (ii) adversarial training in order to construct data representations informative of the target variable, but uninformative of the sensitive attributes. In this paper we propose a Fair Adversarial Instance Re-weighting (FAIR) method, which uses adversarial training to learn instance weighting function that ensures fair predictions. Merging the two paradigms, it inherits desirable properties from both interpretability of reweighting and end-to-end trainability of adversarial training. We propose four different variants of the method and, among other things, demonstrate how the method can be cast in a fully probabilistic framework. Additionally, theoretical analysis of FAIR models' properties is provided. We compare FAIR models to ten other related and state-of-the-art models and demonstrate that FAIR is able to achieve a better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first model that merges reweighting and adversarial approaches by means of a weighting function that can provide inter-pretable information about fairness of individual instances.
PB  - Elsevier, Amsterdam
T2  - Neurocomputing
T1  - FAIR: Fair adversarial instance re-weighting
EP  - 37
SP  - 14
VL  - 476
DO  - 10.1016/j.neucom.2021.12.082
UR  - conv_2621
ER  - 

@article{
author = "Petrović, Andrija and Nikolić, Mladen and Radovanović, Sandro and Delibašić, Boris and Jovanović, Miloš",
year = "2022",
abstract = "With growing awareness of societal impact of artificial intelligence, fairness has become an important aspect of machine learning algorithms. The issue is that human biases towards certain groups of popu-lation, defined by sensitive features like race and gender, are introduced to the training data through data collection and labeling. Two important directions of fairness ensuring research have focused on (i) instance weighting in order to decrease the impact of more biased instances and (ii) adversarial training in order to construct data representations informative of the target variable, but uninformative of the sensitive attributes. In this paper we propose a Fair Adversarial Instance Re-weighting (FAIR) method, which uses adversarial training to learn instance weighting function that ensures fair predictions. Merging the two paradigms, it inherits desirable properties from both interpretability of reweighting and end-to-end trainability of adversarial training. We propose four different variants of the method and, among other things, demonstrate how the method can be cast in a fully probabilistic framework. Additionally, theoretical analysis of FAIR models' properties is provided. We compare FAIR models to ten other related and state-of-the-art models and demonstrate that FAIR is able to achieve a better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first model that merges reweighting and adversarial approaches by means of a weighting function that can provide inter-pretable information about fairness of individual instances.",
publisher = "Elsevier, Amsterdam",
journal = "Neurocomputing",
title = "FAIR: Fair adversarial instance re-weighting",
pages = "37-14",
volume = "476",
doi = "10.1016/j.neucom.2021.12.082",
url = "conv_2621"
}

Petrović, A., Nikolić, M., Radovanović, S., Delibašić, B.,& Jovanović, M.. (2022). FAIR: Fair adversarial instance re-weighting. in Neurocomputing
Elsevier, Amsterdam., 476, 14-37.
https://doi.org/10.1016/j.neucom.2021.12.082
conv_2621

Petrović A, Nikolić M, Radovanović S, Delibašić B, Jovanović M. FAIR: Fair adversarial instance re-weighting. in Neurocomputing. 2022;476:14-37.
doi:10.1016/j.neucom.2021.12.082
conv_2621 .

Petrović, Andrija, Nikolić, Mladen, Radovanović, Sandro, Delibašić, Boris, Jovanović, Miloš, "FAIR: Fair adversarial instance re-weighting" in Neurocomputing, 476 (2022):14-37,
https://doi.org/10.1016/j.neucom.2021.12.082 .,
conv_2621 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
AU  - Marković, Aleksandar
AU  - Suknović, Milija
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2416
AB  - Algorithmic decision making is gaining popularity in today's business. The need for fast, accurate, and complex decisions forces decision-makers to take advantage of algorithms. However, algorithms can create unwanted bias or undesired consequences that can be averted. In this paper, we propose a MAX-MIN fair cross-efficiency data envelopment analysis (DEA) model that solves the problem of high variance cross-efficiency scores. The MAX-MIN cross-efficiency procedure is in accordance with John Rawls's Theory of justice by allowing efficiency and cross-efficiency estimation such that the greatest benefit of the least-advantaged decision making unit is achieved. The proposed mathematical model is tested on a healthcare related dataset. The results suggest that the proposed method solves several issues of cross-efficiency scores. First, it enables full rankings by having the ability to discriminate between the efficiency scores of DMUs. Second, the variance of cross-efficiency scores is reduced, and finally, fairness is introduced through optimization of the minimal efficiency scores.
PB  - IEEE Computer Society
C3  - Proceedings of the Annual Hawaii International Conference on System Sciences
T1  - Achieving MAX-MIN Fair Cross-efficiency scores in Data Envelopment Analysis
EP  - 1530
SP  - 1522
VL  - 2022-January
UR  - conv_3792
ER  - 

@conference{
author = "Radovanović, Sandro and Delibašić, Boris and Marković, Aleksandar and Suknović, Milija",
year = "2022",
abstract = "Algorithmic decision making is gaining popularity in today's business. The need for fast, accurate, and complex decisions forces decision-makers to take advantage of algorithms. However, algorithms can create unwanted bias or undesired consequences that can be averted. In this paper, we propose a MAX-MIN fair cross-efficiency data envelopment analysis (DEA) model that solves the problem of high variance cross-efficiency scores. The MAX-MIN cross-efficiency procedure is in accordance with John Rawls's Theory of justice by allowing efficiency and cross-efficiency estimation such that the greatest benefit of the least-advantaged decision making unit is achieved. The proposed mathematical model is tested on a healthcare related dataset. The results suggest that the proposed method solves several issues of cross-efficiency scores. First, it enables full rankings by having the ability to discriminate between the efficiency scores of DMUs. Second, the variance of cross-efficiency scores is reduced, and finally, fairness is introduced through optimization of the minimal efficiency scores.",
publisher = "IEEE Computer Society",
journal = "Proceedings of the Annual Hawaii International Conference on System Sciences",
title = "Achieving MAX-MIN Fair Cross-efficiency scores in Data Envelopment Analysis",
pages = "1530-1522",
volume = "2022-January",
url = "conv_3792"
}

Radovanović, S., Delibašić, B., Marković, A.,& Suknović, M.. (2022). Achieving MAX-MIN Fair Cross-efficiency scores in Data Envelopment Analysis. in Proceedings of the Annual Hawaii International Conference on System Sciences
IEEE Computer Society., 2022-January, 1522-1530.
conv_3792

Radovanović S, Delibašić B, Marković A, Suknović M. Achieving MAX-MIN Fair Cross-efficiency scores in Data Envelopment Analysis. in Proceedings of the Annual Hawaii International Conference on System Sciences. 2022;2022-January:1522-1530.
conv_3792 .

Radovanović, Sandro, Delibašić, Boris, Marković, Aleksandar, Suknović, Milija, "Achieving MAX-MIN Fair Cross-efficiency scores in Data Envelopment Analysis" in Proceedings of the Annual Hawaii International Conference on System Sciences, 2022-January (2022):1522-1530,
conv_3792 .

TY  - CONF
AU  - Delibašić, Boris
AU  - Radovanović, Sandro
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2415
AB  - A vast application of machine learning and decision-making algorithms for decision support in various areas of life caused the need for the algorithms to take into account additional constraints, such as non-discriminatory behavior or imposing fairness, or social welfare prior to proposing decisions to decision makers. These constraints can be fulfilled by carefully guiding the whole decision-making and data governance process, by adjusting decision-making, data mining and machine learning algorithms to fulfill additional constraints. For example, by adapting CRISP-DM methodology to account for possible biases, by imposing instance-dependent cost-sensitive learning, or enforcing equality in data envelopment analysis as presented in this mini-track.
PB  - IEEE Computer Society
C3  - Proceedings of the Annual Hawaii International Conference on System Sciences
T1  - Introduction to Fairness in Algorithmic Decision Making mini-track
EP  - 1521
SP  - 1520
VL  - 2022-January
UR  - conv_3791
ER  - 

@conference{
author = "Delibašić, Boris and Radovanović, Sandro",
year = "2022",
abstract = "A vast application of machine learning and decision-making algorithms for decision support in various areas of life caused the need for the algorithms to take into account additional constraints, such as non-discriminatory behavior or imposing fairness, or social welfare prior to proposing decisions to decision makers. These constraints can be fulfilled by carefully guiding the whole decision-making and data governance process, by adjusting decision-making, data mining and machine learning algorithms to fulfill additional constraints. For example, by adapting CRISP-DM methodology to account for possible biases, by imposing instance-dependent cost-sensitive learning, or enforcing equality in data envelopment analysis as presented in this mini-track.",
publisher = "IEEE Computer Society",
journal = "Proceedings of the Annual Hawaii International Conference on System Sciences",
title = "Introduction to Fairness in Algorithmic Decision Making mini-track",
pages = "1521-1520",
volume = "2022-January",
url = "conv_3791"
}

Delibašić, B.,& Radovanović, S.. (2022). Introduction to Fairness in Algorithmic Decision Making mini-track. in Proceedings of the Annual Hawaii International Conference on System Sciences
IEEE Computer Society., 2022-January, 1520-1521.
conv_3791

Delibašić B, Radovanović S. Introduction to Fairness in Algorithmic Decision Making mini-track. in Proceedings of the Annual Hawaii International Conference on System Sciences. 2022;2022-January:1520-1521.
conv_3791 .

Delibašić, Boris, Radovanović, Sandro, "Introduction to Fairness in Algorithmic Decision Making mini-track" in Proceedings of the Annual Hawaii International Conference on System Sciences, 2022-January (2022):1520-1521,
conv_3791 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2022
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2408
AB  - Using machine learning algorithms in social environments and systems requires stricter and more detailed control. More specifically, the cost of error in such systems is much higher. Therefore, one should ensure that important decisions, such as whether to convict a person or not based on the previous criminal record, are by the legal requirements and not biased toward a group of people. One can find many many papers in the literature aimed at mitigating or eliminating unwanted bias in machine learning models. A significant part of these efforts add fairness constraint to the mathematical model or adds a regularization term to the loss function. In this paper, we show that optimizing the loss function given the fairness constraint or regularization for unfairness can surprisingly yield unfair solutions. This is due to the linear relaxation of the fairness function. By analyzing the gap between the true value of fairness and the one obtained using linear relaxation, we found that the gap can be as high as around 21% for the COMPAS dataset, and around 35% for the Adult dataset. In addition, we show that the fairness gap is consistent regardless of the strength of the fairness constraint or regularization.
PB  - Elsevier B.V.
C3  - Procedia Computer Science
T1  - Do we Reach Desired Disparate Impact with In-Processing Fairness Techniques?
EP  - 264
IS  - C
SP  - 257
VL  - 214
DO  - 10.1016/j.procs.2022.11.173
UR  - conv_3756
ER  - 

@conference{
author = "Radovanović, Sandro and Delibašić, Boris and Suknović, Milija",
year = "2022",
abstract = "Using machine learning algorithms in social environments and systems requires stricter and more detailed control. More specifically, the cost of error in such systems is much higher. Therefore, one should ensure that important decisions, such as whether to convict a person or not based on the previous criminal record, are by the legal requirements and not biased toward a group of people. One can find many many papers in the literature aimed at mitigating or eliminating unwanted bias in machine learning models. A significant part of these efforts add fairness constraint to the mathematical model or adds a regularization term to the loss function. In this paper, we show that optimizing the loss function given the fairness constraint or regularization for unfairness can surprisingly yield unfair solutions. This is due to the linear relaxation of the fairness function. By analyzing the gap between the true value of fairness and the one obtained using linear relaxation, we found that the gap can be as high as around 21% for the COMPAS dataset, and around 35% for the Adult dataset. In addition, we show that the fairness gap is consistent regardless of the strength of the fairness constraint or regularization.",
publisher = "Elsevier B.V.",
journal = "Procedia Computer Science",
title = "Do we Reach Desired Disparate Impact with In-Processing Fairness Techniques?",
pages = "264-257",
number = "C",
volume = "214",
doi = "10.1016/j.procs.2022.11.173",
url = "conv_3756"
}

Radovanović, S., Delibašić, B.,& Suknović, M.. (2022). Do we Reach Desired Disparate Impact with In-Processing Fairness Techniques?. in Procedia Computer Science
Elsevier B.V.., 214(C), 257-264.
https://doi.org/10.1016/j.procs.2022.11.173
conv_3756

Radovanović S, Delibašić B, Suknović M. Do we Reach Desired Disparate Impact with In-Processing Fairness Techniques?. in Procedia Computer Science. 2022;214(C):257-264.
doi:10.1016/j.procs.2022.11.173
conv_3756 .

Radovanović, Sandro, Delibašić, Boris, Suknović, Milija, "Do we Reach Desired Disparate Impact with In-Processing Fairness Techniques?" in Procedia Computer Science, 214, no. C (2022):257-264,
https://doi.org/10.1016/j.procs.2022.11.173 .,
conv_3756 .

TY  - JOUR
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2021
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2169
AB  - Online learning environments became popular in recent years. Due to high attrition rates, the problem of student dropouts became of immense importance for course designers, and course makers. In this paper, we utilized lasso and ridge logistic regression to create a prediction model for dropout on the Open University database. We investigated how early dropout can be predicted, and why dropouts occur. To answer the first question, we created models for eight different time frames, ranging from the beginning of the course to the mid-term. There are two results based on two definitions of dropout. Results show that at the beginning AUC of the prediction model is 0.549 and 0.661 and rises to 0.681 and 0.869 at mid-term. By analyzing logistic regression coefficients, we showed that at the beginning of the course demographic features of the student and course description features are the most important variables for dropout prediction, while later student activity gains more importance.
PB  - ComSIS Consortium
T2  - Computer Science and Information Systems / ComSIS
T1  - Predicting Dropout in Online Learning Environments
EP  - 978
IS  - 3
SP  - 957
VL  - 18
DO  - 10.2298/CSIS200920053R
UR  - conv_2522
ER  - 

@article{
author = "Radovanović, Sandro and Delibašić, Boris and Suknović, Milija",
year = "2021",
abstract = "Online learning environments became popular in recent years. Due to high attrition rates, the problem of student dropouts became of immense importance for course designers, and course makers. In this paper, we utilized lasso and ridge logistic regression to create a prediction model for dropout on the Open University database. We investigated how early dropout can be predicted, and why dropouts occur. To answer the first question, we created models for eight different time frames, ranging from the beginning of the course to the mid-term. There are two results based on two definitions of dropout. Results show that at the beginning AUC of the prediction model is 0.549 and 0.661 and rises to 0.681 and 0.869 at mid-term. By analyzing logistic regression coefficients, we showed that at the beginning of the course demographic features of the student and course description features are the most important variables for dropout prediction, while later student activity gains more importance.",
publisher = "ComSIS Consortium",
journal = "Computer Science and Information Systems / ComSIS",
title = "Predicting Dropout in Online Learning Environments",
pages = "978-957",
number = "3",
volume = "18",
doi = "10.2298/CSIS200920053R",
url = "conv_2522"
}

Radovanović, S., Delibašić, B.,& Suknović, M.. (2021). Predicting Dropout in Online Learning Environments. in Computer Science and Information Systems / ComSIS
ComSIS Consortium., 18(3), 957-978.
https://doi.org/10.2298/CSIS200920053R
conv_2522

Radovanović S, Delibašić B, Suknović M. Predicting Dropout in Online Learning Environments. in Computer Science and Information Systems / ComSIS. 2021;18(3):957-978.
doi:10.2298/CSIS200920053R
conv_2522 .

Radovanović, Sandro, Delibašić, Boris, Suknović, Milija, "Predicting Dropout in Online Learning Environments" in Computer Science and Information Systems / ComSIS, 18, no. 3 (2021):957-978,
https://doi.org/10.2298/CSIS200920053R .,
conv_2522 .

TY  - CONF
AU  - Rančić, S.
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
PY  - 2021
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2263
AB  - Applying machine learning in real-world applications may have various implications on companies, but individuals as well. Besides obtaining lower costs, faster time to decision and higher accuracy of the decision, automation of decisions can lead to unethical and illegal consequences. More specifically, predictions can systematically discriminate against a certain group of people. This comes mainly due to dataset bias. In this paper, we investigate instances oversampling to improve fairness. We tried several strategies and two techniques, namely SMOTE and random oversampling. Besides traditional oversampling techniques, we tried oversampling of instances based on sensitive attributes as well (i.e. gender or race). We demonstrate on real-world datasets (Adult and COMPAS) that oversampling techniques increase fairness, without greater decrease in predictive accuracy. Oversampling improved fairness up to 15% and AUPRC up to 3% with a loss in AUC of 2%.
PB  - Springer Science and Business Media Deutschland GmbH
C3  - Lecture Notes in Business Information Processing
T1  - Investigating Oversampling Techniques for Fair Machine Learning Models
EP  - 123
SP  - 110
VL  - 414 LNBIP
DO  - 10.1007/978-3-030-73976-8_9
UR  - conv_3674
ER  - 

@conference{
author = "Rančić, S. and Radovanović, Sandro and Delibašić, Boris",
year = "2021",
abstract = "Applying machine learning in real-world applications may have various implications on companies, but individuals as well. Besides obtaining lower costs, faster time to decision and higher accuracy of the decision, automation of decisions can lead to unethical and illegal consequences. More specifically, predictions can systematically discriminate against a certain group of people. This comes mainly due to dataset bias. In this paper, we investigate instances oversampling to improve fairness. We tried several strategies and two techniques, namely SMOTE and random oversampling. Besides traditional oversampling techniques, we tried oversampling of instances based on sensitive attributes as well (i.e. gender or race). We demonstrate on real-world datasets (Adult and COMPAS) that oversampling techniques increase fairness, without greater decrease in predictive accuracy. Oversampling improved fairness up to 15% and AUPRC up to 3% with a loss in AUC of 2%.",
publisher = "Springer Science and Business Media Deutschland GmbH",
journal = "Lecture Notes in Business Information Processing",
title = "Investigating Oversampling Techniques for Fair Machine Learning Models",
pages = "123-110",
volume = "414 LNBIP",
doi = "10.1007/978-3-030-73976-8_9",
url = "conv_3674"
}

Rančić, S., Radovanović, S.,& Delibašić, B.. (2021). Investigating Oversampling Techniques for Fair Machine Learning Models. in Lecture Notes in Business Information Processing
Springer Science and Business Media Deutschland GmbH., 414 LNBIP, 110-123.
https://doi.org/10.1007/978-3-030-73976-8_9
conv_3674

Rančić S, Radovanović S, Delibašić B. Investigating Oversampling Techniques for Fair Machine Learning Models. in Lecture Notes in Business Information Processing. 2021;414 LNBIP:110-123.
doi:10.1007/978-3-030-73976-8_9
conv_3674 .

Rančić, S., Radovanović, Sandro, Delibašić, Boris, "Investigating Oversampling Techniques for Fair Machine Learning Models" in Lecture Notes in Business Information Processing, 414 LNBIP (2021):110-123,
https://doi.org/10.1007/978-3-030-73976-8_9 .,
conv_3674 .

TY  - JOUR
AU  - Delibašić, Boris
AU  - Radovanović, Sandro
AU  - Jayawickrama, Uchitha
PY  - 2021
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2193
PB  - IGI Global, Hershey
T2  - International Journal of Decision Support System Technology
T1  - Special Issue on AI for Intelligent Decision Support Systems Guest Editorial Preface
EP  - IV
IS  - 1
SP  - IV
VL  - 13
UR  - conv_2440
ER  - 

@article{
author = "Delibašić, Boris and Radovanović, Sandro and Jayawickrama, Uchitha",
year = "2021",
publisher = "IGI Global, Hershey",
journal = "International Journal of Decision Support System Technology",
title = "Special Issue on AI for Intelligent Decision Support Systems Guest Editorial Preface",
pages = "IV-IV",
number = "1",
volume = "13",
url = "conv_2440"
}

Delibašić, B., Radovanović, S.,& Jayawickrama, U.. (2021). Special Issue on AI for Intelligent Decision Support Systems Guest Editorial Preface. in International Journal of Decision Support System Technology
IGI Global, Hershey., 13(1), IV-IV.
conv_2440

Delibašić B, Radovanović S, Jayawickrama U. Special Issue on AI for Intelligent Decision Support Systems Guest Editorial Preface. in International Journal of Decision Support System Technology. 2021;13(1):IV-IV.
conv_2440 .

Delibašić, Boris, Radovanović, Sandro, Jayawickrama, Uchitha, "Special Issue on AI for Intelligent Decision Support Systems Guest Editorial Preface" in International Journal of Decision Support System Technology, 13, no. 1 (2021):IV-IV,
conv_2440 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Petrović, Andrija
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2021
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2175
AB  - In today's business, decision-making is heavily dependent on algorithms. Algorithms may originate from operational research, machine learning, but also decision theory. Regardless of their origin, the decision-maker may create unwanted disparities regarding race, gender, or religion. These disparities may further lead to legal consequences. To mitigate unwanted consequences one must adjust either algorithms or decisions. In this paper, we adjust the popular decision-making method TOPSIS to produce utility scores without disparate impact. This is done is by introducing "fairness weight" that is used for the calculation of the utility function of TOPSIS method. Fairness weight should provide the smallest possible intervention needed for a decision without disparate impact. The effectiveness of the proposed solution is shown on the synthetic dataset, as well as on the exemplar dataset regarding criminal justice.
PB  - Fac Organization And Informatics,  Univ Zagreb, Varazdin
C3  - Central European Conference on Information and Intelligent Systems (CECIIS 2021)
T1  - Eliminating Disparate Impact in MCDM: The case of TOPSIS
EP  - 282
SP  - 275
UR  - conv_2762
ER  - 

@conference{
author = "Radovanović, Sandro and Petrović, Andrija and Delibašić, Boris and Suknović, Milija",
year = "2021",
abstract = "In today's business, decision-making is heavily dependent on algorithms. Algorithms may originate from operational research, machine learning, but also decision theory. Regardless of their origin, the decision-maker may create unwanted disparities regarding race, gender, or religion. These disparities may further lead to legal consequences. To mitigate unwanted consequences one must adjust either algorithms or decisions. In this paper, we adjust the popular decision-making method TOPSIS to produce utility scores without disparate impact. This is done is by introducing "fairness weight" that is used for the calculation of the utility function of TOPSIS method. Fairness weight should provide the smallest possible intervention needed for a decision without disparate impact. The effectiveness of the proposed solution is shown on the synthetic dataset, as well as on the exemplar dataset regarding criminal justice.",
publisher = "Fac Organization And Informatics,  Univ Zagreb, Varazdin",
journal = "Central European Conference on Information and Intelligent Systems (CECIIS 2021)",
title = "Eliminating Disparate Impact in MCDM: The case of TOPSIS",
pages = "282-275",
url = "conv_2762"
}

Radovanović, S., Petrović, A., Delibašić, B.,& Suknović, M.. (2021). Eliminating Disparate Impact in MCDM: The case of TOPSIS. in Central European Conference on Information and Intelligent Systems (CECIIS 2021)
Fac Organization And Informatics,  Univ Zagreb, Varazdin., 275-282.
conv_2762

Radovanović S, Petrović A, Delibašić B, Suknović M. Eliminating Disparate Impact in MCDM: The case of TOPSIS. in Central European Conference on Information and Intelligent Systems (CECIIS 2021). 2021;:275-282.
conv_2762 .

Radovanović, Sandro, Petrović, Andrija, Delibašić, Boris, Suknović, Milija, "Eliminating Disparate Impact in MCDM: The case of TOPSIS" in Central European Conference on Information and Intelligent Systems (CECIIS 2021) (2021):275-282,
conv_2762 .

TY  - JOUR
AU  - Rakić, Mia
AU  - Monje, Alberto
AU  - Radovanović, Sandro
AU  - Petkovic-Curcin, Aleksandra
AU  - Vojvodić, Danilo
AU  - Tatić, Zoran
PY  - 2020
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2047
AB  - Background Study objectives were 1) to estimate diagnostic capacity of clinical parameters, receptor activator of nuclear factor kappa-B (RANKL) and osteoprotegerin (OPG) to diagnose healthy peri-implant condition (HI), peri-implant mucositis (PIM) and peri-implantitis (PIMP) by assessing respective diagnostic accuracy, sensitivity, specificity and diagnostic ranges 2) to develop personalized diagnostic model (PDM) for implant monitoring. Methods Split-mouth study included 126 patients and 252 implants (HI = 126, PIM = 57, and PIMP = 69). RANKL and OPG concentrations were estimated in peri-implant crevicular fluid using enzyme-linked immunosorbent assay method and assessed with clinical parameters using routine statistics, while the diagnostic capacity of individual parameters and overall clinical diagnosis were estimated using classifying algorithms. PDM was developed using decision trees. Results Bleeding on probing (BOP), plaque index, and probing depth (PD) were confirmed reliable discriminants between peri-implant health and disease, while increase in PD (PD > 4 mm) and suppuration were good discriminants amongst PIM/PIMP. Bone turnover markers (BTMs) demonstrated presence of bone resorption in PIM; between comparable diagnostic ranges PIM/PIMP, PIMP was clinically distinguished from PIM in about 60% of patients while 40% remained diagnosed as false negatives. PDM demonstrated highest diagnostic capacity (accuracy: 96.27%, sensitivity: 95.00%, specificity: 100%) and defined HI: BOP  LT = 0.25%; PIM: BOP >0.25%, PD  LT = 4.5 mm; PIMP: BOP >0.25%, PD >4.5 mm and RANKL  LT = 19.9 pg/site; PIM: BOP >0.25%, PD >4.5 mm, and RANKL >19.9 pg/site. Conclusions BTMs demonstrated capacity to substantially improve clinical diagnosis of peri-implant conditions. Considering lack of difference in BTMs between PIM/PIMP and cluster of PIM with exceeding BTMs, a more refined definition of peri-implant conditions according to biological characteristics is required for further BTMs validation and appropriate PIMP management.
PB  - Wiley, Hoboken
T2  - Journal of Periodontology
T1  - Is the personalized approach the key to improve clinical diagnosis of peri-implant conditions? The role of bone markers
EP  - 869
IS  - 7
SP  - 859
VL  - 91
DO  - 10.1002/JPER.19-0283
UR  - conv_2261
ER  - 

@article{
author = "Rakić, Mia and Monje, Alberto and Radovanović, Sandro and Petkovic-Curcin, Aleksandra and Vojvodić, Danilo and Tatić, Zoran",
year = "2020",
abstract = "Background Study objectives were 1) to estimate diagnostic capacity of clinical parameters, receptor activator of nuclear factor kappa-B (RANKL) and osteoprotegerin (OPG) to diagnose healthy peri-implant condition (HI), peri-implant mucositis (PIM) and peri-implantitis (PIMP) by assessing respective diagnostic accuracy, sensitivity, specificity and diagnostic ranges 2) to develop personalized diagnostic model (PDM) for implant monitoring. Methods Split-mouth study included 126 patients and 252 implants (HI = 126, PIM = 57, and PIMP = 69). RANKL and OPG concentrations were estimated in peri-implant crevicular fluid using enzyme-linked immunosorbent assay method and assessed with clinical parameters using routine statistics, while the diagnostic capacity of individual parameters and overall clinical diagnosis were estimated using classifying algorithms. PDM was developed using decision trees. Results Bleeding on probing (BOP), plaque index, and probing depth (PD) were confirmed reliable discriminants between peri-implant health and disease, while increase in PD (PD > 4 mm) and suppuration were good discriminants amongst PIM/PIMP. Bone turnover markers (BTMs) demonstrated presence of bone resorption in PIM; between comparable diagnostic ranges PIM/PIMP, PIMP was clinically distinguished from PIM in about 60% of patients while 40% remained diagnosed as false negatives. PDM demonstrated highest diagnostic capacity (accuracy: 96.27%, sensitivity: 95.00%, specificity: 100%) and defined HI: BOP  LT = 0.25%; PIM: BOP >0.25%, PD  LT = 4.5 mm; PIMP: BOP >0.25%, PD >4.5 mm and RANKL  LT = 19.9 pg/site; PIM: BOP >0.25%, PD >4.5 mm, and RANKL >19.9 pg/site. Conclusions BTMs demonstrated capacity to substantially improve clinical diagnosis of peri-implant conditions. Considering lack of difference in BTMs between PIM/PIMP and cluster of PIM with exceeding BTMs, a more refined definition of peri-implant conditions according to biological characteristics is required for further BTMs validation and appropriate PIMP management.",
publisher = "Wiley, Hoboken",
journal = "Journal of Periodontology",
title = "Is the personalized approach the key to improve clinical diagnosis of peri-implant conditions? The role of bone markers",
pages = "869-859",
number = "7",
volume = "91",
doi = "10.1002/JPER.19-0283",
url = "conv_2261"
}

Rakić, M., Monje, A., Radovanović, S., Petkovic-Curcin, A., Vojvodić, D.,& Tatić, Z.. (2020). Is the personalized approach the key to improve clinical diagnosis of peri-implant conditions? The role of bone markers. in Journal of Periodontology
Wiley, Hoboken., 91(7), 859-869.
https://doi.org/10.1002/JPER.19-0283
conv_2261

Rakić M, Monje A, Radovanović S, Petkovic-Curcin A, Vojvodić D, Tatić Z. Is the personalized approach the key to improve clinical diagnosis of peri-implant conditions? The role of bone markers. in Journal of Periodontology. 2020;91(7):859-869.
doi:10.1002/JPER.19-0283
conv_2261 .

Rakić, Mia, Monje, Alberto, Radovanović, Sandro, Petkovic-Curcin, Aleksandra, Vojvodić, Danilo, Tatić, Zoran, "Is the personalized approach the key to improve clinical diagnosis of peri-implant conditions? The role of bone markers" in Journal of Periodontology, 91, no. 7 (2020):859-869,
https://doi.org/10.1002/JPER.19-0283 .,
conv_2261 .

TY  - CONF
AU  - Delibašić, Boris
AU  - Radovanović, Sandro
AU  - Jovanović, Miloš
AU  - Suknović, Milija
PY  - 2020
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2143
AB  - This paper provides an overview of research on ski lift transportation data, a still heavily underused resource in ski resorts. To the best of our knowledge, this is the first paper that provides an overview of the efforts done in analyzing ski lift transportation data with the goal to advance the decision-making process in ski resorts. The paper is separated into three major research directions, the first being the clustering of ski lift transportation data. The second research direction is concerned with the exploitation of ski lift transportation data for ski injury research and prevention. The third research direction is concerned with congestion analysis in ski resorts. We provide directions for future research in the conclusion.
PB  - Springer Science and Business Media B.V.
C3  - Springer Proceedings in Business and Economics
T1  - Improving Decision-Making in Ski Resorts by Analysing Ski Lift Transportation—A Review
EP  - 273
SP  - 265
DO  - 10.1007/978-3-030-21990-1_16
UR  - conv_3589
ER  - 

@conference{
author = "Delibašić, Boris and Radovanović, Sandro and Jovanović, Miloš and Suknović, Milija",
year = "2020",
abstract = "This paper provides an overview of research on ski lift transportation data, a still heavily underused resource in ski resorts. To the best of our knowledge, this is the first paper that provides an overview of the efforts done in analyzing ski lift transportation data with the goal to advance the decision-making process in ski resorts. The paper is separated into three major research directions, the first being the clustering of ski lift transportation data. The second research direction is concerned with the exploitation of ski lift transportation data for ski injury research and prevention. The third research direction is concerned with congestion analysis in ski resorts. We provide directions for future research in the conclusion.",
publisher = "Springer Science and Business Media B.V.",
journal = "Springer Proceedings in Business and Economics",
title = "Improving Decision-Making in Ski Resorts by Analysing Ski Lift Transportation—A Review",
pages = "273-265",
doi = "10.1007/978-3-030-21990-1_16",
url = "conv_3589"
}

Delibašić, B., Radovanović, S., Jovanović, M.,& Suknović, M.. (2020). Improving Decision-Making in Ski Resorts by Analysing Ski Lift Transportation—A Review. in Springer Proceedings in Business and Economics
Springer Science and Business Media B.V.., 265-273.
https://doi.org/10.1007/978-3-030-21990-1_16
conv_3589

Delibašić B, Radovanović S, Jovanović M, Suknović M. Improving Decision-Making in Ski Resorts by Analysing Ski Lift Transportation—A Review. in Springer Proceedings in Business and Economics. 2020;:265-273.
doi:10.1007/978-3-030-21990-1_16
conv_3589 .

Delibašić, Boris, Radovanović, Sandro, Jovanović, Miloš, Suknović, Milija, "Improving Decision-Making in Ski Resorts by Analysing Ski Lift Transportation—A Review" in Springer Proceedings in Business and Economics (2020):265-273,
https://doi.org/10.1007/978-3-030-21990-1_16 .,
conv_3589 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Petrović, A.
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2020
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2130
AB  - Machine learning has been subject to discussion from the legal and ethical points of view in recent years. Automation of the decision-making process can lead to unethical acts with legal consequences. There are examples where the decision made by machine learning systems was unfairly biased toward some group of people. This is mainly because data used for model training were biased and thus developed a predictive model inherited that bias. Therefore, the process of learning a predictive model must be aware and account for the possible bias in the data. In this paper, we propose a modification of the logistic regression algorithm that adds one known and one novel fairness constraints into the process of model learning, thus forcing the predictive model not to create disparate impact and allow equal opportunity for every subpopulation. We demonstrate our model on real-world problems and show that a small reduction in predictive performance can yield a high improvement in disparate impact and equality of opportunity.
PB  - Institute of Electrical and Electronics Engineers Inc.
C3  - INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings
T1  - Enforcing fairness in logistic regression algorithm
DO  - 10.1109/INISTA49547.2020.9194676
UR  - conv_3636
ER  - 

@conference{
author = "Radovanović, Sandro and Petrović, A. and Delibašić, Boris and Suknović, Milija",
year = "2020",
abstract = "Machine learning has been subject to discussion from the legal and ethical points of view in recent years. Automation of the decision-making process can lead to unethical acts with legal consequences. There are examples where the decision made by machine learning systems was unfairly biased toward some group of people. This is mainly because data used for model training were biased and thus developed a predictive model inherited that bias. Therefore, the process of learning a predictive model must be aware and account for the possible bias in the data. In this paper, we propose a modification of the logistic regression algorithm that adds one known and one novel fairness constraints into the process of model learning, thus forcing the predictive model not to create disparate impact and allow equal opportunity for every subpopulation. We demonstrate our model on real-world problems and show that a small reduction in predictive performance can yield a high improvement in disparate impact and equality of opportunity.",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
journal = "INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings",
title = "Enforcing fairness in logistic regression algorithm",
doi = "10.1109/INISTA49547.2020.9194676",
url = "conv_3636"
}

Radovanović, S., Petrović, A., Delibašić, B.,& Suknović, M.. (2020). Enforcing fairness in logistic regression algorithm. in INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings
Institute of Electrical and Electronics Engineers Inc...
https://doi.org/10.1109/INISTA49547.2020.9194676
conv_3636

Radovanović S, Petrović A, Delibašić B, Suknović M. Enforcing fairness in logistic regression algorithm. in INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings. 2020;.
doi:10.1109/INISTA49547.2020.9194676
conv_3636 .

Radovanović, Sandro, Petrović, A., Delibašić, Boris, Suknović, Milija, "Enforcing fairness in logistic regression algorithm" in INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings (2020),
https://doi.org/10.1109/INISTA49547.2020.9194676 .,
conv_3636 .

TY  - CONF
AU  - Kovačević, A.
AU  - Vukićević, Milan
AU  - Radovanović, Sandro
AU  - Delibašić, Boris
PY  - 2020
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2135
AB  - In recent years crowd-voting and crowd-sourcing systems are attracting increased attention in research and industry. As a part of computational social choice (COMSOC) crowd-voting and crowd-sourcing address important societal problems (e.g. participatory budgeting), but also many industry problems (e.g. sentiment analyses, data labeling, ranking and selection, etc.). Consequently, decisions that are based on aggregation of crowd votes do not guarantee high-quality results. Even more, in many cases majority of crowd voters may not be satisfied with final decisions if votes have high heterogeneity. On the other side in many crowd voting problems and settings it is possible to acquire and formalize knowledge and/or opinions from domain experts. Integration of expert knowledge and “Wisdom of crowd” should lead to high-quality decisions that satisfy crowd opinion. In this research, we address the problem of integration of experts domain knowledge with “Wisdom of crowds” by proposing machine learning based framework that enables ranking and selection of alternatives as well as quantification of quality of crowd votes. This framework enables weighting of crowd votes with respect to expert knowledge and procedures for modeling trade-off between crowd and experts satisfaction with final decisions (ranking or selection).
PB  - Springer
C3  - Communications in Computer and Information Science
T1  - CrEx-Wisdom Framework for Fusion of Crowd and Experts in Crowd Voting Environment – Machine Learning Approach
EP  - 144
SP  - 131
VL  - 1260 CCIS
DO  - 10.1007/978-3-030-55814-7_11
UR  - conv_3628
ER  - 

@conference{
author = "Kovačević, A. and Vukićević, Milan and Radovanović, Sandro and Delibašić, Boris",
year = "2020",
abstract = "In recent years crowd-voting and crowd-sourcing systems are attracting increased attention in research and industry. As a part of computational social choice (COMSOC) crowd-voting and crowd-sourcing address important societal problems (e.g. participatory budgeting), but also many industry problems (e.g. sentiment analyses, data labeling, ranking and selection, etc.). Consequently, decisions that are based on aggregation of crowd votes do not guarantee high-quality results. Even more, in many cases majority of crowd voters may not be satisfied with final decisions if votes have high heterogeneity. On the other side in many crowd voting problems and settings it is possible to acquire and formalize knowledge and/or opinions from domain experts. Integration of expert knowledge and “Wisdom of crowd” should lead to high-quality decisions that satisfy crowd opinion. In this research, we address the problem of integration of experts domain knowledge with “Wisdom of crowds” by proposing machine learning based framework that enables ranking and selection of alternatives as well as quantification of quality of crowd votes. This framework enables weighting of crowd votes with respect to expert knowledge and procedures for modeling trade-off between crowd and experts satisfaction with final decisions (ranking or selection).",
publisher = "Springer",
journal = "Communications in Computer and Information Science",
title = "CrEx-Wisdom Framework for Fusion of Crowd and Experts in Crowd Voting Environment – Machine Learning Approach",
pages = "144-131",
volume = "1260 CCIS",
doi = "10.1007/978-3-030-55814-7_11",
url = "conv_3628"
}

Kovačević, A., Vukićević, M., Radovanović, S.,& Delibašić, B.. (2020). CrEx-Wisdom Framework for Fusion of Crowd and Experts in Crowd Voting Environment – Machine Learning Approach. in Communications in Computer and Information Science
Springer., 1260 CCIS, 131-144.
https://doi.org/10.1007/978-3-030-55814-7_11
conv_3628

Kovačević A, Vukićević M, Radovanović S, Delibašić B. CrEx-Wisdom Framework for Fusion of Crowd and Experts in Crowd Voting Environment – Machine Learning Approach. in Communications in Computer and Information Science. 2020;1260 CCIS:131-144.
doi:10.1007/978-3-030-55814-7_11
conv_3628 .

Kovačević, A., Vukićević, Milan, Radovanović, Sandro, Delibašić, Boris, "CrEx-Wisdom Framework for Fusion of Crowd and Experts in Crowd Voting Environment – Machine Learning Approach" in Communications in Computer and Information Science, 1260 CCIS (2020):131-144,
https://doi.org/10.1007/978-3-030-55814-7_11 .,
conv_3628 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Zornić, Nikola
AU  - Delibašić, Boris
AU  - Marković, Aleksandar
AU  - Suknović, Milija
PY  - 2020
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2043
AB  - The allocation of human resources in the managerial environment is a hard task to perform and to learn. The cost of a real-life experience is very precious. Therefore, companies develop managerial games to provide near real-life experience for decision-makers. The teaching process could benefit if the outcomes of the managerial game could be predicted. Namely, the teacher could adjust teaching materials according to the expected result. Besides predicting an outcome, one would like to predict the emotions of the decision-maker. Having this in mind, we employed multi-label prediction models for prediction an outcome of the game and emotions of the decision-maker. The AUC ranges 0.62-0.66 for the classification of emotions, and similar to 0.76 for the outcome of the managerial game.
PB  - Fac Organization And Informatics,  Univ Zagreb, Varazdin
C3  - Central European Conference on Information and Intelligent Systems (CECIIS 2020)
T1  - Predicting the Result of a Managerial Game Using a Multi-Label Prediction Models
EP  - 281
SP  - 275
UR  - conv_2766
ER  - 

@conference{
author = "Radovanović, Sandro and Zornić, Nikola and Delibašić, Boris and Marković, Aleksandar and Suknović, Milija",
year = "2020",
abstract = "The allocation of human resources in the managerial environment is a hard task to perform and to learn. The cost of a real-life experience is very precious. Therefore, companies develop managerial games to provide near real-life experience for decision-makers. The teaching process could benefit if the outcomes of the managerial game could be predicted. Namely, the teacher could adjust teaching materials according to the expected result. Besides predicting an outcome, one would like to predict the emotions of the decision-maker. Having this in mind, we employed multi-label prediction models for prediction an outcome of the game and emotions of the decision-maker. The AUC ranges 0.62-0.66 for the classification of emotions, and similar to 0.76 for the outcome of the managerial game.",
publisher = "Fac Organization And Informatics,  Univ Zagreb, Varazdin",
journal = "Central European Conference on Information and Intelligent Systems (CECIIS 2020)",
title = "Predicting the Result of a Managerial Game Using a Multi-Label Prediction Models",
pages = "281-275",
url = "conv_2766"
}

Radovanović, S., Zornić, N., Delibašić, B., Marković, A.,& Suknović, M.. (2020). Predicting the Result of a Managerial Game Using a Multi-Label Prediction Models. in Central European Conference on Information and Intelligent Systems (CECIIS 2020)
Fac Organization And Informatics,  Univ Zagreb, Varazdin., 275-281.
conv_2766

Radovanović S, Zornić N, Delibašić B, Marković A, Suknović M. Predicting the Result of a Managerial Game Using a Multi-Label Prediction Models. in Central European Conference on Information and Intelligent Systems (CECIIS 2020). 2020;:275-281.
conv_2766 .

Radovanović, Sandro, Zornić, Nikola, Delibašić, Boris, Marković, Aleksandar, Suknović, Milija, "Predicting the Result of a Managerial Game Using a Multi-Label Prediction Models" in Central European Conference on Information and Intelligent Systems (CECIIS 2020) (2020):275-281,
conv_2766 .

TY  - CONF
AU  - Radovanović, Sandro
AU  - Petrović, Andrija
AU  - Delibašić, Boris
AU  - Suknović, Milija
PY  - 2019
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/1900
AB  - Creating predictive models using machine learning algorithms is often understood as a job where Data Scientist provides data to the algorithm without much intervention. With the rise of ethics in machine learning, predictive models need to be made fair. In this paper, we inspect the effects of pre-processing, in-processing and post-processing techniques for making predictive models fair. These techniques are applied to the hospital readmission prediction problem, where gender is considered as a sensitive attribute. The goal of the paper is to check whether unwanted discrimination between female and male in the logistic regression model exists and if exists to alleviate this problem making classifier fair. We employed logistic regression model which obtained AUC = 0.7959 and AUPRC = 0.5263. We have shown that reweighting strategy is a good trade-off between fairness and predictive performance. Namely, fairness is greatly improved, without much sacrificing predictive performance. We also show that adversarial debiasing is a good technique which combines predictive performance and fairness, and Equality of Odds technique optimizes Theil index.
PB  - Fac Organization And Informatics,  Univ Zagreb, Varazdin
C3  - Central European Conference on Information and Intelligent Systems (CECIIS 2019)
T1  - Making hospital readmission classifier fair - What is the cost?
EP  - 331
SP  - 325
UR  - conv_2761
ER  - 

@conference{
author = "Radovanović, Sandro and Petrović, Andrija and Delibašić, Boris and Suknović, Milija",
year = "2019",
abstract = "Creating predictive models using machine learning algorithms is often understood as a job where Data Scientist provides data to the algorithm without much intervention. With the rise of ethics in machine learning, predictive models need to be made fair. In this paper, we inspect the effects of pre-processing, in-processing and post-processing techniques for making predictive models fair. These techniques are applied to the hospital readmission prediction problem, where gender is considered as a sensitive attribute. The goal of the paper is to check whether unwanted discrimination between female and male in the logistic regression model exists and if exists to alleviate this problem making classifier fair. We employed logistic regression model which obtained AUC = 0.7959 and AUPRC = 0.5263. We have shown that reweighting strategy is a good trade-off between fairness and predictive performance. Namely, fairness is greatly improved, without much sacrificing predictive performance. We also show that adversarial debiasing is a good technique which combines predictive performance and fairness, and Equality of Odds technique optimizes Theil index.",
publisher = "Fac Organization And Informatics,  Univ Zagreb, Varazdin",
journal = "Central European Conference on Information and Intelligent Systems (CECIIS 2019)",
title = "Making hospital readmission classifier fair - What is the cost?",
pages = "331-325",
url = "conv_2761"
}

Radovanović, S., Petrović, A., Delibašić, B.,& Suknović, M.. (2019). Making hospital readmission classifier fair - What is the cost?. in Central European Conference on Information and Intelligent Systems (CECIIS 2019)
Fac Organization And Informatics,  Univ Zagreb, Varazdin., 325-331.
conv_2761

Radovanović S, Petrović A, Delibašić B, Suknović M. Making hospital readmission classifier fair - What is the cost?. in Central European Conference on Information and Intelligent Systems (CECIIS 2019). 2019;:325-331.
conv_2761 .

Radovanović, Sandro, Petrović, Andrija, Delibašić, Boris, Suknović, Milija, "Making hospital readmission classifier fair - What is the cost?" in Central European Conference on Information and Intelligent Systems (CECIIS 2019) (2019):325-331,
conv_2761 .

TY  - JOUR
AU  - Rakić, Mia
AU  - Monje, Alberto
AU  - Radovanović, Sandro
AU  - Petković-Ćurčin, Aleksandra
AU  - Vojvodić, Danilo
AU  - Tatić, Zoran
PY  - 2019
UR  - https://rfos.fon.bg.ac.rs/handle/123456789/2547
AB  - Background: Study objectives were 1) to estimate diagnostic capacity of clinical parameters, receptor-activator-nuclear factor kappa-B (RANKL) and osteoprotegerin (OPG) to diagnose healthy peri-implant condition (HI), peri-implant mucositis (PIM) and peri-implantitis (PIMP) by assessing respective diagnostic accuracy, sensitivity, specificity and diagnostic ranges 2) to develop personalized diagnostic model (PDM) for implant monitoring.
Methods: Split-mouth study included 126 patients and 252 implants (HI=126, PIM=57 and PI=69). RANKL and OPG concentrations were estimated in peri-implant crevicular fluid using ELISA method and assessed with clinical parameters using routine statistics, while the diagnostic capacity of individual parameters and overall clinical diagnosis were estimated using classifying algorithms. PDM was developed using decision trees.
Results: Bleeding on probing (BOP), plaque index and probing depth (PD) were confirmed reliable discriminants between peri-implant health and disease, while increase in PD (PD>4mm) and SUP were good discriminants amongst PIM/ PIMP. BTMs demonstrated presence of bone resorption in PIM, comparable diagnostic ranges between PIM/PIMP, PIMP was clinically distinguished from PIM in about 60% of patients while 40% remined diagnosed as false negatives. PDM demonstrated highest diagnostic capacity (accuracy: 96.27%, sensitivity: 95.00%, specificity: 100%) and defined HI: BOP0.25%; PIM: BOP>0.25%, PD4.5mm; PIMP: BOP>0.25%, PD>4.5mm and RANKL19.9 pg/site; PIM: BOP>0.25%, PD>4.5mm and RANKL>19.9 pg/site.
Conclusion: BTMs demonstrated capacity to substantially improve clinical diagnosis of peri-implant conditions. Considering lack of difference in BTMs between PIM/ PIMP and cluster of PIM with exceeding BTMs, a more refined definition of peri-implant conditions according to biological characteristics is required for further BTMs validation and appropriate PIMP management.
PB  - American Academy of Periodontology Journals
T2  - Journal of Periodontology
T1  - Is the Personalized Approach the Key to Improve Clinical Diagnosis of Peri-Implant Conditions? The Role of Bone Markers
EP  - 869
IS  - 7
SP  - 859
VL  - 91
DO  - 10.1002/JPER.19-0283
ER  - 

@article{
author = "Rakić, Mia and Monje, Alberto and Radovanović, Sandro and Petković-Ćurčin, Aleksandra and Vojvodić, Danilo and Tatić, Zoran",
year = "2019",
abstract = "Background: Study objectives were 1) to estimate diagnostic capacity of clinical parameters, receptor-activator-nuclear factor kappa-B (RANKL) and osteoprotegerin (OPG) to diagnose healthy peri-implant condition (HI), peri-implant mucositis (PIM) and peri-implantitis (PIMP) by assessing respective diagnostic accuracy, sensitivity, specificity and diagnostic ranges 2) to develop personalized diagnostic model (PDM) for implant monitoring.
Methods: Split-mouth study included 126 patients and 252 implants (HI=126, PIM=57 and PI=69). RANKL and OPG concentrations were estimated in peri-implant crevicular fluid using ELISA method and assessed with clinical parameters using routine statistics, while the diagnostic capacity of individual parameters and overall clinical diagnosis were estimated using classifying algorithms. PDM was developed using decision trees.
Results: Bleeding on probing (BOP), plaque index and probing depth (PD) were confirmed reliable discriminants between peri-implant health and disease, while increase in PD (PD>4mm) and SUP were good discriminants amongst PIM/ PIMP. BTMs demonstrated presence of bone resorption in PIM, comparable diagnostic ranges between PIM/PIMP, PIMP was clinically distinguished from PIM in about 60% of patients while 40% remined diagnosed as false negatives. PDM demonstrated highest diagnostic capacity (accuracy: 96.27%, sensitivity: 95.00%, specificity: 100%) and defined HI: BOP0.25%; PIM: BOP>0.25%, PD4.5mm; PIMP: BOP>0.25%, PD>4.5mm and RANKL19.9 pg/site; PIM: BOP>0.25%, PD>4.5mm and RANKL>19.9 pg/site.
Conclusion: BTMs demonstrated capacity to substantially improve clinical diagnosis of peri-implant conditions. Considering lack of difference in BTMs between PIM/ PIMP and cluster of PIM with exceeding BTMs, a more refined definition of peri-implant conditions according to biological characteristics is required for further BTMs validation and appropriate PIMP management.",
publisher = "American Academy of Periodontology Journals",
journal = "Journal of Periodontology",
title = "Is the Personalized Approach the Key to Improve Clinical Diagnosis of Peri-Implant Conditions? The Role of Bone Markers",
pages = "869-859",
number = "7",
volume = "91",
doi = "10.1002/JPER.19-0283"
}

Rakić, M., Monje, A., Radovanović, S., Petković-Ćurčin, A., Vojvodić, D.,& Tatić, Z.. (2019). Is the Personalized Approach the Key to Improve Clinical Diagnosis of Peri-Implant Conditions? The Role of Bone Markers. in Journal of Periodontology
American Academy of Periodontology Journals., 91(7), 859-869.
https://doi.org/10.1002/JPER.19-0283

Rakić M, Monje A, Radovanović S, Petković-Ćurčin A, Vojvodić D, Tatić Z. Is the Personalized Approach the Key to Improve Clinical Diagnosis of Peri-Implant Conditions? The Role of Bone Markers. in Journal of Periodontology. 2019;91(7):859-869.
doi:10.1002/JPER.19-0283 .

Rakić, Mia, Monje, Alberto, Radovanović, Sandro, Petković-Ćurčin, Aleksandra, Vojvodić, Danilo, Tatić, Zoran, "Is the Personalized Approach the Key to Improve Clinical Diagnosis of Peri-Implant Conditions? The Role of Bone Markers" in Journal of Periodontology, 91, no. 7 (2019):859-869,
https://doi.org/10.1002/JPER.19-0283 . .