ISSN 2308-4057 (Print),
ISSN 2310-9599 (Online)

A multi-criteria sensory assessment of Cucumis melo (L.) using fuzzy-Eckenrode and fuzzy-TOPSIS methods

Аннотация
The paper introduces a multi-criteria assessment system that can be used for sensory analysis by fuzzy-Eckenrode and fuzzy-TOPSIS methods. Respondents evaluated the sensory characteristics of Cucumis melo (L.), which included aroma, colour, taste, texture, and overall acceptance, after six days of storage. The product was stored under three different temperature conditions: 10°C (B1), 14°C (B2), and room temperature (27–30°C) (B3). The product was also stored at three types of packaging: unpackaged stem (A1), packaged fruit with one layer of banana stem (A2), and packaged fruit with two layers of banana stem (A3). The best result was demonstrated by the Cucumis melo that was stored at 14°C and packaged in a two-layered banana stem (A3B2). Both fuzzy-Eckenrode and fuzzy-TOPSIS method provided an easy, fast, and unambiguous calculation of multi-criteria sensory assessment.
Ключевые слова
Banana stem , hedonic scale , Cucumis melo (L.) , sensory assessment , TOPSIS , Eckenrode
ВВЕДЕНИЕ

Cucumis melo L. is a tropical and sub-tropical fruit that easily decays and rots because of its high-water content (70–95%). For the fruit to maintain its quality and freshness, it has to be handled properly during and after harvesting. A good quality fruit should be fresh, with a smooth, undamaged, and flawless skin. Compared to other cucumbers (Cucumis), Cucumis melo has a greener colour, more crunchy texture, higher water content, and sweeter taste. In addition, Cucumis melo can be harvested at an earlier stage.

Packaging is extremely important in post-harvest handling. It creates proper condition for the fruit to maintain its quality during the desired period. Packaging is a container or wrapper that can help to prevent or reduce damage to the packaged/wrapped object. The main functions of packaging are to keep food products from contamination, to protect them from physical damage, and to inhibit their quality degradation.

In the Province of Aceh (Indonesia), Cucumis melo is usually packaged in traditional manner by using banana stem, because banana leaves are cheap, easy to find, and eco-friendly. The fruit is placed in the middle part of banana stem, which are then folded into two parts (Fig. 1). Banana stem are able to protect the fruit from shocks and damage during transportation from producer to consumer. When ripe, the epidermis of Cucumis melo cracks, and banana stem help keep its shape and texture. Usually, Cucumis melo is protected with a single layer of banana stem.

Fig. 1. Cucumis melo packaged in banana stem.

According to Lukman [1], banana stem is part of banana pseudo stem [1]. Its structure is very different from that of woody plants, because it is an apparent trunk formed by tightly packed, over-leaping stem. The fibre of banana stem are strong and waterproof to both fresh and salt water. The packaging of Cucumis melo with a various amount of banana stem is necessary to preserve its wholeness and texture, because this fruit is easily broken when ripe. The storage temperature varies from room temperature to cold temperature, which is also expected to prolong the shelf life of Cucumis melo.

A quick method to find out consumer acceptance towards the food product is to perform a sensory assessment by collecting respondents’ opinions on the product. This multi-criteria assessment method was completed with a weighting assessment approach, which is usually used in decision making. Therefore, this article introduces a multi-criteria assessment system that performs a sensory analysis by using fuzzy-Eckenrode and the fuzzy-TOPSIS (Technique for Order Performance by Similarity to Ideal Solution) methods. According to the system, the respondents evaluated each product and rated its level of acceptance according to a multi-criteria sensory assessment, which included aroma, colour, taste, texture, and overall acceptance.

Fuzzy logic. Fuzzy logic is a development of the set theory, where each member has a degree of membership that ranges in value between 0 and 1. It means that fuzzy sets can represent interpretation of each value according to the opinion, or decision, and its probability. Rating 0 represents ‘wrong’, rating 1 represents ‘right’, and there are still other numbers between the ‘right’ and ‘wrong’ [2, 3].

In fuzzy sets, there are two attributes. The first one is linguistic attribute: it is a naming of a group which represents a certain situation or condition by using a natural language such as ‘cold’, ‘cool’, ‘normal’, or ‘warm’. The second attribute is numeric: it is a value (number) which shows a measure of a variable, such as 10, 30, 50, etc. [4]. Membership function is a curve that defines how each point in the input room is mapped into the membership value (degree of membership between 0 and 1). If U states universal sets and A is fuzzy function sets in U, so A can be stated as sorted pair as following [2]:

where μA(x) is a membership function that gives value of degree of membership x to fuzzy set A, which is:

In a fuzzy set, there are several membership functions of a new fuzzy set, which result from basic operation of the fuzzy set, i.e.:



Membership function is stated as follows:

In a fuzzy system, there is a linguistic variable. This is a variable that has a value in verbal form in a natural language. Each linguistic variable is related to a certain membership function. Figure 2 gives an example of membership function.

Fig. 2. Membership function.

Fuzzy-Eckenrode. The Eckenrode method was initially known as a weighting multiple criteria method, which was introduced by Robert T. Eckenrode from Dunlop and Association, Inc. in 1965 and has been widely used until today [5–8]. The Eckenrode method is simpler and more efficient in determining the importance weight in a decision [9–11]. The Eckenrode weighting analysis method is one of weighing methods used in determining the degree of importance, or Weight (B), from each Criteria (K) and Sub-criteria (SK), which have been set in decision making [12]. This weight determination is perceived as very important because it affects the final total value of each chosen decision. The concept used in this weighting method is by doing a change of order to value where, for instance, first order (1) has the highest rate (value) and the fifth order (5) has the lowest rate.

Fuzzy-TOPSIS. TOPSIS belongs to the Multiple Attribute Decision Making (MADM), which was firstly introduced by Yoon, Yoon et al. and Hwang et al. [13–15]. It has been widely applied in various studies related to decision making, such as Kumar et al., Han et al., Tyagi, Estrella et. al., Roszkowska et al., Selim et al. [16–21]. TOPSIS can only be implemented for a criterion whose weight has been known or calculated before, because there is a step in TOPSIS which involves the process of multiplication of criterion weight and the alternative value of the criterion.

In many situations, the data available is insufficient for a real life problem, because human assessment, which is considered as preference, is unclear, and the preference cannot be estimated with exact numeric value. The verbal expression, e.g. ‘low’, ‘medium’, ‘high’, etc., is considered as a representation of the decision maker. Thus, fuzzy logic is necessary in making a structured decision of the preference maker. The Fuzzy theory helps to measure the uncertainty associated with human judgement, which is subjective. Therefore, evaluation is necessary to be done in an environment. According to Ningrum et al. and Fadhil et al., fuzzy logic can help improve failure, which happens when only Eckenrode or TOPSIS method is used [4, 22].

ОБЪЕКТЫ И МЕТОДЫ ИССЛЕДОВАНИЯ

This study used Cucumis melo (L.) which was harvested in two months after planting. The harvested Cucumis melo was cleaned by washing and then stored under three different conditions: without banana stem packaging (A1), with one layer of banana stem packaging (A2), and with two layers of banana stem packaging (A3). Cucumis melo was then stored for six days under three temperature regimes: 10°C (B1), 14°C (B2), and at room temperature (27–30°C) (B3).

Procedure of assessment. The multi-criteria sensory assessment of Cucumis melo included aroma, colour, taste, texture, and overall acceptance (Table 1). The attribute weight of respondents’ assessment toward the multi-criteria was determined according to the hedonic scale. The hedonic scale is a preference of respondent’s opinion based on likes or dislikes that are converted into number (Table 2).

Table 1. Attributes of multi-criteria sensory assessment of Cucumis melo
Table 2. Assessment of preference according to hedonic scale

The framework of this study included four steps: (1) selection of respondents and criteria, (2) determination of criterion weight of the assessment by using the fuzzy-Eckenrode method, (3) determination of the best alternative of all treatments by using fuzzy-TOPSIS, and (4) recommendation of the best acceptance from all respondents. Figure 3 shows the complete framework.

Fig. 3. Research framework.

Combinations of storage conditions were as follows:
A1B1: without banana stem-packaging at 10°C;
A1B2: without banana stem-packaging at 14°C;
A1B3: without banana stem-packaging at 27–30°C;
A2B1: with one layer of banana stem-packaging at 10°C;
A2B2: with one layer of banana stem-packaging at 14°C;
A2B3: with one layer of banana stem-packaging at 27–30°C;
A3B1: with two layers of banana stem-packaging at 10°C;
A3B2: with two layers of banana stem-packaging at 14°C;
A3B3: with two layers of banana stem-packaging at 27–30°C.

Fuzzy-Eckenrode method. According to the Eckenrode weight calculation method, the respondents were asked to make a rating (e.g. from R1 until Rn, where n r anking, j = 1 , 2 , 3 ,…, n , r anking j = Rj) for each criterion (criterion i is notated with Ki, which is presented in a number of n criteria, i = 1, 2, 3,…, n) [11]. Table 3 shows the obtained data. Next, Ni was calculated based on Pij and Rn-j.


Table 3. Calculation of criterion weight according to the Eckenrode method

Then, criterion weight Bi (which are B1, B2, B3,…, Bn) was calculated, where i = 1, 2, 3,…, 3, by using the following formula:

To find the level of importance of each sub-criterion within a criterion, the respondents were also asked to rank each sub-criterion within a criterion. Then, by using the same procedure, the weight of each subcriterion was calculated (B1i, the weight of sub-criterion 1 in criterion i). Thus, the weighted weight (BT) from sub-criterion 1 in criterion i was obtained, which was BT1 = B1i·Bi. Then, to find the score of each criterion, the respondents were asked to rate each sub-criterion within each criterion [23].

The assessment of each sub-criterion was calculated by using a geometric mean formula according to the assessment result from all respondents, which was multiplied with the weighted weight of each subcriterion. Each criterion (K1 to K5) was calculated by summing up the total score of all sub-criteria in each criterion. To assess the weighting by the respondents, the fuzzy-Eckenrode method was applied with the value of preference, as shown in Table 4.

Table 4. Scale of weighting comparison among criteria of fuzzy-Eckenrode method

Fuzzy-TOPSIS method. The analysis with the fuzzy-TOPSIS method included the following tasks [24]:

To rank the fuzzy from each decision made, Dk; (k = 1, 2, 3,…, k) can be represented as triangular fuzzy number ˜Rk; (k = 1, 2, 3,…, K) with membership function μ˜R (x).

To produce an appropriate alternative, to determine the criteria of evaluation, and to organise the group of decision-maker. It was assumed that there were m alternatives, n criteria of evaluation, and decision k.

To choose the linguistic variable according to the weight of criterion importance = (wj = lij, mij, uij) and alternative linguistic rankings on criterion (wij) in Triangular Fuzzy Number (TFN).

To do a weight aggregation of each criterion to obtain fuzzy weight aggregate (wij) in criterion Cj and to determine the fuzzy aggregate value from alternative Ai on each criterion Cj.


To build a fuzzy decision matrix.

To do normalisation of the decision matrix, where:

Calculating [rij]mxn.

where U*j = max uij.

To determine the weight normalisation of the fuzzy decision matrix. Based on different importance on each criterion, the fuzzy decision of the weighted normalisation matrix can be arranged as:

where:

To determine fuzzy positive ideal solution (FPIS) S+ and fuzzy negative ideal solution (FNIS) S:


where: v+1 = max {vij3} and v-1 = min {vij1} with vj are TFN normalisation weight.

To calculate the interval between each alternative value and the value of FPIS (Fuzzy Positive Ideal Solution) and FNIS (fuzzy negative ideal solution).

To calculate the closeness coefficient (CCi) and the ranking according to the coefficient value obtained using the following equation:

To rate each alternative by the respondent, we used the fuzzy-TOPSIS method with preference value, as in Table 5.

Table 5. Comparison scale of determination of the fuzzy-TOPSIS method alternative

Figure 4 illustrates the procedure of the analysis.

Fig. 4. Steps of the fuzzy-TOPSIS method analysis.

Selection of respondents. A total of 10 respondents were chosen to do a multi-criteria sensory assessment of Cucumis melo. The respondents were selected according to several criteria. The potential respondents had to:

1. like Cucumis melo, raw or processed;
2. be experienced in sensory assessment;
3. be healthy, as flu, cough, mouth ulcers, etc. can bother the sensory assessment process;
4. be able to distinguish colours.

РЕЗУЛЬТАТЫ И ИХ ОБСУЖДЕНИЕ

Determination of assessment criteria weight. A hedonic scale was used to evaluate the results of determination of respondents’ assessment of criteria weight towards multi-criteria which were considered in the sensory assessment. After that, they were translated into fuzzy logic functions (Table 6).

Table 6. Respondents’ weighting score of criteria based on the fuzzy-Eckenrode method

As for the data of respondents’ assessment towards criteria of importance weight determination from each sensory attribute, the values of lower bound (low), middle (medium), and upper bound (upper) were arranged as summarised on Table 7. The next step was to calculate the score and the weight of each criterion. Figure 5 represents a radar diagram.

Table 7. TFN value of experts’ weighting on criteria of the fuzzy-Eckenrode method
Fig. 5. Radar diagram of criteria weight.

According to the respondents’ assessment of the criteria with the help of the fuzzy-Eckenrode method, the order of criteria weight was obtained from the highest to the lowest: (1) overall acceptance, 0.216; (2) colour, 0.211; (3) aroma, 0.203; (4) taste, 0.191; and (5) texture 0.176.

Determination of the best alternative. The priority of the best alternative from the multi-criteria sensory assessment of Cucumis melo was determined by summarising all respondents’ preferences. The preferences were chosen based on the mode number, i.e. the value that appears most often from each choice of material treatment. The mode number was chosen by the respondents. The next step was to arrange the matrix of the respondents’ assessment on all alternatives (Table 8). The data of respondents’ assessment was then transformed into TFN linguistic data, as presented in Table 9.

Table 8. Matrix of experts’ assessment on alternatives
Table 9. Matrix of respondents’ assessment on alternative in TFN scale

After that, we formulated the normalised weight matrix on each alternative. The value normalisation can be done by using Eqs. (13) and (14). Table 10 shows the results of the TFN value normalisation.

Table 10. Matrix of TFN scale normalisation

Then, we arranged the matrix of multiplication between criteria weights and normalisation value of each alternative. This process can be done by using Eqs. (15) and (16). Table 11 summarises the results of the matrix multiplication.

Table 11. Matrix of multiplication of criteria weights and alternative normalisation values

The next step was to determine the positive ideal solution value (FPIS) S+ and the negative ideal solution value (FNIS) S. When determining both values, the characteristic of data available should be taken into consideration. To obtain both groups of values, one can use Eqs. (17) and (18). Table 12 demonstrates FPIS and FNIS values.

Table 12. Positive ideal solution and negative ideal solution values

After that, the interval between each alternative value and FPIS and FNIS was calculated by using Eqs. (19), (20), and (21). The results of the interval calculation between alternative value toward FPIS and FNIS can be observed from Table 13 and Table 14.

Table 13. Intervals between criteria value and FPIS
Table 14. Interval between criteria value and FNIS

We evaluated the criteria distance value to the fuzzy positive ideal solution (FPIS) and the fuzzy negative ideal solution (FNIS) according to comparison of d+ and d–. It showed preference of product acceptance on a radar diagram (Fig. 6). For instance, the treatment of Cucumis melo without packaging at temperature of 10°C (A1B1) had such d+ and d– values that showed the biggest distance from the positive ideal and the negative ideal.

Fig. 6. Evaluation of d+ and d–.

The final step was to calculate the closeness coefficient (CCi) of each alternative by using Eq. (22). From the calculation result, we obtained ranking from the highest to the lowest (Fig. 7). The biggest coefficient value was the main alternative, which was suggested to be chosen or prioritised, compared to other alternatives based on respondents’ preference (product acceptance).

Fig. 7. Alternative ranking of Cucumis melo product acceptance.

According to the closeness coefficient (CCi), an alternative ranking can be arranged from the biggest to the lowest as follows: two-layer banana stem-packaging at 14°C (A3B2), two-layer banana stem-packaging at 10°C (A3B1), one-layer banana stem-packaging at 14°C (A2B2), one-layer banana stem-packaging at 10°C (A2B1), without banana stem packaging at 14°C (A1B2), one-layer banana stem-packaging at room temperature (A2B3), without banana stem packaging at 10°C (A1B1), without banana stem packaging at room temperature (A1B3), and two-layer banana stem-packaging at room temperature (A3B3) (Fig. 7).

The analysis with fuzzy-TOPSIS approach showed that the respondents preferred Cucumis melo stored in a two-layer banana stem packaging at 14°C (A3B2). Since the scores were fairly close between Cucumis melo stored in a two-layer banana stem packaging at 14°C (A3B2) and Cucumis melo stored in a two-layer banana stem packaging at 10°C (A3B1), both products were favored by consumers (respondents’ preferences).

ВЫВОДЫ

According to the consumer assessment of all types of the six-day storage of Cucumis melo, the optimal storage conditions involved packaging with two layers of banana stem at the temperature of 14°C (A3B2). The fuzzy-Eckenrode and fuzzy-TOPSIS methods were very helpful in calculating the results of the multi-criteria sensory assessment through weighing. They made the process of determining consumers’ acceptance easier, faster, and more certain.

КОНФЛИКТ ИНТЕРЕСОВ

The authors declare no conflict of interest.

БЛАГОДАРНОСТИ

We thank our laboratory assistants at the Post-Harvest Engineering Laboratory, Faculty of Agriculture, Syiah Kuala University, especially Riza Rahmah, for their support and assistance in organising respondents and material preparation for this research.

СПИСОК ЛИТЕРАТУРЫ
  1. Lukman AJ. Pembuatan pulp dari pelepah pisang dengan alat digester [Making of pulp from Musa paradiciasa with a digester]. Semarang: Universitas Diponegoro; 2011. 34 p. (In Indonesian).
  2. Wang L-X. A course in Fuzzy System Control. New Jersey: Prentice-Hall; 1997. 448 p.
  3. Kusumadewi S, Purnomo H. Aplikasi Logika Fuzzy, Untuk Pendukung Keputusan [Fuzzy Logics Application for Decision Making]. Yogyakarta: Graha Ilmu; 2010. (In Indonesian).
  4. Fadhil R, Maarif MS, Bantacut T, Hermawan A. Sistem Penunjang Keputusan Multi Kriteria Untuk Pengembangan Agroindustri Kopi Gayo Menggunakan Pendekatan Fuzzy-Eckenrode dan Fuzzy-TOPSIS [A Multi-Criteria Decision Support System for the Development of Gayo Coffee Agroindustry by Using Fuzzy-Eckenrode and Fuzzy-TOPSIS Approaches]. Jurnal Teknologi Industri Pertanian [Journal of Agroindustrial Technology]. 2017;27(4):103–113. DOI: http://doi.org/10.24961/j.tek.ind.pert.2017.27.1.103. (In Indonesian).
  5. Rahmawati S, Kartika L, Syamsun M, Sayekti A. Strategi Peningkatan Daya Saing UKM Sektor Sapi Perah di Indonesia [The Strategy to Increase the Competitiveness of SMEs in the Dairy Cattle Sector in Indonesia]. Jurnal Aplikasi Manajemen [Journal of Apllication Management]. 2016;14(4):735–746. DOI: http://doi.org/10.18202/jam23026332.14.4.14. (In Indonesian).
  6. Rijai L. Penentuan kriteria ilmiah potensi tumbuhan obat unggulan [Determination of the scientific criteria of potential superior medicinal plants]. Journal of Tropical Pharmacy and Chemistry. 2011;1(2):125–133. DOI: https://doi.org/10.25026/jtpc.v1i2.20. (In Indonesian).
  7. Kao C. Weight determination for consistently ranking alternatives in multiple criteria decision analysis. Applied Mathematical Modelling. 2010;34(7):1779–1787. DOI: https://doi.org/10.1016/j.apm.2009.09.022.
  8. Thompson RL, Iacovou CL, Shirland LE. A Comparison of Evaluation Techniques for Decision Analysis Involving Large Attribute Sets. Journal of Applied Business Research. 2006;22(2):25–38. DOI: https://doi.org/10.19030/jabr.v22i2.1432.
  9. Dadelo S, Turskis Z, Zavadskas EK, Dadeliene R. Multiple Criteria assessment of elite security personal on the basis of ARAS and expert methods. Economic Computation and Economic Cybernetics Studies and Research. 2012;46(4):65–87.
  10. Eckenrode RT. Weighting Multiple Criteria. Management Science. 1965;12(3):180–192. DOI: https://doi.org/10.1287/mnsc.12.3.180.
  11. Salam AR. Analisis Penentuan Pelabuhan Impor Produk Hortikultura: Aplikasi Metode Eckenrode [Analysis of Determination of Port Criteria for Imported Horticultural Products: Application of Eckenrode Method]. Buletin Ilmiah Litbang Perdagangan [Scientific Bulletin of Commerce R&D]. 2014;8(1):1–24. DOI: http://doi.org/10.30908/bilp.v8i1.83. (In Indonesian).
  12. Maarif MS, Tanjung H. Teknik-Teknik Kuantitatif untuk Manajemen [Quantitative techniques for management]. Jakarta: Gasindo; 2003. 188 p. (In Indonesian).
  13. Yoon K. System Selection by Multiple Attribute Decision Making. Cand. eng. sci. diss. Kansas: Kansas State University; 1980. 238 p.
  14. Yoon K., Hwang CL. TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) – A Multiple Attribute Decision Making. 1980.
  15. Hwang CL, Yoon K. Multiple Attributes Decision Making Methods and Applications. Heidelberg: Springer-Verlag; 1981. 269 p. DOI: htts://doi.org/10.1007/978-3-642-48318-9.
  16. Kumar K, Garg H. TOPSIS method based on the connection number of set pair analysis under interval-valued intuitionistic fuzzy set environment. Computational and Applied Mathematics. 2018;37(2):1319–1329. DOI: https://doi.org/10.1007/s40314-016-0402-0.
  17. Han H, Trimi S. A fuzzy TOPSIS method for performance evaluation of reverse logistics in social commerce platforms. Expert Systems with Applications. 2018;103:133–145. DOI: https://doi.org/10.1016/j.eswa.2018.03.003.
  18. Tyagi SK. Making selection using multiple attribute decision-making with intuitionistic fuzzy sets. International Journal of Systems Science: Operations and Logistics. 2018;5(2):149–160. DOI: https://doi.org/10.1080/23302674.2016.1244300.
  19. Estrella FJ, Onar SC, Rodriguez RM, Oztaysi B, Martinez L, Kahraman C. Selecting firms in University technoparks: A hesitant linguistic fuzzy TOPSIS model for heterogeneous contexts. Journal of Intelligent & Fuzzy Systems.2017;33(2):1155–1172. DOI: https://doi.org/10.3233/JIFS-16727.
  20. Roszkowska E, Kacprzak D. The fuzzy saw and fuzzy TOPSIS procedures based on ordered fuzzy numbers. Information Sciences. 2016;369:564–584. DOI: https://doi.org/10.1016/j.ins.2016.07.044.
  21. Selim H, Yunusoglu MG, Balaman SY. A Dynamic Maintenance Planning Framework Based on Fuzzy TOPSIS and FMEA: Application in an International Food Company. Quality and Reliability Engineering International. 2016;32(3):795–804. DOI: https://doi.org/10.1002/qre.1791.
  22. Ningrum M, Sutarman S, Sitepu R. Aplikasi Metode TOPSIS Fuzzy dalam Menentukan Prioritas Kawasan Perumahan di Kecamatan Percut Sei Tuan [Application of fuzzy TOPSIS method in determining the priority of housing areas in Percut Sei Tuan district]. Saintia Matematika. 2013;1(1):101–115. (In Indonesian).
  23. Fadhil R, Maarif MS, Bantacut T, Hermawan A. Perbandingan teknik pengambilan keputusan multi-kriteria antara metode Eckenrode dengan metode fuzzy Eckenrode pada kinerja agroindustri (A comparison of multi-criteria decison making technique between Eckenrode and fuzzy-Eckenrode methods on agroindustry performance). Jurnal Manajemen & Agribisnis [Journal of Management & Agribusiness]. 2017;14(2):109–117. DOI: http://doi.org/10.17358/jma.14.2.109. (In Indonesia).
  24. Rouyendegh BD, Saputro TE. Supplier selection using integrated fuzzy TOPSIS and MCGP: a case study. Procedia – Social and Behavioral Sciences. 2014;116:3957–3970. DOI: https://doi.org/10.1016/j.sbspro.2014.01.874.
Как цитировать?
A multi-criteria sensory assessment of Cucumis melo (L.) using fuzzy-Eckenrode and fuzzy-TOPSIS methods. Foods and Raw Materials, 2019, vol. 7, no. 2, pp. 339-347
DOI
http://doi.org/10.21603/2308-4057-2019-2-339-347
Издатель
Кемеровский государственный университет
htpps://kemsu.ru
ISSN
2308-4057 (Print) /
2310-9599 (Online)
О журнале