DEEP LEARNING NETWORK-BASED

EVALUATION METHOD OF ONLINE

TEACHING QUALITY OF INTERNATIONAL

CHINESE EDUCATION

Wenling Lai*

Tongshi College of Quality Education, Wuchang University of Technology, Wuhan,

Hubei, 430223, China

lwl001325@126.com

Reception: 29/10/2022 Acceptance: 01/01/2023 Publication: 28/01/2023

Suggested citation:

L., Wenling (2023). Deep Learning Network-Based Evaluation method of

Online teaching quality of International Chinese Education. 3C Tecnología.

Glosas de innovación aplicada a la pyme, 12(1), 87-106. https://doi.org/

10.17993/3ctecno.2023.v12n1e43.87-106

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

87

ABSTRACT

The development of vocational education in the information age requires us to think

about the path and strategy of active change. Course teaching quality evaluation

should also shift from passive evaluation of online teaching development to active

construction of a mixed teaching quality evaluation system. In the information age, the

development of teaching resources is dizzying. From paper to digital, from single to

diverse, from offline to online, from scarcity to mass—various changes impact the

traditional teaching model. Aiming at the online teaching quality evaluation of

international Chinese education on the Internet, this paper proposes a method based

on deep learning. Firstly, this paper proposes an index system construction and

evaluation index weighting for online teaching of international Chinese education, and

collects online data as a corpus at the same time. Then construct the

CNN_BiLSTM_Att model, which is composed of the CNN module, the BiLSTM

module and the Att module. Finally, compare with other model experiments. The

experimental results show that CNN_BiLSTM_Att has achieved the best results in the

evaluation index results, with P and F1 reaching 97.89% and 97.85%. Compared with

other models, the overall effect is improved by 2%~5%. From this, the superiority of

the model in the online teaching quality evaluation standard task of this paper can be

obtained.

KEYWORDS

Deep learning technology; teaching evaluation; international Chinese education;

online teaching

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

88

ABSTRACT

The development of vocational education in the information age requires us to think

about the path and strategy of active change. Course teaching quality evaluation

should also shift from passive evaluation of online teaching development to active

construction of a mixed teaching quality evaluation system. In the information age, the

development of teaching resources is dizzying. From paper to digital, from single to

diverse, from offline to online, from scarcity to mass—various changes impact the

traditional teaching model. Aiming at the online teaching quality evaluation of

international Chinese education on the Internet, this paper proposes a method based

on deep learning. Firstly, this paper proposes an index system construction and

evaluation index weighting for online teaching of international Chinese education, and

collects online data as a corpus at the same time. Then construct the

CNN_BiLSTM_Att model, which is composed of the CNN module, the BiLSTM

module and the Att module. Finally, compare with other model experiments. The

experimental results show that CNN_BiLSTM_Att has achieved the best results in the

evaluation index results, with P and F1 reaching 97.89% and 97.85%. Compared with

other models, the overall effect is improved by 2%~5%. From this, the superiority of

the model in the online teaching quality evaluation standard task of this paper can be

obtained.

KEYWORDS

Deep learning technology; teaching evaluation; international Chinese education;

online teaching

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

PAPER INDEX

ABSTRACT

KEYWORDS

1. INTRODUCTION

2. METHODOLOGY

2.1. Construction of online teaching quality evaluation index system

2.2. Weighting of evaluation indicators based on analytic hierarchy process

2.3. CNN_BiLSTM model

2.4. Word vector representation layer

2.5. CNN network

2.6. BiLSTM neural network layers

2.7. Attention mechanism

2.8. Output layer and loss function

3. ACTUAL CASE ANALYSIS AND VERIFICATION

3.1. Data description

3.2. Model prediction and evaluation index

3.3. Comparison of prediction results between different models

4. CONCLUSION

CONFLICT OF INTEREST

REFERENCES

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

89

1. INTRODUCTION

Data in the teaching and learning process, deep learning technology has an impact

on various elements of the education system in the analysis stage, strategy selection

stage and evaluation stage of instructional design. Meet the teaching needs of

teachers who need a lot of investigation and not enough experience guidance, and

provide guidance for "learner-centered" instructional design [1-3].

(1) The analysis phase of instructional design: Conduct a learning needs analysis.

Through the deep learning technology to analyze the student’s online learning

behavior, it is possible to accurately grasp the students' mastery of knowledge.

Learner characteristic analysis, the application of deep learning can help teachers

determine the cognitive development level of learners and analyze the starting ability

of learners[4].

(2) Strategy selection phase: Deep learning technology has changed the traditional

teaching environment of teachers and courseware, and formed an intelligent teaching

and learning environment that can analyze intelligently and assist students'

personalized learning [5-7].

(3) Teaching evaluation stage: Teacher and student evaluation, based on deep

learning data analysis technology, makes the evaluation of students' learning results

more scientific and accurate, reduces the difficulty of analysis, shortens the feedback

cycle, and enables teachers to have a more accurate grasp of students' learning

status. The artificial intelligence technology based on deep learning assists teachers

in supervising and evaluating students' learning behavior, reducing the burden of

teachers' teaching management, and the intelligent marking system helps teachers

reduce repetitive work [8].

International Chinese education is not only a discipline, but also a "national and

national cause", with the dual attributes of discipline and career. As a discipline, since

its inception, international Chinese education has been "growing up with the progress

of the Republic" [9]. Encourage teaching institutions, teachers, students and resource

builders to adjust, change and innovate accordingly. Especially the sudden epidemic

has exacerbated this change. How to effectively construct resources to deal with the

new online teaching quality evaluation method is a problem we need to solve.

Online Chinese teaching for international students provides conditions for liberating

teachers, innovating the research of international Chinese education, solving old

problems, discovering new problems, exploring new laws, and enriching the content of

subject research. It is necessary to make full use of the opportunity of international

Chinese online teaching, data, and teachers’ division of labor and cooperation models

brought by online teaching. On the other hand, it is necessary to start from promoting

the upgrading of international Chinese education and international Chinese education,

improve online Chinese teaching from the aspects of system, model, platform

construction, etc., and lead the development of online Chinese teaching [10-11].

Under traditional conditions [12], the evaluation data is obtained by means of a

questionnaire survey, and then the results are finally obtained through tedious sorting

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

90

1. INTRODUCTION

Data in the teaching and learning process, deep learning technology has an impact

on various elements of the education system in the analysis stage, strategy selection

stage and evaluation stage of instructional design. Meet the teaching needs of

teachers who need a lot of investigation and not enough experience guidance, and

provide guidance for "learner-centered" instructional design [1-3].

(1) The analysis phase of instructional design: Conduct a learning needs analysis.

Through the deep learning technology to analyze the student’s online learning

behavior, it is possible to accurately grasp the students' mastery of knowledge.

Learner characteristic analysis, the application of deep learning can help teachers

determine the cognitive development level of learners and analyze the starting ability

of learners[4].

(2) Strategy selection phase: Deep learning technology has changed the traditional

teaching environment of teachers and courseware, and formed an intelligent teaching

and learning environment that can analyze intelligently and assist students'

personalized learning [5-7].

(3) Teaching evaluation stage: Teacher and student evaluation, based on deep

learning data analysis technology, makes the evaluation of students' learning results

more scientific and accurate, reduces the difficulty of analysis, shortens the feedback

cycle, and enables teachers to have a more accurate grasp of students' learning

status. The artificial intelligence technology based on deep learning assists teachers

in supervising and evaluating students' learning behavior, reducing the burden of

teachers' teaching management, and the intelligent marking system helps teachers

reduce repetitive work [8].

International Chinese education is not only a discipline, but also a "national and

national cause", with the dual attributes of discipline and career. As a discipline, since

its inception, international Chinese education has been "growing up with the progress

of the Republic" [9]. Encourage teaching institutions, teachers, students and resource

builders to adjust, change and innovate accordingly. Especially the sudden epidemic

has exacerbated this change. How to effectively construct resources to deal with the

new online teaching quality evaluation method is a problem we need to solve.

Online Chinese teaching for international students provides conditions for liberating

teachers, innovating the research of international Chinese education, solving old

problems, discovering new problems, exploring new laws, and enriching the content of

subject research. It is necessary to make full use of the opportunity of international

Chinese online teaching, data, and teachers’ division of labor and cooperation models

brought by online teaching. On the other hand, it is necessary to start from promoting

the upgrading of international Chinese education and international Chinese education,

improve online Chinese teaching from the aspects of system, model, platform

construction, etc., and lead the development of online Chinese teaching [10-11].

Under traditional conditions [12], the evaluation data is obtained by means of a

questionnaire survey, and then the results are finally obtained through tedious sorting

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

work. This method requires a lot of time and material resources, and there are

problems such as inaccurate data collected in special questionnaires. Using intelligent

technical means to obtain real-time online course evaluation data and analyze it, the

results obtained by using deep neural network learning technology have high

accuracy, and the feedback results are used as the basis for teachers to change their

learning methods.

Therefore, this paper proposes a method based on deep learning. The main work

is highlighted as follows:

(1) The evaluation standard system is constructed for the online teaching quality

evaluation method of international Chinese education on the Internet.

(2) Design a set of analytical methods to weight the evaluation indicators.

(3) The CNN_BiLSTM_Att network framework is built to realize the online teaching

quality evaluation of international Chinese education [13-14].

2. METHODOLOGY

2.1. CONSTRUCTION OF ONLINE TEACHING QUALITY

EVALUATION INDEX SYSTEM

Online teaching quality evaluation system is an important foundation and guarantee

for online teaching quality, is shown in Fig 1. In order to ensure that the online

teaching quality evaluation index system plays the functions of supervision and

incentives, it is required to follow the objectivity, purpose, consistency,

comprehensiveness and operability of the indicators [15].

(1) The principle of objectivity is important guarantee for the effectiveness and

credibility of online teaching quality evaluation results, and requires scientific

formulation. The process of evaluation criteria should be open and transparent, and

the evaluation results should be collected to ensure the reliability of online teaching

quality evaluation data.

(2) This paper is to improve the level of teaching evaluation quality, stimulate

teachers' enthusiasm for teaching.

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

91

Figure 1. Online teaching quality evaluation index system

(3) The principle of consistency refers to scientific, standardized and systematic

selection of online quality evaluation indicators, requires evaluation indicators to be

based on the laws of online teaching at different ages.

(4) The principle of comprehensiveness requires that online teaching quality

evaluation indicators should cover various cannot be limited to certain aspects and

certain indicators.

(5) The principle of operability requires system is practical and feasible, and can be

carried out continuously and effectively on a large scale.

2.2. WEIGHTING OF EVALUATION INDICATORS BASED ON

ANALYTIC HIERARCHY PROCESS

Online teaching quality evaluation index system based on the questionnaire survey

method and the expert online consultation method based on the analytic hierarchy

process. Colleges and universities at different levels need to consider many factors

such as online course settings, teaching methods, and student source quality when

building an online teaching quality evaluation system [16-18].

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

92

Figure 1. Online teaching quality evaluation index system

(3) The principle of consistency refers to scientific, standardized and systematic

selection of online quality evaluation indicators, requires evaluation indicators to be

based on the laws of online teaching at different ages.

(4) The principle of comprehensiveness requires that online teaching quality

evaluation indicators should cover various cannot be limited to certain aspects and

certain indicators.

(5) The principle of operability requires system is practical and feasible, and can be

carried out continuously and effectively on a large scale.

2.2. WEIGHTING OF EVALUATION INDICATORS BASED ON

ANALYTIC HIERARCHY PROCESS

Online teaching quality evaluation index system based on the questionnaire survey

method and the expert online consultation method based on the analytic hierarchy

process. Colleges and universities at different levels need to consider many factors

such as online course settings, teaching methods, and student source quality when

building an online teaching quality evaluation system [16-18].

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

Table 1. Weights of online teaching quality evaluation indicators

(1) According to the index system of online quality evaluation, target evaluation

layer online teaching quality evaluation, and the criterion evaluation layer includes the

teaching management department, the evaluation of teaching supervision, and

students, the evaluation of the ministry of international Chinese education and the

self-evaluation of Chinese teachers. The scheme evaluation layer is the evaluation

index of each evaluation subject, and a hierarchical structure evaluation model is

constructed.

(2) Constructing a comparative judgment matrix by conducting online special

consultation and analysis on the relative importance of the pairwise comparison

between the evaluation subjects and the evaluation indicators to 10 experts;

(3) Carry out the consistency test, the results show that each judgment matrix CR <

0.1, all passed the test;

(4) The weight of the quality evaluation index, calculation results is shown in Table

1.

2.3. CNN_BILSTM MODEL

Online teaching quality evaluation model for international Chinese education is

constructed convolutional neural network (CNN) and bidirectional long short-term

memory network (BiLSTM)-based. The attention mechanism is introduced on the

basis of BiLSTM and CNN, and structure is shown in Fig 2.

Evaluation subject Weights Evaluation indicators Weights Weights ratio

Teaching

management

department

21. 61%

teacher behavior 50% 10. 805%

student behavior 50% 10. 805%

Teaching

supervision 18. 31%

teaching process 48. 5% 8. 88%

teaching effect 26. 67% 4. 88%

instructional Design 15. 27% 2. 8%

education resources 9. 56% 1. 75%

Student 34.82%

teaching method 45. 35% 15. 79%

teaching content 24. 2% 8. 43%

teaching effect 19. 72% 6. 87%

teaching attitude 10. 72% 3. 73%

Ministry of

international

Chinese education

14. 32%

teaching materials 61. 44% 8. 88%

assessment link 26. 84% 3. 84%

teaching routine 11. 72% 1. 68%

Chinese teacher 10. 95%

teaching readiness 60% 6. 57%

teaching implementation 20% 2. 19%

teaching feedback 20% 2. 19%

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

93

Figure 2. CNN_BiLSTM model

2.4. WORD VECTOR REPRESENTATION LAYER

After the text data is preprocessed, it is vectorized and then input into the online

teaching quality evaluation model for the next step [19]. The corpus data size is about

6.3GB, and the Word2Vec tool is used for large-scale text training to convert the text

into a low-density vector space. This paper selects the Word2Vec tool skip-gram

model to training word-vectors. The skip-gram model takes all the words in the corpus

as the central word, and predicts the lexical information of its context through the

conditional probability distribution of the correspondence between the central word

and the context. The expression formula is as follows:

where ,

is the context, that is, the representation

vector of the surrounding words;

is the center word; word-vectors obtained by

(1)

P

(Wi|Wt)=

P(W

i

|W

t

)

P(Wt)

P(Wi

)

i=t−1,t−2,t+1,t+2

Wi

Wt

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

94

Figure 2. CNN_BiLSTM model

2.4. WORD VECTOR REPRESENTATION LAYER

After the text data is preprocessed, it is vectorized and then input into the online

teaching quality evaluation model for the next step [19]. The corpus data size is about

6.3GB, and the Word2Vec tool is used for large-scale text training to convert the text

into a low-density vector space. This paper selects the Word2Vec tool skip-gram

model to training word-vectors. The skip-gram model takes all the words in the corpus

as the central word, and predicts the lexical information of its context through the

conditional probability distribution of the correspondence between the central word

and the context. The expression formula is as follows:

where , is the context, that is, the representation

vector of the surrounding words; is the center word; word-vectors obtained by

(1)

P(Wi|Wt)=P(Wi|Wt)

P(Wt)P(Wi)

i=t−1,t−2,t+1,t+2

Wi

Wt

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

training as

, where n is the total number of text words. The word

embedding layer converts words into

. The training parameters of the

Word2Vec is shown in Table 2.

Table 2. Word2Vec model parameters

2.5. CNN NETWORK

As a special type of forward neural network, CNN is used in the field of natural

language processing (NLP) in recent years [20]

. The basic structure is divided into

three parts, input layer, convolutional layer and pooling layer, and fully connected

layer, as shown in Fig 3

. The features extracted by the convolution layer first

represent the text in the form of word vector matrix, and then scan the matrix through

convolution kernels of different sizes. During the scanning process, the parameter

values of the filters composed of the convolution kernels are fixed. After filtering new

feature map is mapped, and all elements on the feature map come from filters with

consistent parameters.

(1) Enter the evaluation index text input sequence

, the

pre-trained word vector is

. The convolution formula for the number of words h in

each window is:

where, is the convolution result; ReLU is the nonlinear activation function; i is the

number of words taken per convolution.

(2) The text sequence is n, and the window length is n-h+1. Result formula is:

(3) Then, the pooling operation is performed on the result of the convolutional

layer according to the pooling layer, and the output sequence features the parameters

and calculation next layer to prevent over-fitting.

[W1,W2,...,Wn]

[x1,x2,...,xn]

parameter value Parameter meaning

sg 1 training model selection Skip-gram

window 5 window size

min_count 3 Minimum number of occurrences of a

word

vector_size 350 vector dimension

epoch 8 number of iterations

hs 0 negative sampling

negative 6 number of negative samples

S= {x1,x2,...,xn}

Rd

(2)

ci=ReLU(ωwi:i+h−1)+b

ci

(3)

C=[C1,C2,…,Cn−h+1]

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

95

(4) Finally, the CNN model convolves the contextual semantic content of the

word window to better represent the local features of the text sequence.

Figure 3. CNN network structure example

2.6. BILSTM NEURAL NETWORK LAYERS

LSTM is a special recurrent neural network (RNN) composed of one cell unit and

three gates, is shown in Fig 4

. The cell unit is the core computing power and records

the current computing state. Forget gates, input gates, and output gates regulate the

flow of information to and from memory cells. The forget gate clears the memory cells

of useless information. The input gate selects the input information of the current

memory cell. The output gate determines the final output of the information, so that

the storage unit can effectively store the semantic information of a longer sequence.

Figure 4. LSTM network structure example

C

t

δ tanhδ δ

tanh

h

t - 1

h

t

C

t - 1

h

t

X

t

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

96

(4) Finally, the CNN model convolves the contextual semantic content of the

word window to better represent the local features of the text sequence.

Figure 3. CNN network structure example

2.6. BILSTM NEURAL NETWORK LAYERS

LSTM is a special recurrent neural network (RNN) composed of one cell unit and

three gates, is shown in Fig 4. The cell unit is the core computing power and records

the current computing state. Forget gates, input gates, and output gates regulate the

flow of information to and from memory cells. The forget gate clears the memory cells

of useless information. The input gate selects the input information of the current

memory cell. The output gate determines the final output of the information, so that

the storage unit can effectively store the semantic information of a longer sequence.

Figure 4. LSTM network structure example

C

t

δ tanhδ δ

tanh

h

t - 1

h

t

C

t - 1

h

t

X

t

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

(1) The LSTM unit calculation process is as follows:

where is the input text vector;

is the sigmoid function and tanh is the activation

function; are the input gate, output gate and forget gate, respectively.

(2) The information unit stored at time t is

, where the input gate and forget

gate are not used to adjust the information unit.

where,

are the weights of different gate control mechanisms

on the input text vector ;

are the weights of different gate

control mechanisms on the hidden layer vector ;

are bias

vectors.

(3) Then, the unit information input gate at the previous moment is stored in .

where, the hidden-layer is the output gate and the storage gate .

(4) Finally, the forward and backward outputs of the LSTM unit at time t are

concatenated by using and building the BiLSTM network layer is shown in Fig 5.

where n represents the vector set.

(4)

(5)

(6)

ot=δ(Woxt+Uoht−1+bo)

ft=δ(Wfxt+Ufht−1+bf)

it=δ(Wixt+Uiht−1+bi)

xt

δ

it,ot,ft

ct

(7)

ct= tanh(Wcxt+Ucht−1+bc)

Wi,Wo,WfandWc

xt

Ui,Uo,UfandUc

ht−1

bi,bo,bfandbc

ct

(8)

(9)

ht=ottanh(ct)

ct=ftct−1+it~

ct

ht

ct

(10)

ht= [→

ht,←

ht]∈Rn

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

97

Figure 5. BiLSTM network structure example

2.7. ATTENTION MECHANISM

Although BiLSTM neural network can establish context-related semantic vector

information, it does not highlight the relevance of current semantic information and

context [21-22]. Introducing the attention mechanism at the output of the BiLSTM

layer can effectively emphasize the importance in the contextual information, and

enhance the feature expression of semantic information. The attention mechanism is

shown in Fig 6.

Figure 6. attention mechanism structure

(1) First, calculate the weight score as shown in the formula:

where, is the weight matrix; is the BiLSTM output vector; is the bias vector.

ei

(11)

ei= tanh(Wiht+bi)

Wi

ht

bi

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

98

Figure 5. BiLSTM network structure example

2.7. ATTENTION MECHANISM

Although BiLSTM neural network can establish context-related semantic vector

information, it does not highlight the relevance of current semantic information and

context [21-22]. Introducing the attention mechanism at the output of the BiLSTM

layer can effectively emphasize the importance in the contextual information, and

enhance the feature expression of semantic information. The attention mechanism is

shown in Fig 6.

Figure 6. attention mechanism structure

(1) First, calculate the weight score as shown in the formula:

where, is the weight matrix; is the BiLSTM output vector; is the bias vector.

ei

(11)

ei= tanh(Wiht+bi)

Wi

ht

bi

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

(2) Then, adopt the softmax function to calculate the attention mechanism

weight score.

(3) Finally, the point multiplication and accumulation operations are performed on

the output vector of the BiLSTM layer and the weight vector to obtain the output

Attention of the attention layer.

2.8. OUTPUT LAYER AND LOSS FUNCTION

A fully connected layer is introduced after the attention layer [23]

. First, the

weighted vector of text features is mapped into the label space of evaluation

categories. Then the Dropout mechanism is introduced after the fully connected layer

to avoid the weight update only relying on some features and model overfitting.

Finally, the softmax classifier is evaluation category to which the text belongs, and the

model prediction result is directly output.

Among them, set the softmax classifier loss function as the overall training of the

model:

where,

is the label normalization probability; y is the true label probability; the

Adam optimizer is set to continuously update the model parameters and continuously

reduce the loss function value of the model.

3. ACTUAL CASE ANALYSIS AND VERIFICATION

3.1. DATA DESCRIPTION

In the 2019-2021 online storage data of International Chinese Education Online

Teaching Quality Evaluation, text is randomly selected as the corpus. The experiment

is carried out by means of cross-validation, and the training set 80%, validation set

10% and test set 10%.

The development environment is Linux system, GPU uses NVIDIA GeForce RTX

2080Ti (11GB), Python version 3.6.5, framework uses pytorch1.7 and tensorflow1.15

version, CUDA uses version 10.1.

(12)

P

i=

exp(e

i

)

∑

n

i=1 exp(

e

i)

ht

Pi

(13)

A

ttention = 

n

∑

i=1

Pih

t

(12)

L

oss

(

y, ^

y

)

=−

k

∑

i=1

yiln^

y

i

^

y

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

99

The experiment adopts a 3-layer CNN model architecture[24-25]. The word vector

convolution windows are set to 3, 4, and 5 respectively. After the pooling operation,

the outputs of each layer are fused to enrich the local features of the context. Set the

number of LSTM units to 128 and the dropout ratio to 0.5. A multi-class cross-entropy

loss function is used. Set the batch sample size to 32, the number of training rounds

to 20, optimizer Adam, and cross-validation to evaluate the prediction performance,

where models hyperparameter settings are shown in Table 3.

Table 3. Setting of experimental parameters

3.2. MODEL PREDICTION AND EVALUATION INDEX

The experimental evaluation system includes recall rate (R), precision rate (P) and

F1

(F-measure) as indicators for evaluating model performance. The specific formulas

of each standard are as follows:

where, TP is the correct number of correct text predictions; FP is the correct

number of incorrect text predictions; FN is the correct number of correct text

predictions; F1 is the harmonic mean of precision and recall.

3.3. COMPARISON OF PREDICTION RESULTS BETWEEN

DIFFERENT MODELS

Mechanism of CNN neural network, LSTM and Attention mechanism in model

fusion is also discussed. Ten groups of comparative experiments are set up, and the

input is word2vec pre-trained word vector. To verify the influence of various models on

the expression and extraction of text features when processing text sequences. The

comparative experiment is constructed as follows.

parameter value parameter value

vee_win 100 activation relu

vec_dim 4 lstm_untis 128

lr 0.001 dropout 0.5

max_len 100 optimizer Adam

mum_filters 128 batch_size 32

kernel_size 3、4、5epochs 20

(14)

(15)

(16)

F

1=

2PR

P+R

R

=

TP

TP +FN

P

=

TP

TP +FP

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

100

The experiment adopts a 3-layer CNN model architecture[24-25]. The word vector

convolution windows are set to 3, 4, and 5 respectively. After the pooling operation,

the outputs of each layer are fused to enrich the local features of the context. Set the

number of LSTM units to 128 and the dropout ratio to 0.5. A multi-class cross-entropy

loss function is used. Set the batch sample size to 32, the number of training rounds

to 20, optimizer Adam, and cross-validation to evaluate the prediction performance,

where models hyperparameter settings are shown in Table 3.

Table 3. Setting of experimental parameters

3.2. MODEL PREDICTION AND EVALUATION INDEX

The experimental evaluation system includes recall rate (R), precision rate (P) and

F1 (F-measure) as indicators for evaluating model performance. The specific formulas

of each standard are as follows:

where, TP is the correct number of correct text predictions; FP is the correct

number of incorrect text predictions; FN is the correct number of correct text

predictions; F1 is the harmonic mean of precision and recall.

3.3. COMPARISON OF PREDICTION RESULTS BETWEEN

DIFFERENT MODELS

Mechanism of CNN neural network, LSTM and Attention mechanism in model

fusion is also discussed. Ten groups of comparative experiments are set up, and the

input is word2vec pre-trained word vector. To verify the influence of various models on

the expression and extraction of text features when processing text sequences. The

comparative experiment is constructed as follows.

parameter

value

parameter

value

vee_win

100

activation

relu

vec_dim

4

lstm_untis

128

lr

0.001

dropout

0.5

max_len

100

optimizer

Adam

mum_filters

128

batch_size

32

kernel_size

3、4、5

epochs

20

(14)

(15)

(16)

F1=2PR

P+R

R=TP

TP +FN

P=TP

TP +FP

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

(1) CNN model: CNN performs convolution, pooling, and Flatten operations on

word vectors. The input extracts the local features, and uses the fully connected layer

to reduce the dimension. Finally, the softmax classifier is used to output the prediction

result.

(2) TextCNN model: Set 3 convolution kernel windows of different sizes.

Convolutional layer and pooling layer with the same parameters, splicing the output

vector of the pooling layer by line, enriching the semantic expression of local features

of the text.

(3) LSTM model: The input sequence is used for backward semantic modeling, the

high-level features of the text are extracted, the two fully connected layers are

connected to reduce the dimension, and the prediction result is directly output.

(4) CNN_LSTM model: First use CNN to extract the local features, and then use

LSTM to extract the backward semantic information output by CNN.

(5) LSTM_CNN model: first use LSTM for backward semantic modeling, and then

use CNN to extract local features from the output of LSTM.

(6) CNN-BiLSTM model: First use CNN to extract the local features and then use

BiLSTM to extract the forward and backward semantic information output by CNN,

and further construct the feature expression of the text.

(7) BiLSTM model: Forward and backward semantic modeling is performed on the

input sequence, high-level features of the text are extracted, two fully connected

layers are connected to reduce the dimension, and the prediction result is directly

output.

(8) CNN_Att model: CNN extracts the local features of the input sequence, and the

Attention mechanism weights the text features to reduce the impact of noise features

on the classification effect.

(9) BiLSTM_Att model: BiLSTM constructs the contextual semantic information of

the input sequence, extracts the high-level features of the text, and the Attention

mechanism weights the text features to reduce the impact of noise features on the

classification effect.

(10) CNN_BiLSTM_Att model: CNN extracts the local features of the input

sequence, and then uses BiLSTM to extract the forward and backward semantic

information output by the CNN, and further constructs the feature expression of the

text. The Attention mechanism weights the text features to reduce the influence of

noise features on the classification effect.

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

101

Table 4. Results of different models.

Figure 7. Execution time of models

Model P/% R/% F1/%

CNN 86.93 87.23 86.31

TextCNN 88.31 87.62 89.64

LSTM 89.20 90.31 91.24

CNN_LSTM 90.98 91.98 90.31

LSTM_CNN 92.35 93.09 93.02

CNN-BiLSTM 94.13 93.92 93.97

BiLSTM 93.16 94.36 92.49

CNN_Att 96.20 96.08 93.52

BiLSTM_Att 94.12 93.92 93.49

CNN_BiLSTM_Att 97.89 97.76 97.85

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

102

Table 4. Results of different models.

Figure 7. Execution time of models

Model

P/%

R/%

F1/%

CNN

86.93

87.23

86.31

TextCNN

88.31

87.62

89.64

LSTM

89.20

90.31

91.24

CNN_LSTM

90.98

91.98

90.31

LSTM_CNN

92.35

93.09

93.02

CNN-BiLSTM

94.13

93.92

93.97

BiLSTM

93.16

94.36

92.49

CNN_Att

96.20

96.08

93.52

BiLSTM_Att

94.12

93.92

93.49

CNN_BiLSTM_Att

97.89

97.76

97.85

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

Figure 8. Precision trend of validation set of models

After multiple rounds of experiments and cross-validation of the experimental

results, the evaluation results of various baseline models and fusion models are

shown in Table 4, and the model execution time is shown in Fig 7

. Visually

demonstrate the superiority of the CNN_BiLSTM_Att model, the training process of

each model is analyzed, and the accuracy change process of the validation set for

each model training process is shown in Fig 8.

From the experimental results in Table 4, CNN_BiLSTM_Att model proposed in this

paper has achieved the results in the evaluation index results, with F1 reaching

97.85%. Compared with other models, the overall effect is improved by 2%~5%. The

superiority of this model can be obtained. According to the change trend of the

accuracy rate of the validation set of various algorithm models during the training

process, due to the characteristics of the corpus, the various models with CNN as the

baseline basically reach the convergence state after 10 epochs, and the accuracy rate

is high. The model with LSTM as the baseline has a turbulent trend as a whole, and it

basically reaches convergence after 18 epochs. The variation trend of the accuracy

rate of the validation set of the model in this paper is the best, reaching a state of

convergence after 7 epochs, and the accuracy rate converges at 97.89%, which is

significantly higher than other models in the comparison experiments, which further

verifies the effectiveness and robustness of the model in this paper.

4. CONCLUSION

Aiming at the characteristics of online teaching quality evaluation of international

Chinese education on the Internet, this paper first proposes the online teaching quality

evaluation method of international Chinese education on the Internet to construct the

evaluation standard system and to give weights to the evaluation indicators. Then,

combined with the characteristics of CNN, LSTM and Attention mechanism,

CNN_BiLSTM_Att-based Chinese education online teaching quality evaluation.

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

103

(1) The model extracts text local features within word windows through a multi-layer

CNN structure. At the same time, the local feature representation of the concatenated

text is used as the input of BiLSTM.

(2) BiLSTM performs forward and backward text semantic modeling to obtain high-

level feature representations of text sequences.

(3) The Attention mechanism performs feature weighting to reduce the influence of

noise features.

The experimental results show that the execution efficiency and accuracy of the

CNN_BiLSTM_Att model have achieved excellent results in various model

comparison experiments, which are suitable for online teaching quality evaluation of

international Chinese education. In the following research, we will focus on further

analysis from the aspects of word vector encoding, attention mechanism algorithm,

overall model structure and model hyperparameter settings to improve the overall

efficiency of the model.

CONFLICT OF INTEREST

The authors declared that there is no conflict of interest

REFERENCES

(1) Wu, Y., & Wang, J. (2022). Three-stage blended Chinese teaching online and

offline for international students: a case study on Chinese teaching for

international students in s university. Journal of Higher Education Research,

3(2), 207-211.

(2) Yang, H., Chen, J., & Zhan, X. (2022). "Integrated" Ideological and Political

Education System for TCSOL Major. The International Conference on Artificial

Intelligence and Logistics Engineering. Springer, Cham.

(3) Kong, L., Woods, O., & Zhu, H. (2022). The (de) territorialized appeal of

international schools in china: forging brands, boundaries and inter-

belonging in segregated urban space. Urban Studies, 59.

(4) Owens, L. M. D., & Kadakia, C. (2022). Instructional Design in a Digital Age:

A New Model Is Taking Hold. The Learning Ideas Conference. Springer, Cham.

(5) Yan, Y. (2021). Analysis on the strategies of information technology to

improve autonomous learning in higher vocational english teaching.

(6) Liu, L., & Tsai, S. B. (2021). Intelligent recognition and teaching of english

fuzzy texts based on fuzzy computing and big data. Wireless

Communications and Mobile Computing, 2021(1), 1-10.

(7) Kaire, U., Mari, K., & Liisa, P. (2018). Relations between students'

perceptions of the teaching-learning environment and teachers'

approaches to teaching: a qualitative study. Journal of Further & Higher

Education, 1-20.

(8) Mishra, A. R., Chandel, A., & Saeidi, P. (2022). Low-carbon tourism strategy

evaluation and selection using interval-valued intuitionistic fuzzy additive

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

104

(1) The model extracts text local features within word windows through a multi-layer

CNN structure. At the same time, the local feature representation of the concatenated

text is used as the input of BiLSTM.

(2) BiLSTM performs forward and backward text semantic modeling to obtain high-

level feature representations of text sequences.

(3) The Attention mechanism performs feature weighting to reduce the influence of

noise features.

The experimental results show that the execution efficiency and accuracy of the

CNN_BiLSTM_Att model have achieved excellent results in various model

comparison experiments, which are suitable for online teaching quality evaluation of

international Chinese education. In the following research, we will focus on further

analysis from the aspects of word vector encoding, attention mechanism algorithm,

overall model structure and model hyperparameter settings to improve the overall

efficiency of the model.

CONFLICT OF INTEREST

The authors declared that there is no conflict of interest

REFERENCES

(1) Wu, Y., & Wang, J. (2022). Three-stage blended Chinese teaching online and

offline for international students: a case study on Chinese teaching for

international students in s university. Journal of Higher Education Research,

3(2), 207-211.

(2) Yang, H., Chen, J., & Zhan, X. (2022). "Integrated" Ideological and Political

Education System for TCSOL Major. The International Conference on Artificial

Intelligence and Logistics Engineering. Springer, Cham.

(3) Kong, L., Woods, O., & Zhu, H. (2022). The (de) territorialized appeal of

international schools in china: forging brands, boundaries and inter-

belonging in segregated urban space. Urban Studies, 59.

(4) Owens, L. M. D., & Kadakia, C. (2022). Instructional Design in a Digital Age:

A New Model Is Taking Hold. The Learning Ideas Conference. Springer, Cham.

(5) Yan, Y. (2021). Analysis on the strategies of information technology to

improve autonomous learning in higher vocational english teaching.

(6) Liu, L., & Tsai, S. B. (2021). Intelligent recognition and teaching of english

fuzzy texts based on fuzzy computing and big data. Wireless

Communications and Mobile Computing, 2021(1), 1-10.

(7) Kaire, U., Mari, K., & Liisa, P. (2018). Relations between students'

perceptions of the teaching-learning environment and teachers'

approaches to teaching: a qualitative study. Journal of Further & Higher

Education, 1-20.

(8) Mishra, A. R., Chandel, A., & Saeidi, P. (2022). Low-carbon tourism strategy

evaluation and selection using interval-valued intuitionistic fuzzy additive

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

ratio assessment approach based on similarity measures. Environment,

Development and Sustainability: A Multidisciplinary Approach to the Theory and

Practice of Sustainable Development, 24.

(9) Sang, G., Valcke, M., Braak, J. V., Tondeur, J., & Chang, Z. (2011). Predicting

ict integration into classroom teaching in chinese primary schools:

exploring the complex interplay of teacher-related variables. Journal of

Computer Assisted Learning, 27(2), 160-172.

(10) Chen, Z. F., Zhi-Gang, Y. U., Liu, S., & Li-ping. (2010). Research on software

engineering one frame with four aspects in practical teaching system.

Computer Education (9), 136-139.

(11) Hong, TAN, College, of, International, & Studies, et al. (2014). The problems in

rural english teaching and the optimization path: a study based on the

chinese general social survey data. Asian Agricultural Research (8), 4.

(12) Vandebroek, I., Thomas, E., Sanca, S., Damme, P. V., & Kimpe, N. D. (2008).

Comparison of health conditions treated with traditional and biomedical

health care in a quechua community in rural bolivia. Journal of Ethnobiology

and Ethnomedicine, 4(1), 1.

(13) Chen, W., Ye, S., Xu, G., & Yang, L. (2022). Futures Trend Forecasting Based

onAttention Hybrid Neural Network. The International Conference on Natural

Computation, Fuzzy Systems and Knowledge Discovery. Springer, Cham.

(14) Ying L, Qian Nan Z, Fu Ping W, et al. (2021). Adaptive weights learning in

CNN feature fusion for crime scene investigation image classification[J].

Connection Science, 33(3), 719-734.

(15) Zhao, P., & Peng, J. (2001). High efficient and intensified use of urban land

and its evaluation index system. Resources Science.

(16) Liu, F., & Hai, H. L. (2005). The voting analytic hierarchy process method for

selecting supplier. International Journal of Production Economics, 97(3),

308-317.

(17) Nydick, R. L., & Hill, R. P. (1992). Using the analytic hierarchy process to

structure the supplier selection procedure. International Journal of

Purchasing & Materials Management, 28(2), 31-36.

(18) Mei, & Liang. (2011). Research of index weighting on evaluation index

system of partner selection in technical standard alliance by using fuzzy

analytic hierarchy process. Energy Procedia, 13, 9922-9929.

(19) Sadr, H., & Soleimandarabi, M. N. (2022). Acnn-tl: attention-based

convolutional neural network coupling with transfer learning and

contextualized word representation for enhancing the performance of

sentiment classification. The Journal of Supercomputing, 78(7), 10149-10175.

(20) Shan W. (2022). Digital streaming media distribution and transmission

process optimisation based on adaptive recurrent neural network[J].

Connection Science, 34(1), 1169-1180.

(21) Wang, Xuan, Xu, Ruifeng, He, & Yulan, et al. (2017). Improving sentiment

analysis via sentence type classification using bilstm-crf and cnn. Expert

Systems with Application.

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

105

(22) X. Ning, K. Gong, W. Li, and L. Zhang. (2021). JWSAA: Joint Weak Saliency

and Attention Aware for person re-identification, Neurocomputing, 453,

801-811. https://doi.org/10.1016/j.neucom.2020.05.106.

(23) W Cai, B Zhai, Y Liu. (2021). Quadratic polynomial guided fuzzy C-means

and dual attention mechanism for medical image segmentation. Displays,

70, 102106. https://doi.org/10.1016/j.displa.2021.102106.

(24) Kaseng, F., Lezama, P., Inquilla, R., y Rodriguez, C. (2020). Evolution and

advance usage of Internet in Peru.

3C TIC. Cuadernos de desarrollo aplicados

a las TIC, 9(4), 113-127. https://doi.org/10.17993/3ctic.2020.94.113-127

(25) Yan Kang,Jinling Song,Mingming Bian,Haipeng Feng & Salama Mohamed.

(2022). Red tide monitoring method in coastal waters of Hebei Province

based on decision tree classification. Applied Mathematics and Nonlinear

Sciences, 7(1), 43-60. https://doi.org/10.2478/AMNS.2022.1.00051.

https://doi.org/10.17993/3ctecno.2023.v12n1e43.87-106

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143

Ed.43 | Iss.12 | N.1 January - March 2023

106