RESEARCH ON INNOVATION OF DAILY
IDEOLOGICAL AND POLITICAL EDUCATION
FOR COLLEGE STUDENTS BASED ON DEEP
LEARNING MODEL
Xianwei Zhang*
College of Economics, Shenzhen Polytechnic, Shenzhen, Guangdong, 518055,
China
zxw5460@126.com
Yueyan Zhang
Department of Pharmacy, The Second People’s Hospital of Longgang District of
Shenzhen, Shenzhen, Guangdong, 518112, China
Reception: 12/11/2022 Acceptance: 04/01/2023 Publication: 31/01/2023
Suggested citation:
Z., Xianwei and Z., Yueyan. (2023). Research on Innovation of Daily
Ideological and Political Education for College Students based on Deep
Learning Model. 3C Tecnología. Glosas de innovación aplicada a la pyme,
12(1), 108-125. https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
108
RESEARCH ON INNOVATION OF DAILY
IDEOLOGICAL AND POLITICAL EDUCATION
FOR COLLEGE STUDENTS BASED ON DEEP
LEARNING MODEL
Xianwei Zhang*
College of Economics, Shenzhen Polytechnic, Shenzhen, Guangdong, 518055,
China
zxw5460@126.com
Yueyan Zhang
Department of Pharmacy, The Second People’s Hospital of Longgang District of
Shenzhen, Shenzhen, Guangdong, 518112, China
Reception: 12/11/2022 Acceptance: 04/01/2023 Publication: 31/01/2023
Suggested citation:
Z., Xianwei and Z., Yueyan. (2023). Research on Innovation of Daily
Ideological and Political Education for College Students based on Deep
Learning Model. 3C Tecnología. Glosas de innovación aplicada a la pyme,
12(1), 108-125. https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
ABSTRACT
Various network information is mixed, which has a great impact education, with
continuous development informatization. However, development of informatization has
provided convenience for the daily ideological political education, effectively solved
time and space limitation daily ideological, and sustainable development. Therefore,
positively influence formation of college students' noble morality. The informatization
education resources can be effectively integrated, and the utilization rate resources
improved. Information resources of ideological and political education, we propose a
complete block diagram of the daily ideological system of college students. First,
design a complete interactive analysis questionnaire for college student’s role of daily
ideological and political education. Through questionnaire survey method, the survey
and statistical weight scores were conducted to analyze the proportion of each
indicator. Then, the framework of education in the network environment is adopted,
which includes, class tutoring learning, class interactive learning, class in-depth study,
process evaluation and feedback evaluation. Learn through a period of ideological
and political education. Collect data as our training corpus. Finally, the training
prediction model BERT-BiLSTM-CRF-based trained. Prediction of F1 BERT-BiLSTM-
CRF -based can reach 91.09%.
KEYWORDS
Deep learning; ideological; political education; educational innovation; online
education
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
109
PAPER INDEX
ABSTRACT
KEYWORDS
1. INTRODUCTION
2. METHODOLOGY
2.1. Interaction analysis method of ideological and political education
2.2. Framework of ideological and political education in the network environment
2.3. BERT-BiLSTM-CRF construction
2.3.1. BERT and normalization
2.3.2. BiLSTM
2.3.3. Attention mechanism
2.3.4. CRF
3. EXPERIMENTAL RESULTS AND ANALYSIS
3.1. Dataset and training environment configuration
3.2. Dataset labeling and evaluation metrics
3.3. Results and analysis
CONCLUSION
CONFLICT OF INTEREST
REFERENCES
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
110
PAPER INDEX
ABSTRACT
KEYWORDS
1. INTRODUCTION
2. METHODOLOGY
2.1. Interaction analysis method of ideological and political education
2.2. Framework of ideological and political education in the network environment
2.3. BERT-BiLSTM-CRF construction
2.3.1. BERT and normalization
2.3.2. BiLSTM
2.3.3. Attention mechanism
2.3.4. CRF
3. EXPERIMENTAL RESULTS AND ANALYSIS
3.1. Dataset and training environment configuration
3.2. Dataset labeling and evaluation metrics
3.3. Results and analysis
CONCLUSION
CONFLICT OF INTEREST
REFERENCES
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
1. INTRODUCTION
Daily ideological, political education refers to political education activities that
characterized by practicality, pertinence, and interactivity in addition to ideological and
political theory courses [1-3]. Thematic education, campus culture, community
activities, mental health education, social practice, financial aid education.
With development social informatization, daily ideological political education of
college students has always been focus attention [4-6]. Exchanges and social
information are numerous and complex [7]. The circulation of some information affects
the formation of college students' ideological character. Today's daily ideological
education guide cultivation morality through teachers' reasoning as in the past.
Corresponding innovations and changes, which can positively affect the formation
noble morality information resources [8]. Through the mining and utilization of new
and modern network information resources in informatization, the informatization
education resources can be effectively integrated [9].
At present, informatization and data, and various fields have begun to the industry
[10-12]. Innovation work of college students, how times and develop scientifically data
technology, deep learning technology to important issue faced [13-15]. Promote
development of ideological and political education, so that content of daily ideological
and political education is increasingly enriched and the means are increasingly
perfected. Online resources for teaching have the following effects:
(1) The concept of mining resources, rejects information technology methods such
as multimedia courseware for teaching [16].
(2) The information-based educational resources the information-based resources
are simply applied to the daily ideological, and have not exerted the greatest effect
[17].
(3) Information platforms are flooded complex information, and some unhealthy
resources are also invisibly absorbed by students, which directly affects healthy
development of college students' political education [18].
Data mining utilization of information resources high-quality information education
resources play the growth [19]. Correct data helps to promote the cultivation of high
morality and correct outlook on life and values for college students. It is required deep
understanding of the importance of information resource mining and to carry out
effective mining and utilization healthy education [20]. Through deep learning
technology recommendation and prediction ultimately improve daily training program
for college students.
Neural network gradually develops and matures [21-23]. Emergence word vectors
can solve problem of data sparseness in high-dimensional space, and can also add
more features. The classification and recognition method Bi-directional Long Short-
Term Memory (BiLSTM)-based has improved accuracy compared with traditional
methods [24]. In addition, many pre-training models such as Bidirectional Encoder
Representations from Transformers (BERT), Long Short-Term Memory (LSTM)
networks, Transformer, etc. have recently been used, combined with self-attention
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
111
mechanism, transfer learning, etc. to improve classification accuracy methods [25].
BERT-BiLSTM-CRF classification method based on BERT and BERT-BiLSTM is
compared with CRF, Convolutional Neural Network (CNN), LSTM and other methods,
and obtained. Higher classification accuracy [26].
Therefore, we propose a training plan based on improving the recommendation
and prediction through deep learning technology, ultimately. Among them, the main
work is highlighted as follows:
(1) Design a complete set of interactive analysis questionnaires. Through the
questionnaire survey method, the survey and statistical weight scores were conducted
to analyze the proportion of each indicator.
(2) Adopt the framework in the network environment, which includes five aspects:
class tutoring learning, class interactive learning, class in-depth study, process
evaluation and feedback evaluation. Learn through a period of ideological and political
education. Collect data as our training corpus.
(3) Design and propose CNN、CNN-CRF、LSTM、LSTM-CRF、BiLSTM、
BiLSTM-CRF and BERT-BiLSTM-CRF. The experimental results are analyzed and
discussed.
2. METHODOLOGY
2.1. INTERACTION ANALYSIS METHOD OF IDEOLOGICAL AND
POLITICAL EDUCATION
Research application of new methods of data in ideological and political education.
Deep learning analysis data method can predict and analyze the students' network
thinking and behavior. Through mining and data to establish a deep learning model, it
can realize the whole-process and full-sample analysis of individual students or
groups, and realize personalized recommendation for innovation and reform[27-28].
We conduct online questionnaire surveys and statistical analysis through the following
five aspects, is shown Table 1.
Table 1. Questionnaire
Student Group Category\Evaluation
Proportion
Proportion of evaluation grades (%)
0 1 2 3 4 Weights
Different types
of students
"985" college students 6.0 5.1 28.6 36.0 24.3
Non-"985" college students 3.1 6.0 27.4 38.1 25.4
Students of
different
disciplines
Humanities 3.8 7.2 25.4 37.0 26.6
Social studies 3.8 6.0 28.7 34.7 25.8
Science and Engineering 1.9 7.9 25.4 40.0 24.8
Agricultural disciplines 2.9 6.1 41.2 25.8 23.0
Medical disciplines 1.8 4.5 29.7 37.9 26.1
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
112
mechanism, transfer learning, etc. to improve classification accuracy methods [25].
BERT-BiLSTM-CRF classification method based on BERT and BERT-BiLSTM is
compared with CRF, Convolutional Neural Network (CNN), LSTM and other methods,
and obtained. Higher classification accuracy [26].
Therefore, we propose a training plan based on improving the recommendation
and prediction through deep learning technology, ultimately. Among them, the main
work is highlighted as follows:
(1) Design a complete set of interactive analysis questionnaires. Through the
questionnaire survey method, the survey and statistical weight scores were conducted
to analyze the proportion of each indicator.
(2) Adopt the framework in the network environment, which includes five aspects:
class tutoring learning, class interactive learning, class in-depth study, process
evaluation and feedback evaluation. Learn through a period of ideological and political
education. Collect data as our training corpus.
(3) Design and propose CNN、CNN-CRF、LSTM、LSTM-CRF、BiLSTM、
BiLSTM-CRF and BERT-BiLSTM-CRF. The experimental results are analyzed and
discussed.
2. METHODOLOGY
2.1. INTERACTION ANALYSIS METHOD OF IDEOLOGICAL AND
POLITICAL EDUCATION
Research application of new methods of data in ideological and political education.
Deep learning analysis data method can predict and analyze the students' network
thinking and behavior. Through mining and data to establish a deep learning model, it
can realize the whole-process and full-sample analysis of individual students or
groups, and realize personalized recommendation for innovation and reform[27-28].
We conduct online questionnaire surveys and statistical analysis through the following
five aspects, is shown Table 1.
Table 1. Questionnaire
Student Group Category\Evaluation
Proportion
Proportion of evaluation grades (%)
1
2
3
4
Weights
Different types
of students
"985" college students
5.1
28.6
36.0
24.3
Non-"985" college students
6.0
27.4
38.1
25.4
Students of
different
disciplines
Humanities
7.2
25.4
37.0
26.6
Social studies
6.0
28.7
34.7
25.8
Science and Engineering
7.9
25.4
40.0
24.8
Agricultural disciplines
6.1
41.2
25.8
23.0
Medical disciplines
4.5
29.7
37.9
26.1
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
(1) College students' overall evaluation of campus cultural activities.
(2) The situation of the Party and Youth League organizations where different
groups of college students are located to carry out organizational life. College
students of different age groups, disciplines, and political outlooks have significant
differences in the conditions of their party and youth organizations to carry out
organizational life.
(3) The satisfaction evaluation of college students on the activities of the Party and
Youth League.
(4) The development status education for college students.
(5) Participating in student associations and the satisfaction evaluation of student
association activities.
2.2. FRAMEWORK OF IDEOLOGICAL AND POLITICAL
EDUCATION IN THE NETWORK ENVIRONMENT
Effective use of network resources ideological and political educator. Improve work
effectiveness is urgent. We build a set of ideological education work framework in the
network environment, Fig 1.
Figure 1. Network Environment
Students of
different
ages
Undergraduate (freshman) 4.0 7.1 28.3 36.6 24.0
Undergraduate (Sophomore) 4.2 6.9 29.1 20.8 29.2
Undergraduate (Junior) 4.2 4.9 25.7 35.2 30
Undergraduate (senior year) 2.9 13.8 23.3 23.9 36.1
Postgraduate 3.0 6.9 27.7 38.1 23.4
PhD student 2.5 4.8 27.5 36.3 29.8
Students with cadre
experience (yes/no)
Officer experience (yes) 3.0 8.2 26.4 38.7 23.7
Cadre Experience (No) 2.7 6.4 27.1 38.1 25.7
Political
status
Party member (yes) 3.1 6.7 26.6 38.1 25.5
Party member (no) 4.2 7.9 35.1 32.8 20.0
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
113
(1) Pre-class tutoring: Pre-class tutoring is the preparatory work for classroom
teaching. The key point of tutoring learning is to use the ideological education and
teaching materials provided by teachers to allow students to independently find out
the ideological tasks and difficulties of the study unit, so that classroom listening is
more targeted. The method is to use the "online ideological and political education" or
"Communist Youth League ideological education" video function module of the
network to issue the pre-class autonomous learning content (the content can be
discussion or divergent thinking, etc.) to each student or each group.
(2) Interactive learning in class: Questionnaires about students' difficult points, and
then use the questionnaire survey module to interact with students. At the same time,
teach knowledge points by watching and other methods.
(3) In-depth study after class: In-depth study after class refers to an important
process in which teachers use question answering discussions or tests to conduct
complementary learning with students after class teaching. Through this stage,
consolidate achieve the teaching purpose of in-depth understanding and further
improvement.
(4) Process assessment and evaluation: Assessment and evaluation refers to the
process in which teachers need to evaluate the teaching link in a timely manner,
which can be carried out by using the assessment question bank (note that this
process is the data automatically generated by the system). Teachers only need to
give praise to students with outstanding performance and good learning effect
motivate other students.
(5) Feedback or evaluation: Students give feedback on the pre-class, after-class,
and assessment. Final calculation ratio is converted into a score, in which we divide
the percentile system into four grades, 0~20, 20~40, 40~60, 60~80 and 80~100, and
collect the information in Table 1
and the selected courses. Information such as name
and viewing time are used as a corpus.
2.3. BERT-BILSTM-CRF CONSTRUCTION
Language preprocessing, a language task for classification, has been a hot
research topic. Compared with traditional language models, it is more conducive to
classification for classification tasks. Collected in this paper contains text and scores.
For the mixed text of text and scores, corresponding features are extracted in the first
layer. BERT-BiLSTM-CRF framework is shown in Fig 2. Divided into 4 layers:
(1) BERT and normalization layer. BERT can represent polysemy, and the corpus is
pre-trained by BERT, as shown in Fig 3. At the same time, normalization is performed
for the score features, and the value range of all features is between [0, 1].
(2) Second layer: BiLSTM layer. BiLSTM uses forward-LSTM and backward -LSTM
to capture the contextual features of the text.
(3) Third layer: Attention layer, which different levels of contextual information,
assigns different weights to it, and captures the latent semantic features between
texts.
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
114
(1) Pre-class tutoring: Pre-class tutoring is the preparatory work for classroom
teaching. The key point of tutoring learning is to use the ideological education and
teaching materials provided by teachers to allow students to independently find out
the ideological tasks and difficulties of the study unit, so that classroom listening is
more targeted. The method is to use the "online ideological and political education" or
"Communist Youth League ideological education" video function module of the
network to issue the pre-class autonomous learning content (the content can be
discussion or divergent thinking, etc.) to each student or each group.
(2) Interactive learning in class: Questionnaires about students' difficult points, and
then use the questionnaire survey module to interact with students. At the same time,
teach knowledge points by watching and other methods.
(3) In-depth study after class: In-depth study after class refers to an important
process in which teachers use question answering discussions or tests to conduct
complementary learning with students after class teaching. Through this stage,
consolidate achieve the teaching purpose of in-depth understanding and further
improvement.
(4) Process assessment and evaluation: Assessment and evaluation refers to the
process in which teachers need to evaluate the teaching link in a timely manner,
which can be carried out by using the assessment question bank (note that this
process is the data automatically generated by the system). Teachers only need to
give praise to students with outstanding performance and good learning effect
motivate other students.
(5) Feedback or evaluation: Students give feedback on the pre-class, after-class,
and assessment. Final calculation ratio is converted into a score, in which we divide
the percentile system into four grades, 0~20, 20~40, 40~60, 60~80 and 80~100, and
collect the information in Table 1 and the selected courses. Information such as name
and viewing time are used as a corpus.
2.3. BERT-BILSTM-CRF CONSTRUCTION
Language preprocessing, a language task for classification, has been a hot
research topic. Compared with traditional language models, it is more conducive to
classification for classification tasks. Collected in this paper contains text and scores.
For the mixed text of text and scores, corresponding features are extracted in the first
layer. BERT-BiLSTM-CRF framework is shown in Fig 2. Divided into 4 layers:
(1) BERT and normalization layer. BERT can represent polysemy, and the corpus is
pre-trained by BERT, as shown in Fig 3. At the same time, normalization is performed
for the score features, and the value range of all features is between [0, 1].
(2) Second layer: BiLSTM layer. BiLSTM uses forward-LSTM and backward -LSTM
to capture the contextual features of the text.
(3) Third layer: Attention layer, which different levels of contextual information,
assigns different weights to it, and captures the latent semantic features between
texts.
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
(4) Fourth layer: CRF to ensure the predicted labels. CRF to decode and label the
output results, and extract and classify entities.
Figure 2. BERT-BiLSTM-CRF framework
Figure 3. BERT's network structure
2.3.1. BERT AND NORMALIZATION
Language models through one-hot, Word2Vec, ELMO, GPT to BERT. However,
module adopted in this paper performs pre-processing on sentences, according to the
characteristics of Chinese word segmentation, the method of whole word Mask is
applied to Chinese. In the whole word Mask.
Deep network based on "self-attention mechanism", and its encoder structure is
shown in Fig 4. Do not have the ability to obtain the sequence of the entire sentence
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
115
like RNN, so to solve this problem, Transformer adds position encoding before data
preprocessing, and sums it with the input vector data to get the relative position.
Figure 4. Transformer
First, word segmentation text sequence is obtained through word segmentation
processing. Part of the word segmentation sequence is used for whole word Mask, in
which a special mark [CLS] is added at the beginning of the sequence, and the
sentences are separated by a mark [SEP].
For score value, we use normalization processing. First, the distance digital feature
(column) is regarded as the unit 1, and then we look at the ratio of the distance
between x and the minimum value to the total distance. The final output is a
percentage between [0,1].
where Min and Max are the minimum and maximum values of the property,
respectively.
where, are word vector matrices, and is the Embedding dimension.
The multi-head attention mechanism is projected through multiple different linear
transformation pairs:
Ṓ
Masked
multi-head-
attention
Normalize
Multi-head-
attention
Normalize
Feed forward
Normalize
Positional encoding
N
Input text
(1)
X−Min
Ma x −Min
(2)
A
ttention(Q, K, V)=Sof tma x(
QKT
d
k
)V
Q, K, V
dk
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
116
like RNN, so to solve this problem, Transformer adds position encoding before data
preprocessing, and sums it with the input vector data to get the relative position.
Figure 4. Transformer
First, word segmentation text sequence is obtained through word segmentation
processing. Part of the word segmentation sequence is used for whole word Mask, in
which a special mark [CLS] is added at the beginning of the sequence, and the
sentences are separated by a mark [SEP].
For score value, we use normalization processing. First, the distance digital feature
(column) is regarded as the unit 1, and then we look at the ratio of the distance
between x and the minimum value to the total distance. The final output is a
percentage between [0,1].
where Min and Max are the minimum and maximum values of the property,
respectively.
where, are word vector matrices, and is the Embedding dimension.
The multi-head attention mechanism is projected through multiple different linear
transformation pairs:
Ṓ
Masked
multi-head-
attention
Normalize
Multi-head-
attention
Normalize
Feed forward
Normalize
Positional encoding
N
Input text
(1)
X−Min
Ma x −Min
(2)
Attention(Q, K, V)=Sof tma x(QKT
dk
)V
Q, K, V
dk
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
where W is the weight matrix. Location information in different spaces.
ReLU and linear activation function form a fully connected feedforward network
(FFN).
where the output of the multi-head attention mechanism is denoted as Z, and b is
the bias vector.
2.3.2. BILSTM
The relevant information of the previous moment cannot be used for the next
moment. Recurrent Neural Networks (RNNs) have this capability. However, it is
difficult to learn relevant information when the predicted points are far away from the
dependent relevant information. LSTM can solve this problem very well and learn
long-term dependency information. LSTM addresses the exploding or vanishing
gradients that occur during RNN training. Unit structure is shown in Fig 5.
Figure 5. LSTM
(3)
(4)
head1=Attention(QWQ
i,K WK
i,V WV
i)
MultiHead(Q, K, V)=Concat(head1,head2,…,headn)W0
(5)
FNN(Z)= max(0,Z W1+b1)W2+b2
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
117
The common function of input and forgetting gate is to discard the useless
sequence information. For entire structure, the result of multiplying the output gate. Its
structure is expressed by the formula as follows:
where is the input text vector;
is the sigmoid function; tanh is the activation
function; are the input gate, output gate and forget gate, respectively.
Unit of information stored is
, where input gate and forget gate are not used to
adjust the information cell.
where,
are the weights of different gate control mechanisms
on the input text vector ;
are the weights of different gate
control mechanisms on the hidden layer vector ;
are bias
vectors.
Unit information of forget gate and previous moment is stored in .
where, the hidden layer
is calculated by the output gate and the storage gate
.
At the same time, input a word vector B output from the RERT pre-training
language model to BiLSTM, represents the input data at time t, ,
and
and .
Finally, the two are combined to obtain the output of BiLSTM at
, and
the forward and backward outputs of the LSTM unit at time t are spliced. Get a
sequence of hidden states .
where n represents the vector set.
2.3.3. ATTENTION MECHANISM
BiLSTM can obtain contextual information, but does not highlight the latent
semantic correlation between current sequence feature information and contextual
(6)
(7)
(8)
ft=δ(Wfxt+Ufht−1+bf)
it=δ(Wixt+Uiht−1+bi)
ot=δ(Woxt+Uoht−1+bo)
xt
δ
it,ot,ft
cellt
(10)
cellt= tanh(Wcxt+Ucht−1+bc)
Wi,Wo,WfandWc
xt
Ui,Uo,UfandUc
ht−1
bi,bo,bfandbc
cellt
(11)
(12)
ht=ottanh(cellt)
cellt=ftcellt−1+it
~
cellt
ht
cellt
Xt
→
ht
←
ht
→
ht= (→
h0,→
h1,…,→
ht)
←
ht= (←
h0,←
h1,…,←
ht)
ht= [→
ht,←
ht]
t0,t1,...,ti
(13)
ht= [→
ht,←
ht]∈Rn
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
118
The common function of input and forgetting gate is to discard the useless
sequence information. For entire structure, the result of multiplying the output gate. Its
structure is expressed by the formula as follows:
where is the input text vector; is the sigmoid function; tanh is the activation
function; are the input gate, output gate and forget gate, respectively.
Unit of information stored is , where input gate and forget gate are not used to
adjust the information cell.
where, are the weights of different gate control mechanisms
on the input text vector ; are the weights of different gate
control mechanisms on the hidden layer vector ; are bias
vectors.
Unit information of forget gate and previous moment is stored in .
where, the hidden layer is calculated by the output gate and the storage gate
.
At the same time, input a word vector B output from the RERT pre-training
language model to BiLSTM, represents the input data at time t, , and
and .
Finally, the two are combined to obtain the output of BiLSTM at , and
the forward and backward outputs of the LSTM unit at time t are spliced. Get a
sequence of hidden states .
where n represents the vector set.
2.3.3. ATTENTION MECHANISM
BiLSTM can obtain contextual information, but does not highlight the latent
semantic correlation between current sequence feature information and contextual
(6)
(7)
(8)
ft=δ(Wfxt+Ufht−1+bf)
it=δ(Wixt+Uiht−1+bi)
ot=δ(Woxt+Uoht−1+bo)
xt
δ
it,ot,ft
cellt
(10)
cellt= tanh(Wcxt+Ucht−1+bc)
Wi,Wo,WfandWc
xt
Ui,Uo,UfandUc
ht−1
bi,bo,bfandbc
cellt
(11)
(12)
ht=ottanh(cellt)
cellt=ftcellt−1+it
~
cellt
ht
cellt
Xt
→
ht
←
ht
→
ht= (→
h0,→
h1,…,→
ht)
←
ht= (←
h0,←
h1,…,←
ht)
ht= [→
ht,←
ht]
t0,t1,...,ti
(13)
ht= [→
ht,←
ht]∈Rn
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
information. Therefore, an attention layer is added after the BiLSTM network to mine
the latent semantic features between texts.
First, word vector sequence into BiLSTM to extract contextual features.
where, the attention weight is ;
is the feature vector output by the BiLSTM
layer.
Then, the attention mechanism different weights to the different feature vectors of
the text.
attention weight probability vector is .
Finally, contextual features and latent semantic features is generated.
where, the attention weights are configured as .
2.3.4. CRF
CRF chooses conditional random fields for sequence labeling. Classification task,
dependencies between adjacent labels. CRF can obtain an optimal prediction
sequence through the relationship between adjacent tags. It can make up for the
shortcomings of BiLSTM.
First, input any sequence .
Then, number n words and the number of k labels. For prediction sequence
, the score function:
where,
represents score; A represents transfer score matrix; The probability that
the Y is generated is:
Finally, decoder the largest score.
(14)
vt= tanh(ht)
vt
ht
(15)
P
t=
exp(v
t
)
∑
n
t−1exp(
v
t)
Pt
(16)
α
t
=∑n
t−1P
t
ht
αt
X=(x0,x1, …, xn)
Y=(y0,y1,…,yn)
(17)
S
(X, Y)=
n
∑
i=0
Ayi,yi+1 +
n
∑
i=1
Pi,y
i
Aij
(18)
P
(YX)=
es(X,Y)
∑
Y∈YX
s(X,Y)
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
119
where, Y represents real annotation sequence; represents possible annotation
sequences.
3. EXPERIMENTAL RESULTS AND ANALYSIS
3.1. DATASET AND TRAINING ENVIRONMENT
CONFIGURATION
The corpus used in this experiment includes student names, place names, course
names, and ages. The experimental data mainly comes from the daily ideological
education storage data of a college from 2019 to 2022, including 502 electronic
feedback questionnaires with a total of 16,465,469 words. The data in the electronic
feedback questionnaire was 10% as the test set, 10% as the validation set and 80%
as the training set. IData, the marked data will not be changed, and if there are
missing features, 0 will be added.
Table 3. Experimental Environment and Hyperparameters
Training process, Adam optimizer is used; learning rate is 0.001. At the same time,
LSTM_dim is 200, batch_size is 64, and max_seq_len is 128. In order to prevent the
overfitting problem. The specific hyperparameter settings and training environment
configuration is shown in Table 3.
3.2. DATASET LABELING AND EVALUATION METRICS
The BIO system, which has 7 labels. In this paper, the recall rate R, precision rate
P and F1 value are used to evaluate the performance of the model. The calculation
methods of each evaluation index are as follows:
A is the total number of entities, and B is the predicted number.
(19)
(20)
Y* =argma xs(X,Y)Y∈YX
I
n
(
P(Y X)
)
= s(X, Y)−In( ∑
Y∈YX
s(X,Y)
)
YX
Category configuration configuration
Hardware
CPU GTX 2080Ti RAM 128GB
GPU E5-2650L V3-8 video memory 11GB
operating system Ubuntu 18.04 Linux 64
Software Python 3.6.12 Tensorflow 2.2.0
CUDA 11.0
Parameter Transformer layer 12 Hidden layer dimension 768
optimizer Adam learning rate 0.001
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
120
where, Y represents real annotation sequence; represents possible annotation
sequences.
3. EXPERIMENTAL RESULTS AND ANALYSIS
3.1. DATASET AND TRAINING ENVIRONMENT
CONFIGURATION
The corpus used in this experiment includes student names, place names, course
names, and ages. The experimental data mainly comes from the daily ideological
education storage data of a college from 2019 to 2022, including 502 electronic
feedback questionnaires with a total of 16,465,469 words. The data in the electronic
feedback questionnaire was 10% as the test set, 10% as the validation set and 80%
as the training set. IData, the marked data will not be changed, and if there are
missing features, 0 will be added.
Table 3. Experimental Environment and Hyperparameters
Training process, Adam optimizer is used; learning rate is 0.001. At the same time,
LSTM_dim is 200, batch_size is 64, and max_seq_len is 128. In order to prevent the
overfitting problem. The specific hyperparameter settings and training environment
configuration is shown in Table 3.
3.2. DATASET LABELING AND EVALUATION METRICS
The BIO system, which has 7 labels. In this paper, the recall rate R, precision rate
P and F1 value are used to evaluate the performance of the model. The calculation
methods of each evaluation index are as follows:
A is the total number of entities, and B is the predicted number.
(19)
(20)
Y* =argma xs(X,Y)Y∈YX
In(P(Y X))= s(X, Y)−In( ∑
Y∈YX
s(X,Y))
YX
Category
configuration
configuration
Hardware
CPU
GTX 2080Ti
RAM
128GB
GPU
E5-2650L V3-8
video memory
11GB
operating system
Ubuntu 18.04
Linux 64
Software
Python
3.6.12
Tensorflow
2.2.0
CUDA
11.0
Parameter
Transformer layer
12
Hidden layer dimension
768
optimizer
Adam
learning rate
0.001
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3.3. RESULTS AND ANALYSIS
On the dataset, CNN、CNN-CRF、LSTM、LSTM-CRF、BiLSTM、BiLSTM-CRF
and BERT-BiLSTM-CRF are used for performance analysis, are shown in Table 5.
(1) CNN model and the LSTM model, it can be seen that LSTM is better than the
training dataset in this paper.
(2) CNN, LSTM model and CNN-CRF, LSTM-CRF model, it can be seen that after
adding CRF module, the F1 value is improved. This is because CRF can take full
advantage of the association of adjacent tags.
(3) LSTM-CRF and the BiLSTM-CRF. BiLSTM performs better than LSTM,
because LSTM use the following information.
(4) BiLSTM-CRF and the BERT-BiLSTM-CRF, the F1 value improved, because the
BERT deeply extract text semantic information and fully characterize polysemy.
(5) Among them, the attention mechanism makes the model focus more on finding
input information more relevant to the current output, strengthens the latent semantic
correlation between current information and contextual information, and improves the
accuracy of prediction.
Table 5. Comparison of the effects of each model (/%)
At the same time, the time required for each model training round is also
compared, as shown in Table 6. Traditional model is about 15 times that of the BERT-
BiLSTM-CRF. Training time of the BERT-BiLSTM-CRF is the least among all models,
indicating that the BERT of the whole word Mask has higher training efficiency.
Compares and analyzes the update of the F1
value in the first 25 rounds, as shown in
Fig 6.
(21)
(22)
(23)
F
1=
2PR
P+R
×100
%
P
=
a
B
×100
%
R
=
a
A
×100
%
Model P R
CNN 80.83 79.52 79.67
LSTM 81.59 82.18 81.37
CNN-CRF 81.16 80.87 80.15
LSTM-CRF 82.68 83.12 82.34
BiLSTM 83.48 83.12 83.02
BiLSTM-CRF 85.87 85.45 85.09
BERT-BiLSTM-CRF 91.63 90.56 91.09
F1
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
121
Table 6. Training time (/s)
Figure 6. Experimental results
CONCLUSION
In this paper, information resources are used to positively influence the formation of
college students' noble morality. Informatization education resources can be
effectively integrated, and the utilization rate of the ideological and political education
resources can be improved. (1) Design complete interactive analysis questionnaires
for college students to evaluate the role of daily ideological and political education.
Through questionnaire survey method, survey and statistical weight scores were
conducted to analyze the proportion of each indicator.
(2) Adopt the ideological political education work framework, which includes five
aspects: class tutoring, class interactive learning, class in-depth study, process
evaluation and feedback evaluation. Learn through a period of ideological and political
education. Collect data as our training corpus.
Model Time
CNN 1089
LSTM 1834
CNN-CRF 227
LSTM-CRF 392
BiLSTM 452
BiLSTM-CRF 351
BERT-BiLSTM-CRF 120
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
122
Table 6. Training time (/s)
Figure 6. Experimental results
CONCLUSION
In this paper, information resources are used to positively influence the formation of
college students' noble morality. Informatization education resources can be
effectively integrated, and the utilization rate of the ideological and political education
resources can be improved. (1) Design complete interactive analysis questionnaires
for college students to evaluate the role of daily ideological and political education.
Through questionnaire survey method, survey and statistical weight scores were
conducted to analyze the proportion of each indicator.
(2) Adopt the ideological political education work framework, which includes five
aspects: class tutoring, class interactive learning, class in-depth study, process
evaluation and feedback evaluation. Learn through a period of ideological and political
education. Collect data as our training corpus.
Model
Time
CNN
1089
LSTM
1834
CNN-CRF
227
LSTM-CRF
392
BiLSTM
452
BiLSTM-CRF
351
BERT-BiLSTM-CRF
120
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
(3) Design and propose CNN、CNN-CRF、LSTM、LSTM-CRF、BiLSTM、
BiLSTM-CRF and BERT-BiLSTM-CRF. Experimental results are analyzed and
discussed.
Finally, the training prediction model based on the BERT-BiLSTM-CRF model is
trained and compared with other models. Experimental results show prediction
accuracy based on the BERT-BiLSTM-CRF is the best. Deep learning prediction is
only a reference direction, and practical application.
CONFLICT OF INTEREST
The authors declared that there is no conflict of interest
REFERENCES
(1) Wang, D. (2005). Construct of the content system of ideological and
political education in human concern. Teaching & Research.
(2) Shi, Y. (2006). On the teaching students in accordance with their aptitude in
the ideological and political education of students in vocational colleges.
Vocational Education & Economic Research(Journal of Loudi Vocational &
Technical College), 14(5), 679-691.
(3) Jia, C., & Dan, L. (2016). On ways to infiltrate ecological civilization
education into ideological and political education of higher vocational
colleges. Science education article collects, 56(4), 337-341.
(4) Geng, S., & Kang, Y. (2012). The innovation of the subject in ideological and
political education from the prospective of social interconstitutive theory.
Social Sciences Journal of Universities in Shanxi, 441(1), 99-104.
(5) Zhang, T. C., & Xiao, F. S. (2010). The red resources and ideological and
political education of university students. Teaching and Research.
(6) Li, J., & Zhang, T. (2011). Three smart combinations of ideological and
political education in colleges and universities. Social Sciences Journal of
Universities in Shanxi, 22(2003), 4038-4046.
(7) Yan, L. I., Zeng, W. L., & Hai-Tao, H. E. (2010). An analysis of the college
students' ideological and political education vector under the new media
environment. Journal of Chongqing University of Posts and
Telecommunications (Social Science Edition).
(8) Chen, W. B. (2001). A further study on the regularity in the process of
ideological and political education. Journal of Central China Normal
University (Humanities and Social Sciences).
(9) Pan, Y., & Tu, K. (2011). Method innovation of undergraduate ideological
and political education based on network environment. Procedia
Engineering, 15(1), 2752-2756.
(10) Yang, H., Chen, J., & Zhan, X. (2022). "Integrated" Ideological and Political
Education System for TCSOL Major. The International Conference on Artificial
Intelligence and Logistics Engineering. Springer, Cham.
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
123
(11) Si, Y. (2022). On the influence of party history education on ideological and
political education for medical students. Journal of Higher Education
Research, 3(2), 141-145.
(12) Tao, T., & Lv, X. (2022). Construction of ideological and political education
in colleges and universities based on the carrier of smartphone. Security
and Communication Networks.
(13) Yu, L., Chen, Y., & Wang, L. (2021). Bibliometric analysis of research
literature on ideological and political education by curriculum based on
sati. Journal of Physics: Conference Series, 1852(4), 042072.
(14) Li, X., Dong, Y., Jiang, Y., & Ogunmola, G. A. (2021). Analysis of the teaching
quality of college ideological and political education based on deep
learning. Journal of Interconnection Networks.
(15) Sun, J., & University, N. N. (2020). Classical marxist writers' discussions on
the subject of ideological and political education. Journal of Northwest
Normal University(Social Sciences).
(16) Yu, C. (2019). Kuram ve uygulamada etm blmler educational sciences:
theory & practice 2989 innovative methods of ideological and political
education for college students based on ideological cognition science.
(17) Cheng, P., Yang, L., Niu, T., & Li, B. (2022). On the ideological and political
education of material specialty courses under the background of the
internet. Journal of Higher Education Research, 3(1), 79-82.
(18) Zhou, Y. (2022). The application of curriculum ideology and politics in the
training of judicial vocational education talents. Journal of Higher Education
Research, 3(2), 155-159.
(19) Wu, X. (2022). Research on the reform of ideological and political teaching
evaluation method of college english course based on "online and offline"
teaching. Journal of Higher Education Research, 3(1), 87-90.
(20) Chen, B., & Pan, X. (2022). Practice and exploration of integrating
curriculum ideology and politics into digital logic circuit course. Open
Journal of Social Sciences, 10(4), 8.
(21) Bai X, Wang X, Liu X. (2021). Explainable Deep Learning for Efficient and
Robust Pattern Recognition: A Survey of Recent Developments. Pattern
Recognition, 108102.
(22) Shafay, M., Ahmad, R. W., Salah, K., Yaqoob, I., Jayaraman, R., & Omar, M.
(2022). Blockchain for deep learning: review and open challenges. Cluster
Computing, 1-25.
(23) Wang C, Zhou J, Xiao B. (2021). Uncertainty Estimation for Stereo Matching
Based on Evidential Deep Learning. Pattern Recognition, 1. https://doi.org/
10.1016/j.patcog.2021.108498
(24) W. Cai, D. Liu, X. Ning. (2021). Voxel-based Three-view Hybrid Parallel
Network for 3D Object Classification. Displays, 69(1).
(25) Shan W. (2022). Digital streaming media distribution and transmission
process optimisation based on adaptive recurrent neural network[J].
Connection Science, 34(1), 1169-1180.
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
124
(11) Si, Y. (2022). On the influence of party history education on ideological and
political education for medical students. Journal of Higher Education
Research, 3(2), 141-145.
(12) Tao, T., & Lv, X. (2022). Construction of ideological and political education
in colleges and universities based on the carrier of smartphone. Security
and Communication Networks.
(13) Yu, L., Chen, Y., & Wang, L. (2021). Bibliometric analysis of research
literature on ideological and political education by curriculum based on
sati. Journal of Physics: Conference Series, 1852(4), 042072.
(14) Li, X., Dong, Y., Jiang, Y., & Ogunmola, G. A. (2021). Analysis of the teaching
quality of college ideological and political education based on deep
learning. Journal of Interconnection Networks.
(15) Sun, J., & University, N. N. (2020). Classical marxist writers' discussions on
the subject of ideological and political education. Journal of Northwest
Normal University(Social Sciences).
(16) Yu, C. (2019). Kuram ve uygulamada etm blmler educational sciences:
theory & practice 2989 innovative methods of ideological and political
education for college students based on ideological cognition science.
(17) Cheng, P., Yang, L., Niu, T., & Li, B. (2022). On the ideological and political
education of material specialty courses under the background of the
internet. Journal of Higher Education Research, 3(1), 79-82.
(18) Zhou, Y. (2022). The application of curriculum ideology and politics in the
training of judicial vocational education talents. Journal of Higher Education
Research, 3(2), 155-159.
(19) Wu, X. (2022). Research on the reform of ideological and political teaching
evaluation method of college english course based on "online and offline"
teaching. Journal of Higher Education Research, 3(1), 87-90.
(20) Chen, B., & Pan, X. (2022). Practice and exploration of integrating
curriculum ideology and politics into digital logic circuit course. Open
Journal of Social Sciences, 10(4), 8.
(21) Bai X, Wang X, Liu X. (2021). Explainable Deep Learning for Efficient and
Robust Pattern Recognition: A Survey of Recent Developments. Pattern
Recognition, 108102.
(22) Shafay, M., Ahmad, R. W., Salah, K., Yaqoob, I., Jayaraman, R., & Omar, M.
(2022). Blockchain for deep learning: review and open challenges. Cluster
Computing, 1-25.
(23) Wang C, Zhou J, Xiao B. (2021). Uncertainty Estimation for Stereo Matching
Based on Evidential Deep Learning. Pattern Recognition, 1. https://doi.org/
10.1016/j.patcog.2021.108498
(24) W. Cai, D. Liu, X. Ning. (2021). Voxel-based Three-view Hybrid Parallel
Network for 3D Object Classification. Displays, 69(1).
(25) Shan W. (2022). Digital streaming media distribution and transmission
process optimisation based on adaptive recurrent neural network[J].
Connection Science, 34(1), 1169-1180.
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
(26) Ying L, Qian Nan Z, Fu Ping W. (2021). Adaptive weights learning in CNN
feature fusion for crime scene investigation image classification[J].
Connection Science, 33(3), 719-734.
(27) Kaseng, F., Lezama, P., Inquilla, R., y Rodriguez, C. (2020). Evolution and
advance usage of Internet in Peru.
3C TIC. Cuadernos de desarrollo aplicados
a las TIC, 9(4), 113-127. https://doi.org/10.17993/3ctic.2020.94.113-127
(28) Guan Fuyu,Cao Jie,Ren Jie & Song Wenli. (2021). The teaching of sports
science of track and field-based on nonlinear mathematical equations.
Applied Mathematics and Nonlinear Sciences, 7(1), 191-198. https://doi.org/
10.2478/amns.2021.2.00155
https://doi.org/10.17993/3ctecno.2023.v12n1e43.108-125
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.43 | Iss.12 | N.1 January - March 2023
125