Some weights of the model checkpoint at bert-base-multilingual-cased were not used when initializing

 

분류를 위한 BERT 모델 생성

• transformers의 BertForSequenceClassification 모듈을 이용중 해당 문구가 떴다.

Some weights of the model checkpoint at bert-base-multilingual-cased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.predictions.transfor
m.dense.weight',
 'cls.predictions.transfor
m.LayerNorm.bias',
 'cls.predictions.transfor
m.dense.bias',
 'cls.seq_relationship.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transfor
m.LayerNorm.weight',
 'cls.predictions.bias']
- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-multilingual-cased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

 

Bert-base-multilingual-cased에서 모델 체크포인트의 일부 가중치는 BertForSequenceClassification을 초기화할 때 사용되지 않았다고 한다..

 

 

this is just a remark by the Huggingface library - no need to worry. We are using the BERT implementation of Huggingface internally. You are doing everything correctly here. When executing the train code (as you do), you train JEREX (and fine-tune into BERT) on a down-stream task (end-to-end relation extraction) and you can then use the model for prediction.

 

->

Huggingface 라이브러리의 리마크 이고 걱정할 필요가 없다고 한다. 내부적으로 Huggingface의 BERT 구현을 사용하고 있고 모든 것을 올바르게 수행하고 있다고 한다.

 

답변 출처 : https://github.com/lavis-nlp/jerex/issues/2