1.1 姝ｅ悜鏈€澶у尮閰�
鎬濇兂
寰炲乏鍒板彸鐩″彲鑳藉妰鍒嗗嚭涓€娈甸€ｇ簩(x霉)瀛楃锛屼娇寰楀叾绛変簬瑭炲吀涓殑鏌愬€嬭锛岀劧鍚庡皣閫欐閫ｇ簩(x霉)瀛楃鎻愬彇鍑轰締锛屽皪浣欎笅鐨勯儴鍒嗛€茶鍚屾ǎ鐨勬搷浣�銆傚鏋滅涓€鍊嬪瓧绗︿笉鏄鍏镐腑浠讳綍涓€鍊嬭鐨勫墠缍达紝閭ｄ箞閫欏€嬪瓧绗﹀柈鐛ㄤ綔鐐轰竴鍊嬭銆�
绠楁硶
杓稿叆锛氫竴鍊嬫湭鍒嗚鐨勫彞瀛怱锛屼竴鍊嬭鍏窪
杓稿嚭锛氫竴鍊嬪簭鍒楃殑瑭�
1.2 閫嗗悜鏈€澶у尮閰�
璺熸鍚戞渶澶у尮閰嶇殑鍞竴涓嶅悓鏄緸鍙冲埌宸︾洝鍙兘鍔冨垎鍑轰竴娈甸€ｇ簩(x霉)瀛楃銆�
1.3 闆欏悜鏈€澶у尮閰�
姝х京鎸囧皪浜庝竴鍊嬪彞瀛愭湁澶氬€嬪垎瑭炵祼鏋�銆傛饥瑾炴枃鏈腑 90.0%宸﹀彸鐨勫彞瀛愶紝F(xi脿n)MM 鍜� BMM 鐨勫垏鍒嗗畬鍏ㄩ噸鍚堜笖姝ｇ⒑锛�9.0%宸﹀彸鐨勫彞瀛� FMM 鍜� BMM 鍒囧垎涓嶅悓锛屼絾鍏朵腑蹇呮湁涓€鍊嬫槸姝ｇ⒑鐨�(姝х京妾㈡脯鎴愬姛)锛屽彧鏈変笉鍒�1.0 %鐨勫彞瀛�锛屾垨鑰� FMM 鍜� BMM 鐨勫垏鍒嗛洊閲嶅悎鍗绘槸閷殑锛屾垨鑰匜MM 鍜� BMM 鍒囧垎 涓嶅悓浣嗗叐鍊嬮兘涓嶅皪(姝х京妾㈡脯澶辨晽)銆俒1]
闆欏悜鏈€澶у尮閰嶆寚寰炴鍚戞渶澶у尮閰嶅拰閫嗗悜鏈€澶у尮閰嶆硶鐨勭祼鏋滀腑閬告搰鏈€婊胯冻涓枃鍒嗚鍘熷墖鐨勪竴鍊嬪垎瑭炵祼鏋溿€傝┏瑕婣mbiguity Resolution in Chinese Word Segmentation鍜屾寤轰腑鏂囧垎瑭炲櫒 - 闆欏悜鏈€澶у尮閰嶆硶
2. 鏈€鐭矾寰戝垎瑭瀃2]
2.1 鏈€鐭矾寰戝垎瑭�/鏈€灏戣瑾炲垎瑭�
鏈€灏戣瑾炲垎瑭炴槸鍦ㄦ墍鏈夌殑鍒囧垎閬庣▼涓伕鎿囪瑾炴暩(sh霉)閲忔渶灏戠殑鍒嗚绲愭灉锛屽鏋滃嚭鐝�(xi脿n)瑭炶獮鏁�(sh霉)閲忕浉鍚岀殑鎯呮硜锛屽氨閬告搰瑭炶獮闀峰害鐨勬柟宸皬鐨勯偅鍊嬪垎瑭炵祼鏋�銆�
鏈€灏戣瑾炲垎瑭炰篃鍙渶鐭矾寰戝垎瑭烇紝鍥犵偤姹傛渶灏戣瑾炲晱椤屽彲浠ヨ畩鎴愭眰鏈夊悜鐒＄挵(hu谩n)鍦�(DAG)鐨勬渶鐭矾寰戝晱椤�銆備綅缃�(position)瀹氱京鐐烘瘡鍊嬪瓧绗﹀悗闈㈢殑浣嶇疆锛屽鏋滀竴鍊嬪彞瀛愮殑瀛楃鏄緸1鍒皀锛岄偅涔堜綅缃氨鏄緸0鍒皀锛�0鏄涓€鍊嬪瓧鍓嶉潰鐨勪綅缃�锛宯鏄渶鍚庝竴鍊嬪瓧鍚庨潰鐨勪綅缃�銆�
0浠� 1瑾� 2鐨� 3纰� 4瀵� 5鍦� 6鐞�7
2.2 鍏ㄥ垏鍒�
灏囦笂杩版墍鏈夊彲鑳界殑璺緫閮界暥浣滃垎瑭炵祼鏋滆繑鍥�锛屽氨鏄叏鍒囧垎鏂规硶銆�
2.3 鏈€澶ф鐜囧垎瑭�
鏈€澶ф鐜囧垎瑭炴槸鏈€鐭矾寰戝垎瑭炵殑璁婄ó锛屽湪鏈€鐭矾寰戜腑鎵€鏈夐倞鐨勬瑠閲嶉兘鏄�1锛屽鏋滄垜鍊戞妸閭婄殑娆婇噸鏇挎彌鎴愰倞灏嶆噳鐨勮瑾炵殑姒傜巼锛屾妸鏈€鐭矾寰戞浛鎻涙垚鏈€澶ф鐜囪矾寰�锛屽垎瑭炵殑绠楁硶灏辫畩鎴愪簡鏈€澶ф鐜囧垎瑭炰簡銆�
瑭炶獮鐨勬鐜囩磩绛変簬闋绘暩(sh霉)闄や互鎵€鏈夎鐨勭附鏁�(sh霉)锛岀敱浜庢眰鏈€澶ф鐜囬渶瑕佺敤鍒版鐜囩殑閫ｄ箻锛屽鏋滄妸姒傜巼鏇挎彌鎴愬皪鏁�(sh霉)锛岄€ｄ箻灏辫畩鎴愪簡閫ｅ姞銆�
鐩告瘮鏈€鐭矾寰戝垎瑭�锛岃鍏镐笉鍍呰瑷樿級姣忓€嬭鍖紝閭勮瑷樿級姣忓€嬭鍖嚭鐝�(xi脿n)鐨勯牷鐜�銆�
jieba鐨勫熀绀庡垎瑭炲櫒浣跨敤鐨勫氨鏄渶澶ф鐜囧垎瑭�銆�
2.4 N鏈€鐭矾寰戝垎瑭瀃3]
鏈€鐭矾寰戝垎瑭炲彧杩斿洖涓€鍊嬬祼鏋�锛屽叏鍒囧垎杩斿洖鎵€鏈夊彲鑳界殑绲愭灉锛孨鏈€鐭矾寰戝垎瑭炴槸鍏╄€呯殑鎶樹腑锛岃繑鍥炶矾寰戞渶鐭殑鍓峃鍊嬪垎瑭炵祼鏋�銆傜浉鎳夊湴锛屼篃鏈塏鏈€澶ф鐜囧垎瑭�
3. N鍏冭獮瑷€妯″瀷鍒嗚[4]
4. HMM鍒嗚[5]
鎶婂垎瑭炵暥鍋氫竴鍊嬪簭鍒楁娉ㄥ晱椤�锛屽簭鍒楀柈鍏冩槸瀛楋紝搴忓垪妯欑敖鏈塀鍜孍锛屽垎鍒ヤ唬琛ㄩ潪瑭炲熬鍜岃灏俱€傞偅涔堜竴鍊婨BEBEEBBEB搴忓垪浠ｈ〃鐨勫垎瑭炵祼鏋滃氨鏄疎 BE BE E BBE B銆�
HMM瑾炲涓嬶紝闅辩媭鎱�(t脿i)鏄簭鍒楁绨�锛岃娓簭鍒楁槸鏈垎瑭炵殑鍙ュ瓙銆傚垎瑭炲晱椤屽氨鏄В纰煎晱椤�(绲﹀畾瑙€娓簭鍒楋紝姹傛渶鏈夊彲鑳界殑鐙€鎱�(t脿i)搴忓垪)銆�
HMM鐢ㄤ簬鍙︿竴鍊嬪簭鍒楁娉ㄧ殑渚嬪瓙瑕媅6]
鍙冭€�
婕㈣獮鑷嫊鍒嗚鐮旂┒瑭曡堪 http://59.108.48.5/course/mining/12-13spring/%E5%8F%82%E8%80%83%E6%96%87%E7%8C%AE/02-01%E6%B1%89%E8%AF%AD%E8%87%AA%E5%8A%A8%E5%88%86%E8%AF%8D%E7%A0%94%E7%A9%B6%E8%AF%84%E8%BF%B0.pdf
婕㈣獮鍒嗚鍒濇帰 https://unordered.org/timelines/59cd4b0655001000
鍩轰簬 涓€ 鏈€鐭矾寰戞柟娉曠殑涓枃瑭炶獮绮楀垎妯″瀷 http://jcip.cipsc.org.cn/UserFiles/File/353鍩轰簬N_鏈€鐭矾寰戞柟娉曠殑涓枃瑭炶獮绮楀垎妯″瀷_寮佃彲骞�.pdf
涓枃鍒嗚绠楁硶绨′粙 https://lujiaying.github.io/posts/2018/01/Chinese-word-segmentation/
鐢℉MM鍋氫腑鏂囧垎瑭� http://www.52nlp.cn/itenyh鐗�-鐢╤mm鍋氫腑鏂囧垎瑭炰竴锛氭ā鍨嬫簴鍌�
闅遍Μ鐖惧彲澶ā鍨嬭鎬ф娉ㄥ強鍏禤ython瀵︾従(xi脿n) https://zhuanlan.zhihu.com/p/48260272
鍘熸枃閺堟帴锛歨ttps://zhuanlan.zhihu.com/p/92102484
鍘熸枃鍑鸿檿锛歨ttp://corpus.njau.edu.cn/wiki/006
鐩搁棞鏂囩珷闁辫畝
浠€涔堟槸瑾炴枡搴�
骞惧€嬪父瑕嬬殑婕㈣獮瑾炴枡搴垎鏋�
浜掕伅(li谩n)缍蹭笂闁嬫斁鐨勪腑鏂囪獮鏂欏韩鏈夊摢浜�
鍒嗚浠嬬垂