<_9Y�$ � @ s* d d l m Z m Z m Z d d l m Z m Z m Z d d l m Z m
Z
m Z d d l m
Z
m Z m Z d d l m Z m Z m Z Gd d � d e � Z Gd d � d e � Z Gd
d � d e � Z Gd d
� d
e � Z Gd d � d e � Z Gd d � d e � Z Gd d � d e � Z d S)� )�EUCTW_CHAR_TO_FREQ_ORDER�EUCTW_TABLE_SIZE� EUCTW_TYPICAL_DISTRIBUTION_RATIO)�EUCKR_CHAR_TO_FREQ_ORDER�EUCKR_TABLE_SIZE� EUCKR_TYPICAL_DISTRIBUTION_RATIO)�GB2312_CHAR_TO_FREQ_ORDER�GB2312_TABLE_SIZE�!GB2312_TYPICAL_DISTRIBUTION_RATIO)�BIG5_CHAR_TO_FREQ_ORDER�BIG5_TABLE_SIZE� BIG5_TYPICAL_DISTRIBUTION_RATIO)�JIS_CHAR_TO_FREQ_ORDER�JIS_TABLE_SIZE� JIS_TYPICAL_DISTRIBUTION_RATIOc @ sp e Z d Z d Z d Z d Z d Z d d � Z d d � Z d d
� Z d d � Z
d
d � Z d d � Z d S)�CharDistributionAnalysisi g�G�z��?g{�G�z�?� c C sD d | _ d | _ d | _ d | _ d | _ d | _ | j � d S)N)�_char_to_freq_order� _table_size�typical_distribution_ratio�_done� _total_chars� _freq_chars�reset)�self� r �/chardistribution.py�__init__. s z!CharDistributionAnalysis.__init__c C s d | _ d | _ d | _ d S)z reset analyser, clear any stateF� N)r r r )r r r r r = s z CharDistributionAnalysis.resetc C st | d k r | j | � } n d } | d k rp | j d 7_ | | j k rp d | j | k rp | j d 7_ d S)z"feed a character with known length� r r i N���)� get_orderr r r r )r �charZchar_len�orderr r r �feedF s z CharDistributionAnalysis.feedc C sr | j d k s! | j | j k r( | j S| j | j k rk | j | j | j | j } | | j k rk | S| j S)z(return confidence based on existing datar )r r �MINIMUM_DATA_THRESHOLD�SURE_NOr �SURE_YES)r �rr r r �get_confidenceT s ! z'CharDistributionAnalysis.get_confidencec C s | j | j k S)N)r �ENOUGH_DATA_THRESHOLD)r r r r �got_enough_datad s z(CharDistributionAnalysis.got_enough_datac C s d S)Nr r r )r �byte_strr r r r! i s z"CharDistributionAnalysis.get_orderN)
�__name__�
__module__� __qualname__r* r' r& r% r r r$ r) r+ r! r r r r r ( s r c s. e Z d Z � f d d � Z d d � Z � S)�EUCTWDistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) �superr0 r r r r r r r )r )� __class__r r r r s z"EUCTWDistributionAnalysis.__init__c C s6 | d } | d k r. d | d | d d Sd Sd S)Nr �� �^ r � r r )r r, �
first_charr r r r! x s
z#EUCTWDistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r r0 q s r0 c s. e Z d Z � f d d � Z d d � Z � S)�EUCKRDistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) r1 r7 r r r r r r r )r )r2 r r r � s z"EUCKRDistributionAnalysis.__init__c C s6 | d } | d k r. d | d | d d Sd Sd S)Nr � r4 r r5 r r )r r, r6 r r r r! � s
z#EUCKRDistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r r7 � s r7 c s. e Z d Z � f d d � Z d d � Z � S)�GB2312DistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) r1 r9 r r r r r r
r )r )r2 r r r � s z#GB2312DistributionAnalysis.__init__c C sI | d | d } } | d k rA | d k rA d | d | d Sd Sd S)Nr r r8 r5 r4 r r )r r, r6 � second_charr r r r! � s z$GB2312DistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r r9 � s r9 c s. e Z d Z � f d d � Z d d � Z � S)�Big5DistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) r1 r; r r r r r r
r )r )r2 r r r � s z!Big5DistributionAnalysis.__init__c C sd | d | d } } | d k r\ | d k rE d | d | d d Sd | d | d Sn d Sd S) Nr r � r5 � �? �@ r r )r r, r6 r: r r r r! � s z"Big5DistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r r; � s r; c s. e Z d Z � f d d � Z d d � Z � S)�SJISDistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) r1 r@ r r r r r r r )r )r2 r r r � s z!SJISDistributionAnalysis.__init__c C s� | d | d } } | d k r> | d k r> d | d } n1 | d k rk | d k rk d | d d } n d S| | d } | d
k r� d } | S)
Nr r � � � �� �� � r? � r r r )r r, r6 r: r# r r r r! � s z"SJISDistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r r@ � s r@ c s. e Z d Z � f d d � Z d d � Z � S)�EUCJPDistributionAnalysisc s2 t t | � j � t | _ t | _ t | _ d S)N) r1 rH r r r r r r r )r )r2 r r r � s z"EUCJPDistributionAnalysis.__init__c C s6 | d } | d k r. d | d | d d Sd Sd S)Nr � r4 r5 r r r )r r, r" r r r r! � s
z#EUCJPDistributionAnalysis.get_order)r- r. r/ r r! r r )r2 r rH � s rH N) Z euctwfreqr r r Z euckrfreqr r r Z
gb2312freqr r r
Zbig5freqr r r
Zjisfreqr r r �objectr r0 r7 r9 r; r@ rH r r r r �<module> s I