The basic PHP N-gram Functions

1. Function reads all n-grams of finger-prints into a multidimensional array where the first dimension is a string with the language taken from the filename of the *.lm file, following the naming convention ⟨language-encoding⟩.lm (e.g. polish-iso8859_2.lm ), the second dimension contains 400 n-grams each (all finger-prints available for download have 400 pieces of n-grams with the size of 1-to-4 letters).
<?php
function getFingerprint($dir, $nb_grams = 400) {
    $pattern = "*.lm";
    chdir($dir);
    $files = glob($pattern);
    foreach ($files as $readfile) {
        if (is_file($readfile)) {
            $bsnm = basename($readfile, ".lm");
            $handle = fopen($readfile, 'r');
            for ($i=0; $i < $nb_grams; $i++) {
                $line = fgets($handle);
                $part = explode(" ", $line);
                $lm_ng[$bsnm][]= trim($part[0]);
            }
        } 
    }
    return $lm_ng;
}
?>

2. Creates an array of n-grams of the given string ($string) that needs to be analyzed. Array contains not more than 350 of the top frequent n-grams ($ng_number), starting with the most frequent one, where the key contains the n-gram and the value contains the normalized ranking number. The third, optional parameter ($ng_max_chars) defines the max. number of characters of a n-grams. If you use the downloaded finger-prints from this website, then this parameter mustn't be changed. This function can also be used as a basis to generate a finger-print for a new language (also see: PHP Class LangDetect).
<?php
function createNGrams($string, $ng_number=350, $ng_max_chars=4) {
    $array_words = explode(" ", $string);
    //iterate over each word, each character, all possible n-grams
    foreach($array_words as $word) {
        $word = "_". $word . "_";
        $length = strlen($word);
        for ($pos=0; $pos < $length; $pos++){ //pos within word
            for ($chars=0; $chars<$ng_max_chars; $chars++) {//ng-length
                if (($pos + $chars) < $length) {//not beyond end of word
                     $array_ngram[] = substr($word, $pos, $chars+1);
                 }
             }
         }
    }
    //count-> value(frequency, int)... key(ngram, string)
    $ng_frequency = array_count_values($array_ngram);
    //sort array by value(frequency) desc
    arsort($ng_frequency);
    //use only top frequent ngrams
    $most_frequent = array_slice($ng_frequency, 0, $ng_number);
    foreach ($most_frequent as $ng => $number_frequencey){
        $sub_ng[] = $ng;
    }
    return $sub_ng;
}
?>

3. Compares array of the submitted string ($sub_ng, created in function #2) to the multidimentional finger-print array ($lm_ng, created in funtction #1) and assigns a total number of ranking-deviation points (Δ-points) to each finger-print/language. Returns ordered array with key -> basename of .lm file (language) and value -> no. of Δ-points. The optional third parameter $max_delta (may) reduce evaluation time when set smaller than 140,000 (350*400).
<?php
function compareNGrams($sub_ng, $lm_ng, $max_delta = 140000) {
    foreach ($lm_ng as $lm_basename => $language) {
        $delta = 0;
        //compare each ngram of input text to current lm-array
        foreach ($sub_ng as $key => $existing_ngram){
            //match
            if(in_array($existing_ngram, $language)) {
                $delta += abs($key - array_search($existing_ngram, $language));
            //no match
            } else {
                $delta += 400;
            }
            //abort: this language already differs too much
            if ($delta > $max_delta) {
                break;
             }
        } //end comparison with current language
        //include only non-aborted languages in result array
        if ($delta < ($max_delta - 400)) {
            $result[$lm_basename] = $delta;
        }
    } //end comparison all languages
    if(!isset($result)) {
      $result = '';
    } else {
        asort($result);
     }
    return $result;
}
?>

4. Utility function for language recognition in text files
<?php
function extractText($readfile, $limit_lines = -1) {
    $string = '';
    if (is_file($readfile)) {
        $handle = fopen($readfile, 'r');
         $line_num = 1;
        while (!feof($handle)) {
            //default -1 (read all lines)
            if ($limit_lines == $line_num){
                break;
              }
              //line with max length of 2^19
            $line = trim(fgets($handle, 528288));
            if ($line != "") {
                $string .= " ". $line;
                $line_num++;
            }
        }
        fclose($handle);
    } else {echo "*** Text file NOT FOUND<br>";}
    return $string;
}
?>


Simple DEMO example