Skip to content

S
stm-complaint-monthly-statistics
Commit 19646cd9 authored by Akerke Makhmut's avatar Akerke Makhmut
Browse files
Options

Initial commit

parents
Pipeline #7631 canceled with stages
      Hide whitespace changes
      Inline Side-by-side
      Showing with 1101 additions and 0 deletions
      .gitignore 0 → 100644
      View file @ 19646cd9
      # Byte-compiled / optimized / DLL files
      __pycache__/
      *.py[cod]
      *$py.class
      # C extensions
      *.so
      # Distribution / packaging
      .Python
      build/
      develop-eggs/
      dist/
      downloads/
      eggs/
      .eggs/
      lib/
      lib64/
      parts/
      sdist/
      var/
      wheels/
      pip-wheel-metadata/
      share/python-wheels/
      *.egg-info/
      .installed.cfg
      *.egg
      MANIFEST
      # PyInstaller
      # Usually these files are written by a python script from a template
      # before PyInstaller builds the exe, so as to inject date/other infos into it.
      *.manifest
      *.spec
      # Installer logs
      pip-log.txt
      pip-delete-this-directory.txt
      # Unit test / coverage reports
      htmlcov/
      .tox/
      .nox/
      .coverage
      .coverage.*
      .cache
      nosetests.xml
      coverage.xml
      *.cover
      *.py,cover
      .hypothesis/
      .pytest_cache/
      # Translations
      *.mo
      *.pot
      # Django stuff:
      *.log
      local_settings.py
      db.sqlite3
      db.sqlite3-journal
      # Flask stuff:
      instance/
      .webassets-cache
      # Scrapy stuff:
      .scrapy
      # Sphinx documentation
      docs/_build/
      # PyBuilder
      target/
      # Jupyter Notebook
      .ipynb_checkpoints
      # IPython
      profile_default/
      ipython_config.py
      # pyenv
      .python-version
      # pipenv
      # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
      # However, in case of collaboration, if having platform-specific dependencies or dependencies
      # having no cross-platform support, pipenv may install dependencies that don't work, or not
      # install all needed dependencies.
      #Pipfile.lock
      # PEP 582; used by e.g. github.com/David-OConnor/pyflow
      __pypackages__/
      # Celery stuff
      celerybeat-schedule
      celerybeat.pid
      # SageMath parsed files
      *.sage.py
      # Environments
      .env
      .venv
      env/
      venv/
      ENV/
      env.bak/
      venv.bak/
      # Spyder project settings
      .spyderproject
      .spyproject
      # Rope project settings
      .ropeproject
      # mkdocs documentation
      /site
      # mypy
      .mypy_cache/
      .dmypy.json
      dmypy.json
      # Pyre type checker
      .pyre/
      \ No newline at end of file
      README.md 0 → 100644
      View file @ 19646cd9
      # Alem STM monthly statistics of complaint
      Monthly statistics of complaint to evaluate the quality of STM models(stm-complaint-identifier).
      Using library psycopg2 and pandas.
      ## Installation
      Install all requirements using
      `pip install -r requirements.txt`
      Then run python script main.py
      \ No newline at end of file
      complainant_list.txt 0 → 100644
      View file @ 19646cd9
      https://www.facebook.com/gaziztortbay
      https://www.facebook.com/NurtasAdambay
      https://www.instagram.com/nurtasadambay
      https://www.facebook.com/aruzhan.sain
      https://www.instagram.com/aruzhan_sain
      https://www.facebook.com/murataben
      https://www.instagram.com/muratabenov
      https://www.facebook.com/profile.php?id=100007447955436
      https://www.instagram.com/jokharutebekov
      https://www.facebook.com/mukhtar.taizhan
      https://www.facebook.com/seimargulan
      https://www.instagram.com/margulan_seissembai
      https://www.facebook.com/yelikbayev
      https://www.instagram.com/yelikbayev
      https://t.me/yelikbayev
      https://www.facebook.com/asselbayandarova
      https://www.instagram.com/asselbayandar
      https://www.facebook.com/profile.php?id=100005174520907
      https://www.instagram.com/mukhamedzhan_tazabek
      https://www.facebook.com/yrashev
      https://www.instagram.com/yrashev
      https://t.me/yrashev
      https://www.facebook.com/rakhimbaev
      https://www.instagram.com/rakhimbaevaidyn
      https://www.facebook.com/mukhtar.ablyazov
      https://www.facebook.com/dosyms
      https://www.instagram.com/dsatpayev
      https://www.facebook.com/serik.abasshakh
      https://www.instagram.com/serik_abasshakh_officialpage
      https://www.facebook.com/profile.php?id=100007981087790
      https://www.instagram.com/aigul_orynbek_
      https://www.facebook.com/profile.php?id=100001010054707
      https://www.instagram.com/saule_abedinova
      https://www.facebook.com/profile.php?id=100001010054707
      https://www.facebook.com/syrym.abdrakhmanov
      https://www.instagram.com/syrymabdrakhmanov
      https://www.facebook.com/aidos.sarym
      https://www.facebook.com/profile.php?id=1309358542
      https://www.facebook.com/beibit.alibekov
      https://www.instagram.com/alibekovkz
      https://www.facebook.com/janna.akhmetova.9
      https://www.facebook.com/beysenkuranbek
      https://www.facebook.com/aizhan.hamit
      https://www.facebook.com/profile.php?id=100001137038283
      https://www.facebook.com/akossanovkz
      https://www.facebook.com/BagdatMussin
      https://www.instagram.com/bagdatmussin
      https://www.facebook.com/aida.jexen
      https://www.instagram.com/aidajexen
      https://www.facebook.com/adella.smith
      https://www.instagram.com/adelsmith
      https://www.facebook.com/sayasat
      https://www.instagram.com/sayasatnurbek
      https://www.facebook.com/profile.php?id=100001377979947
      https://www.instagram.com/akhmetbeknursila_
      https://www.facebook.com/vadim.boreiko
      https://www.instagram.com/vadim.boreiko
      https://www.facebook.com/azamat.sulu
      https://www.instagram.com/gauharutelbaeva
      https://www.facebook.com/zhanbolat.mamay
      https://www.instagram.com/mamayzhanbolat
      https://www.facebook.com/zhanna.ilicheva
      https://www.facebook.com/auyesbay
      https://www.instagram.com/qanattan
      https://www.facebook.com/sadyrbay
      https://www.instagram.com/askhatsadyrbay
      https://www.facebook.com/alem.faighters
      https://www.facebook.com/alikhan.baimenov
      https://www.instagram.com/baimen.alikhan
      https://www.facebook.com/profile.php?id=100000820171797
      https://www.instagram.com/beibutsarybai_spaicial
      https://www.facebook.com/dinara.satzhan
      https://www.facebook.com/sholpan.ali
      https://www.instagram.com/sholpanali
      https://www.facebook.com/profile.php?id=100004690417636
      https://www.instagram.com/dinaelgezek
      https://www.facebook.com/bahyt.syzdykova
      https://www.facebook.com/marat.tolibayev.rus
      https://www.instagram.com/tolibayevmarat
      https://www.facebook.com/sterling.kz
      https://www.instagram.com/deniskrivosheyev
      https://www.facebook.com/aiman.umarova
      https://www.facebook.com/guljan.tama
      https://www.facebook.com/ddos.attack.77
      https://www.facebook.com/profile.php?id=100011178488080
      https://www.facebook.com/kanat.tassibekov
      https://www.facebook.com/kazybek.isa
      https://www.facebook.com/inga.imanbay
      https://www.facebook.com/gani.kassymov2
      https://www.facebook.com/jolimbet.makishev
      https://www.facebook.com/darhan.hidirali
      https://www.facebook.com/aigul.solovyeva
      https://www.facebook.com/profile.php?id=100000571768189
      https://www.facebook.com/e.sairov
      https://www.facebook.com/profile.php?id=100002257610561
      https://www.facebook.com/profile.php?id=100009134098700
      https://www.facebook.com/a.baitasov
      https://www.facebook.com/profile.php?id=100001971955237
      https://www.facebook.com/kuat.dombai
      https://www.facebook.com/zhasaral.kuan
      https://www.facebook.com/profile.php?id=100001059647434
      https://www.facebook.com/azimbay.gali
      https://www.facebook.com/salta.virgilio
      https://www.facebook.com/aidynolzhaev
      https://www.facebook.com/sultanbek.sultangaliev
      https://www.facebook.com/azamatkhan.amirtay
      https://www.facebook.com/berik.abdygaliuly
      https://www.instagram.com/isenova/?hl=ru
      https://www.youtube.com/channel/UC7YT0kUzs8dmAxZqOeiFakw
      https://www.facebook.com/profile.php?id=1629020939
      https://www.facebook.com/profile.php?id=100025135371568
      https://www.facebook.com/tolganaj.umbetalieva
      https://www.facebook.com/burikhan.nurmukhamedov
      https://www.facebook.com/bota.jardemalie.1
      https://www.facebook.com/battalova
      https://www.facebook.com/makhat.sadyk
      https://www.facebook.com/aizhan.hamit
      https://www.facebook.com/zharylkap.kalybay
      https://www.facebook.com/profile.php?id=100001132596778
      https://www.facebook.com/m.yeleussizov
      https://www.facebook.com/profile.php?id=100001544868184
      https://www.facebook.com/kadyrzanov
      https://www.facebook.com/profile.php?id=100002912954870
      https://www.facebook.com/dauren.kuat
      https://www.facebook.com/oraz.alimbekov
      https://www.facebook.com/profile.php?id=100001238430195
      https://www.facebook.com/profile.php?id=100001791234500
      https://www.facebook.com/ulansh
      https://www.facebook.com/abisenbayev
      https://www.facebook.com/svoik
      https://www.facebook.com/karashuk
      https://www.instagram.com/bayzakova
      https://www.instagram.com/meirzhach_tv
      https://www.instagram.com/nagimuwaa
      https://www.facebook.com/dkb74
      https://www.facebook.com/galym.baituk
      https://www.facebook.com/ir.petrushova
      https://www.facebook.com/marat.uatkan
      https://www.facebook.com/ruslan.erbota
      https://www.facebook.com/kazhigulova
      https://www.facebook.com/mukhtar.senggirbay
      https://www.facebook.com/profile.php?id=100003403233694
      https://www.facebook.com/kborash
      https://www.facebook.com/bolat.smi
      https://www.facebook.com/profile.php?id=100010550858469
      https://www.facebook.com/profile.php?id=100005107905833
      https://www.facebook.com/aruzhan.sain
      https://www.facebook.com/murataben
      https://www.facebook.com/mukhtar.taizhan
      https://www.facebook.com/yrashev
      https://www.facebook.com/syrym.abdrakhmanov
      https://www.facebook.com/aidos.sarym
      https://www.facebook.com/dosyms
      https://www.facebook.com/janna.akhmetova.9
      https://www.facebook.com/beibit.alibekov
      https://www.facebook.com/zhanna.ilicheva
      https://www.facebook.com/profile.php?id=100007981087790
      https://www.facebook.com/profile.php?id=100001137038283
      https://www.facebook.com/karin.erlan
      https://www.facebook.com/sholpan.ali
      https://www.facebook.com/sayasat
      https://www.facebook.com/galym.baituk
      https://www.facebook.com/gulnara.bazhkenova
      https://www.facebook.com/aida.jexen
      https://www.facebook.com/rakhim.oshakbayev
      https://www.facebook.com/MahabbatESEN
      https://www.facebook.com/anigmetov
      https://www.facebook.com/talgat.kaliyev
      https://www.facebook.com/beknurcom
      https://www.facebook.com/omirzhan.abdihalyh?ref=br_rs
      https://www.facebook.com/askar.japarov
      https://www.facebook.com/profile.php?id=100001607753833
      https://www.facebook.com/abdizhapparov
      https://www.facebook.com/e.sairov
      https://www.facebook.com/o.khudaibergenov
      https://www.facebook.com/Altayev.Online/
      https://www.facebook.com/yelikbayev
      https://www.facebook.com/anuarnurpeissov
      https://www.facebook.com/profile.php?id=1309358542
      https://www.facebook.com/KudaibergenKairat
      https://www.facebook.com/gaziztortbay
      https://www.facebook.com/seimargulan
      https://www.facebook.com/rakhimbaev
      https://www.facebook.com/marat.tolibayev.rus
      https://www.facebook.com/zeinulla.kakimzhanov
      https://www.facebook.com/suleimenov
      https://www.facebook.com/profile.php?id=100025362450708
      https://www.facebook.com/marat.bakkulov
      https://www.facebook.com/mukanov.scco
      https://www.facebook.com/profile.php?id=100012459317728
      https://www.facebook.com/mukhoryapov?ref=br_rs
      https://www.facebook.com/abbasovmiras
      https://www.facebook.com/a.baitasov?ref=br_rs
      https://www.facebook.com/ibraimov
      https://www.facebook.com/bolatbek.aliyev?ref=br_rs
      https://www.facebook.com/kairat.mazhibayev
      https://www.facebook.com/profile.php?id=100006280989434&ref=br_rs
      https://www.facebook.com/shakhanov
      https://www.facebook.com/MuhtarAblazov
      https://www.facebook.com/akossanovkz
      https://www.facebook.com/zhanbolat.mamay
      https://www.facebook.com/guljan.tama
      https://www.facebook.com/sergey.duvanov.14
      https://www.facebook.com/serikzhan.mambetalin
      https://www.facebook.com/inga.imanbay
      https://www.facebook.com/profile.php?id=100011569305686
      https://www.facebook.com/bulat.atabayev
      https://www.facebook.com/leila.khrapunova
      https://www.facebook.com/ayansharip
      https://www.facebook.com/battalova
      https://www.facebook.com/aydos.sadykov
      https://www.facebook.com/psychea7
      https://www.facebook.com/profile.php?id=100000678976508
      https://www.facebook.com/svoik
      https://www.facebook.com/mahambet2017
      https://www.facebook.com/lukpanchik
      https://www.facebook.com/profile.php?id=100000703637445
      https://www.facebook.com/ir.petrushova?ref=br_rs
      https://www.facebook.com/Elbacity.press
      https://www.facebook.com/serik.abasshakh
      https://www.facebook.com/100002165244684
      https://www.facebook.com/1080857456
      https://www.facebook.com/100002245213610
      https://www.facebook.com/4401876
      https://www.facebook.com/100003364372126
      https://www.facebook.com/1529028214
      https://www.facebook.com/100002352469612
      https://www.facebook.com/100000712167966
      https://www.facebook.com/100000785314601
      https://www.facebook.com/100001138387024
      https://www.facebook.com/100007981087790
      https://www.facebook.com/100001137038283
      https://www.facebook.com/100035453503339
      https://www.facebook.com/100002304468925
      https://www.facebook.com/511427997
      https://www.facebook.com/100001873373266
      https://www.facebook.com/100002559177756
      https://www.facebook.com/100002602097140
      https://www.facebook.com/100000534280007
      https://www.facebook.com/570766666
      https://www.facebook.com/1458541877
      https://www.facebook.com/100000815740810
      https://www.facebook.com/100003344807836
      https://www.facebook.com/1239295897
      https://www.facebook.com/100000796572126
      https://www.facebook.com/100001607753833
      https://www.facebook.com/100003418466127
      https://www.facebook.com/100001014586146
      https://www.facebook.com/100013736706014
      https://www.facebook.com/372098733644112
      https://www.facebook.com/570885949
      https://www.facebook.com/1344593204
      https://www.facebook.com/1309358542
      https://www.facebook.com/100001384945242
      https://www.facebook.com/100000622298146
      https://www.facebook.com/1667112463
      https://www.facebook.com/100006773412357
      https://www.facebook.com/100001533059079
      https://www.facebook.com/100006451400893
      https://www.facebook.com/12331353
      https://www.facebook.com/100025362450708
      https://www.facebook.com/100004054543844
      https://www.facebook.com/100007005212135
      https://www.facebook.com/100012459317728
      https://www.facebook.com/100001573778290
      https://www.facebook.com/100000680485871
      https://www.facebook.com/100001146006725
      https://www.facebook.com/649639903
      https://www.facebook.com/100002522644830
      https://www.facebook.com/100003790667556
      https://www.facebook.com/100006280989434
      https://www.facebook.com/187627844663126
      https://www.facebook.com/177385019097311
      https://www.facebook.com/100001741778707
      https://www.facebook.com/100000985927750
      https://www.facebook.com/100004945740471
      https://www.facebook.com/100001240674075
      https://www.facebook.com/730471102
      https://www.facebook.com/100001300204657
      https://www.facebook.com/100011569305686
      https://www.facebook.com/100001425365046
      https://www.facebook.com/100004635300771
      https://www.facebook.com/100001639461840
      https://www.facebook.com/1323615255
      https://www.facebook.com/100003894706734
      https://www.facebook.com/100003422037379
      https://www.facebook.com/100000678976508
      https://www.facebook.com/100000171367602
      https://www.facebook.com/100002194936830
      https://www.facebook.com/100000459877449
      https://www.facebook.com/100000703637445
      https://www.facebook.com/100005118844895
      https://www.facebook.com/195184244315139
      https://www.facebook.com/100001153087084
      https://t.me/nonamestan
      https://telegram.me/house_of_cards_kz
      https://t.me/tokaev_glasses
      https://telegram.me/vlastkz
      https://t.me/s/aqsaqalnotes
      https://t.me/Lysinaa
      https://t.me/geo_2019
      https://t.me/aqsaqalkz
      https://t.me/oyanqazaqstan
      https://t.me/valdaiclub
      https://t.me/selteyev_view
      https://t.me/shatalkanal
      https://t.me/GenPr
      https://t.me/kazonline
      https://t.me/astana109112
      https://t.me/shezire
      https://t.me/azattygtany
      https://t.me/nurotanchannel
      https://t.me/shadowcab
      https://t.me/QazChannel1
      https://t.me/Alinasadym
      https://telegram.me/bakittibala
      https://t.me/dvkvideo
      https://t.me/karlag_v2
      https://t.me/delirium_kz
      https://t.me/alzhanovchannel
      https://telegram.me/bakittiboidak
      https://t.me/qpolls
      https://t.me/dvkshtab
      https://t.me/elbasa01
      https://t.me/volonteerrdvk
      https://t.me/chernovik01
      https://t.me/rukh2k19
      https://t.me/shugylakz
      https://t.me/vksnews
      https://t.me/s/Delo_mam
      https://t.me/shatalkanal
      https://t.me/shugylakz
      https://t.me/banks_of_Kazakhstan
      https://t.me/qyzyqstan
      https://t.me/shonbay
      https://t.me/karagandagoo
      https://t.me/atamekenbusiness
      https://t.me/tengenomika
      https://t.me/strogiiagashka
      https://t.me/balgabaev_rinat
      https://t.me/qazaqstories
      https://t.me/tipavloshock
      https://ttttt.me/BILIMNEWSS
      https://t.me/biznesbastau
      https://t.me/financekaz
      https://telegram.me/kazebitda
      https://www.instagram.com/taraz_v_obektive
      https://www.instagram.com/taraz_life
      https://www.instagram.com/zellotaraz
      https://www.instagram.com/vtvs_taraz
      https://www.instagram.com/onlinetaraz
      https://www.instagram.com/taraz_today
      https://www.instagram.com/kris.p.taraz
      https://www.facebook.com/profile.php?id=100010253200942
      https://www.facebook.com/profile.php?id=100003257380163
      https://www.facebook.com/profile.php?id=100003528069497
      https://www.facebook.com/profile.php?id=100011333649773
      https://www.facebook.com/profile.php?id=100009394546001
      https://www.facebook.com/profile.php?id=100010411426905
      https://www.facebook.com/profile.php?id=100011386286168
      https://www.facebook.com/profile.php?id=100006973481607
      https://www.facebook.com/profile.php?id=100002365172383
      https://www.facebook.com/profile.php?id=100008258132211
      https://www.facebook.com/profile.php?id=100001183779083
      https://www.facebook.com/profile.php?id=100007874474268
      https://www.facebook.com/profile.php?id=100005700399422
      https://www.facebook.com/profile.php?id=100005250876623
      https://www.facebook.com/profile.php?id=100002699386765
      https://www.facebook.com/profile.php?id=100007541023938
      https://www.facebook.com/profile.php?id=100006137914515
      https://www.facebook.com/profile.php?id=100003044169970
      https://www.facebook.com/profile.php?id=100005495055568
      https://www.facebook.com/profile.php?id=100002010592476
      https://www.facebook.com/profile.php?id=100003313552212
      https://www.facebook.com/profile.php?id=100007651868363
      https://www.facebook.com/profile.php?id=100002549873492
      https://www.facebook.com/profile.php?id=100008147526736
      https://www.facebook.com/profile.php?id=100011101237605
      https://www.facebook.com/profile.php?id=100003393333950
      https://www.facebook.com/profile.php?id=100008823271490
      https://www.facebook.com/profile.php?id=100004424494168
      https://www.facebook.com/profile.php?id=100007192691199
      https://www.facebook.com/profile.php?id=100012511185240
      https://www.facebook.com/profile.php?id=100001197434004
      https://www.facebook.com/profile.php?id=100001586707356
      https://www.facebook.com/profile.php?id=100017723141771
      https://www.facebook.com/profile.php?id=100013314043394
      https://www.facebook.com/profile.php?id=100012458558061
      https://www.facebook.com/profile.php?id=100007206944542
      https://www.facebook.com/profile.php?id=100000602960517
      https://www.facebook.com/profile.php?id=100004446005723
      https://www.facebook.com/profile.php?id=100014020365938
      https://www.facebook.com/profile.php?id=100007644212988
      https://www.facebook.com/profile.php?id=100012357720979
      https://www.facebook.com/profile.php?id=100016708847968
      https://www.facebook.com/profile.php?id=100005298798559
      https://www.facebook.com/profile.php?id=100002854438606
      https://www.facebook.com/profile.php?id=100008109031321
      https://www.facebook.com/profile.php?id=100001171959294
      https://www.facebook.com/profile.php?id=100023864563332
      https://www.facebook.com/profile.php?id=100001280528156
      https://www.facebook.com/profile.php?id=100001524621907
      https://www.facebook.com/profile.php?id=100004323962631
      https://www.facebook.com/profile.php?id=100011499890649
      https://www.facebook.com/profile.php?id=100021950499809
      https://www.facebook.com/profile.php?id=100001195629385
      https://www.facebook.com/profile.php?id=100001168419350
      https://www.facebook.com/profile.php?id=100003726192559
      https://www.facebook.com/profile.php?id=100001984450083
      https://www.facebook.com/profile.php?id=100003234595614
      https://www.facebook.com/profile.php?id=100029611879462
      https://www.facebook.com/profile.php?id=100009431954961
      https://www.facebook.com/profile.php?id=100009645844163
      https://www.facebook.com/profile.php?id=100001782734537
      https://www.facebook.com/profile.php?id=100024477819547
      https://www.facebook.com/profile.php?id=100001857951768
      https://www.facebook.com/profile.php?id=100004603446657
      https://www.facebook.com/profile.php?id=100007774812904
      https://www.facebook.com/profile.php?id=100008870852166
      https://www.facebook.com/profile.php?id=100001872700963
      https://www.facebook.com/profile.php?id=100006925656878
      https://www.facebook.com/profile.php?id=100034752789037
      https://www.facebook.com/profile.php?id=100033035580537
      https://www.facebook.com/profile.php?id=100022904678274
      https://www.facebook.com/profile.php?id=100002681260956
      https://www.facebook.com/profile.php?id=100023017176413
      https://www.facebook.com/profile.php?id=100004884965839
      https://www.facebook.com/profile.php?id=100021832687656
      https://www.facebook.com/profile.php?id=100012310757967
      https://www.facebook.com/profile.php?id=100032803217030
      https://www.facebook.com/profile.php?id=100001172139610
      https://www.facebook.com/profile.php?id=100000481449275
      https://www.facebook.com/profile.php?id=100005422394394
      https://www.facebook.com/profile.php?id=100009478759086
      https://www.facebook.com/profile.php?id=100000696817018
      https://www.facebook.com/profile.php?id=100007012150660
      https://www.facebook.com/profile.php?id=100007869100947
      https://www.facebook.com/profile.php?id=100001191069192
      https://www.facebook.com/profile.php?id=100023278057294
      https://www.facebook.com/profile.php?id=1766074137
      https://www.facebook.com/profile.php?id=100001740980521
      https://www.facebook.com/profile.php?id=100023825711035
      https://www.facebook.com/profile.php?id=100005292488016
      https://www.facebook.com/profile.php?id=100005234322134
      https://www.facebook.com/profile.php?id=100003528735250
      https://www.facebook.com/profile.php?id=100010281788867
      https://www.facebook.com/profile.php?id=100010941427717
      https://www.facebook.com/profile.php?id=100013511760256
      https://www.facebook.com/profile.php?id=100014516466084
      https://www.facebook.com/profile.php?id=100020692553681
      https://www.facebook.com/profile.php?id=100002572170147
      https://www.facebook.com/profile.php?id=100001176577346
      https://www.facebook.com/profile.php?id=100007392169266
      https://www.facebook.com/profile.php?id=100011501980154
      https://www.facebook.com/profile.php?id=100025721145060
      https://www.facebook.com/profile.php?id=100002024817843
      https://www.facebook.com/profile.php?id=100013980372641
      https://www.facebook.com/profile.php?id=100011506781097
      https://www.facebook.com/profile.php?id=100034339489440
      https://www.facebook.com/profile.php?id=100010489573294
      https://www.facebook.com/profile.php?id=100009999913288
      https://www.facebook.com/profile.php?id=100022461669016
      https://www.facebook.com/profile.php?id=100024093363749
      https://www.facebook.com/profile.php?id=100024114300345
      https://www.facebook.com/profile.php?id=100007545932567
      https://www.facebook.com/profile.php?id=100001775893933
      https://www.facebook.com/profile.php?id=100003976933679
      https://www.facebook.com/profile.php?id=100004823658617
      https://www.facebook.com/profile.php?id=100028334623232
      https://www.facebook.com/profile.php?id=100002915595545
      https://www.facebook.com/profile.php?id=100027444426222
      https://www.facebook.com/profile.php?id=100028250829824
      https://www.facebook.com/profile.php?id=100027344864363
      https://www.facebook.com/profile.php?id=100011902604751
      https://www.facebook.com/profile.php?id=100030689690509
      https://www.facebook.com/profile.php?id=100026941063133
      https://www.facebook.com/profile.php?id=100003530328284
      https://www.facebook.com/profile.php?id=100012116979107
      https://www.facebook.com/profile.php?id=100022773200186
      https://www.facebook.com/profile.php?id=100030508742197
      https://www.facebook.com/profile.php?id=100035249354452
      https://www.facebook.com/profile.php?id=100005174415769
      https://www.facebook.com/profile.php?id=100003108875021
      https://www.facebook.com/profile.php?id=100006558888578
      https://www.facebook.com/profile.php?id=100001871133074
      https://www.facebook.com/profile.php?id=100028183964316
      https://www.facebook.com/profile.php?id=100035115683543
      https://www.facebook.com/profile.php?id=100015405335038
      https://www.facebook.com/profile.php?id=100003409664869
      https://www.facebook.com/profile.php?id=100035115683543
      https://www.facebook.com/profile.php?id=100003429464802
      https://www.facebook.com/profile.php?id=100019148498334
      https://www.facebook.com/profile.php?id=100019148498334
      https://www.facebook.com/profile.php?id=100035476563607
      https://www.facebook.com/profile.php?id=100013368940463
      https://www.facebook.com/profile.php?id=100006605985835
      https://www.facebook.com/profile.php?id=100002511372774
      https://www.facebook.com/profile.php?id=100005008789250
      https://www.facebook.com/profile.php?id=100003360673845
      https://www.facebook.com/profile.php?id=100001576383581
      https://www.facebook.com/profile.php?id=100034561294049
      https://www.facebook.com/profile.php?id=100034529623338
      https://www.facebook.com/profile.php?id=100033489376870
      https://www.facebook.com/profile.php?id=100001937878407
      https://www.facebook.com/profile.php?id=100010487624506
      https://www.facebook.com/profile.php?id=100001686298303
      https://www.facebook.com/profile.php?id=100028872812939
      https://www.facebook.com/profile.php?id=100035293170350
      https://www.facebook.com/profile.php?id=100026391712592
      https://www.facebook.com/profile.php?id=100008253274559
      https://www.facebook.com/profile.php?id=100034549412131
      https://www.facebook.com/profile.php?id=100033030979847
      https://www.facebook.com/profile.php?id=100035759582298
      https://www.facebook.com/profile.php?id=100006944553336
      https://www.facebook.com/profile.php?id=100030328693704
      https://www.facebook.com/profile.php?id=100006323345106
      https://www.facebook.com/profile.php?id=100011670932903
      https://www.facebook.com/profile.php?id=100011074099189
      https://www.facebook.com/profile.php?id=100000850613635
      https://www.facebook.com/profile.php?id=100010096304645
      https://www.facebook.com/profile.php?id=100023535486415
      https://www.facebook.com/profile.php?id=100010331567689
      https://www.facebook.com/profile.php?id=100016158610104
      https://www.facebook.com/profile.php?id=100017758771391
      https://www.facebook.com/profile.php?id=100030384062005
      https://www.facebook.com/profile.php?id=100016892495834
      https://www.facebook.com/profile.php?id=100034238468184
      https://www.facebook.com/profile.php?id=100010572104410
      https://www.facebook.com/profile.php?id=100024589790557
      https://www.facebook.com/profile.php?id=100002983658228
      https://www.facebook.com/profile.php?id=100002994123437
      https://www.facebook.com/profile.php?id=100005652068692
      https://www.facebook.com/profile.php?id=100014082128416
      https://www.facebook.com/profile.php?id=100009201655537
      https://www.facebook.com/profile.php?id=100027590362221
      https://www.facebook.com/profile.php?id=100009564105527
      https://www.facebook.com/profile.php?id=100010422510189
      https://www.facebook.com/profile.php?id=100006106272686
      https://www.facebook.com/profile.php?id=100033147692445
      https://www.facebook.com/profile.php?id=100004429350510
      https://www.facebook.com/profile.php?id=100008663050450
      https://www.facebook.com/profile.php?id=100007590756123
      https://www.facebook.com/profile.php?id=100003992166971
      https://www.facebook.com/profile.php?id=100033960893381
      https://www.facebook.com/profile.php?id=100001119999391
      https://www.facebook.com/profile.php?id=100033280219569
      https://www.facebook.com/profile.php?id=100024359690959
      https://www.facebook.com/profile.php?id=100030331132723
      https://www.facebook.com/profile.php?id=100002368107109
      https://www.facebook.com/profile.php?id=100021910334534
      https://www.facebook.com/profile.php?id=100001171929272
      https://www.facebook.com/profile.php?id=100018674768632
      https://www.facebook.com/profile.php?id=100011956392883
      https://www.facebook.com/profile.php?id=100023994520674
      https://www.facebook.com/profile.php?id=100023947336587
      https://www.facebook.com/profile.php?id=100001197429373
      https://www.facebook.com/profile.php?id=100003765814250
      https://www.facebook.com/profile.php?id=100003130844212
      https://www.facebook.com/profile.php?id=100028575970583
      https://www.facebook.com/profile.php?id=100003435408802
      https://www.facebook.com/profile.php?id=100034080470482
      https://www.facebook.com/profile.php?id=100010432561229
      https://www.facebook.com/profile.php?id=100025121057451
      https://www.facebook.com/profile.php?id=100029272763083
      https://www.facebook.com/profile.php?id=100030713274236
      https://www.facebook.com/profile.php?id=100004787314634
      https://www.facebook.com/profile.php?id=100011580816661
      https://www.facebook.com/profile.php?id=100003947902715
      https://www.facebook.com/profile.php?id=100010812394053
      https://www.facebook.com/profile.php?id=100006511785643
      https://www.facebook.com/profile.php?id=100005602690857
      https://www.facebook.com/profile.php?id=100010125405300
      https://www.facebook.com/profile.php?id=100007559679025
      https://www.facebook.com/profile.php?id=100001760698656
      https://www.facebook.com/profile.php?id=100004478676916
      https://www.facebook.com/profile.php?id=100024836565364
      https://www.facebook.com/profile.php?id=100004055753061
      https://www.facebook.com/profile.php?id=100003560258802
      https://www.facebook.com/profile.php?id=100005664811609
      https://www.facebook.com/profile.php?id=100007851709440
      https://www.facebook.com/profile.php?id=100014878896068
      https://www.facebook.com/profile.php?id=100012945481943
      https://www.facebook.com/100010253200942
      https://www.facebook.com/100003257380163
      https://www.facebook.com/100003528069497
      https://www.facebook.com/100011333649773
      https://www.facebook.com/100009394546001
      https://www.facebook.com/100010411426905
      https://www.facebook.com/100011386286168
      https://www.facebook.com/100006973481607
      https://www.facebook.com/100002365172383
      https://www.facebook.com/100008258132211
      https://www.facebook.com/100001183779083
      https://www.facebook.com/100007874474268
      https://www.facebook.com/100005700399422
      https://www.facebook.com/100005250876623
      https://www.facebook.com/100002699386765
      https://www.facebook.com/100007541023938
      https://www.facebook.com/100006137914515
      https://www.facebook.com/100003044169970
      https://www.facebook.com/100005495055568
      https://www.facebook.com/100002010592476
      https://www.facebook.com/100003313552212
      https://www.facebook.com/100007651868363
      https://www.facebook.com/100002549873492
      https://www.facebook.com/100008147526736
      https://www.facebook.com/100011101237605
      https://www.facebook.com/100003393333950
      https://www.facebook.com/100008823271490
      https://www.facebook.com/100004424494168
      https://www.facebook.com/100007192691199
      https://www.facebook.com/100012511185240
      https://www.facebook.com/100001197434004
      https://www.facebook.com/100001586707356
      https://www.facebook.com/100017723141771
      https://www.facebook.com/100013314043394
      https://www.facebook.com/100012458558061
      https://www.facebook.com/100007206944542
      https://www.facebook.com/100000602960517
      https://www.facebook.com/100004446005723
      https://www.facebook.com/100014020365938
      https://www.facebook.com/100007644212988
      https://www.facebook.com/100012357720979
      https://www.facebook.com/100016708847968
      https://www.facebook.com/100005298798559
      https://www.facebook.com/100002854438606
      https://www.facebook.com/100008109031321
      https://www.facebook.com/100001171959294
      https://www.facebook.com/100023864563332
      https://www.facebook.com/100001280528156
      https://www.facebook.com/100001524621907
      https://www.facebook.com/100004323962631
      https://www.facebook.com/100011499890649
      https://www.facebook.com/100021950499809
      https://www.facebook.com/100001195629385
      https://www.facebook.com/100001168419350
      https://www.facebook.com/100003726192559
      https://www.facebook.com/100001984450083
      https://www.facebook.com/100003234595614
      https://www.facebook.com/100029611879462
      https://www.facebook.com/100009431954961
      https://www.facebook.com/100009645844163
      https://www.facebook.com/100001782734537
      https://www.facebook.com/100024477819547
      https://www.facebook.com/100001857951768
      https://www.facebook.com/100004603446657
      https://www.facebook.com/100007774812904
      https://www.facebook.com/100008870852166
      https://www.facebook.com/100001872700963
      https://www.facebook.com/100006925656878
      https://www.facebook.com/100034752789037
      https://www.facebook.com/100033035580537
      https://www.facebook.com/100022904678274
      https://www.facebook.com/100002681260956
      https://www.facebook.com/100023017176413
      https://www.facebook.com/100004884965839
      https://www.facebook.com/100021832687656
      https://www.facebook.com/100012310757967
      https://www.facebook.com/100032803217030
      https://www.facebook.com/100001172139610
      https://www.facebook.com/100000481449275
      https://www.facebook.com/100005422394394
      https://www.facebook.com/100009478759086
      https://www.facebook.com/100000696817018
      https://www.facebook.com/100007012150660
      https://www.facebook.com/100007869100947
      https://www.facebook.com/100001191069192
      https://www.facebook.com/100023278057294
      https://www.facebook.com/1766074137
      https://www.facebook.com/100001740980521
      https://www.facebook.com/100023825711035
      https://www.facebook.com/100005292488016
      https://www.facebook.com/100005234322134
      https://www.facebook.com/100003528735250
      https://www.facebook.com/100010281788867
      https://www.facebook.com/100010941427717
      https://www.facebook.com/100013511760256
      https://www.facebook.com/100014516466084
      https://www.facebook.com/100020692553681
      https://www.facebook.com/100002572170147
      https://www.facebook.com/100001176577346
      https://www.facebook.com/100007392169266
      https://www.facebook.com/100011501980154
      https://www.facebook.com/100025721145060
      https://www.facebook.com/100002024817843
      https://www.facebook.com/100013980372641
      https://www.facebook.com/100011506781097
      https://www.facebook.com/100034339489440
      https://www.facebook.com/100010489573294
      https://www.facebook.com/100009999913288
      https://www.facebook.com/100022461669016
      https://www.facebook.com/100024093363749
      https://www.facebook.com/100024114300345
      https://www.facebook.com/100007545932567
      https://www.facebook.com/100001775893933
      https://www.facebook.com/100003976933679
      https://www.facebook.com/100004823658617
      https://www.facebook.com/100028334623232
      https://www.facebook.com/100002915595545
      https://www.facebook.com/100027444426222
      https://www.facebook.com/100028250829824
      https://www.facebook.com/100027344864363
      https://www.facebook.com/100011902604751
      https://www.facebook.com/100030689690509
      https://www.facebook.com/100026941063133
      https://www.facebook.com/100003530328284
      https://www.facebook.com/100012116979107
      https://www.facebook.com/100022773200186
      https://www.facebook.com/100030508742197
      https://www.facebook.com/100035249354452
      https://www.facebook.com/100005174415769
      https://www.facebook.com/100003108875021
      https://www.facebook.com/100006558888578
      https://www.facebook.com/100001871133074
      https://www.facebook.com/100028183964316
      https://www.facebook.com/100035115683543
      https://www.facebook.com/100015405335038
      https://www.facebook.com/100003409664869
      https://www.facebook.com/100035115683543
      https://www.facebook.com/100003429464802
      https://www.facebook.com/100019148498334
      https://www.facebook.com/100019148498334
      https://www.facebook.com/100035476563607
      https://www.facebook.com/100013368940463
      https://www.facebook.com/100006605985835
      https://www.facebook.com/100002511372774
      https://www.facebook.com/100005008789250
      https://www.facebook.com/100003360673845
      https://www.facebook.com/100001576383581
      https://www.facebook.com/100034561294049
      https://www.facebook.com/100034529623338
      https://www.facebook.com/100033489376870
      https://www.facebook.com/100001937878407
      https://www.facebook.com/100010487624506
      https://www.facebook.com/100001686298303
      https://www.facebook.com/100028872812939
      https://www.facebook.com/100035293170350
      https://www.facebook.com/100026391712592
      https://www.facebook.com/100008253274559
      https://www.facebook.com/100034549412131
      https://www.facebook.com/100033030979847
      https://www.facebook.com/100035759582298
      https://www.facebook.com/100006944553336
      https://www.facebook.com/100030328693704
      https://www.facebook.com/100006323345106
      https://www.facebook.com/100011670932903
      https://www.facebook.com/100011074099189
      https://www.facebook.com/100000850613635
      https://www.facebook.com/100010096304645
      https://www.facebook.com/100023535486415
      https://www.facebook.com/100010331567689
      https://www.facebook.com/100016158610104
      https://www.facebook.com/100017758771391
      https://www.facebook.com/100030384062005
      https://www.facebook.com/100016892495834
      https://www.facebook.com/100034238468184
      https://www.facebook.com/100010572104410
      https://www.facebook.com/100024589790557
      https://www.facebook.com/100002983658228
      https://www.facebook.com/100002994123437
      https://www.facebook.com/100005652068692
      https://www.facebook.com/100014082128416
      https://www.facebook.com/100009201655537
      https://www.facebook.com/100027590362221
      https://www.facebook.com/100009564105527
      https://www.facebook.com/100010422510189
      https://www.facebook.com/100006106272686
      https://www.facebook.com/100033147692445
      https://www.facebook.com/100004429350510
      https://www.facebook.com/100008663050450
      https://www.facebook.com/100007590756123
      https://www.facebook.com/100003992166971
      https://www.facebook.com/100033960893381
      https://www.facebook.com/100001119999391
      https://www.facebook.com/100033280219569
      https://www.facebook.com/100024359690959
      https://www.facebook.com/100030331132723
      https://www.facebook.com/100002368107109
      https://www.facebook.com/100021910334534
      https://www.facebook.com/100001171929272
      https://www.facebook.com/100018674768632
      https://www.facebook.com/100011956392883
      https://www.facebook.com/100023994520674
      https://www.facebook.com/100023947336587
      https://www.facebook.com/100001197429373
      https://www.facebook.com/100003765814250
      https://www.facebook.com/100003130844212
      https://www.facebook.com/100028575970583
      https://www.facebook.com/100003435408802
      https://www.facebook.com/100034080470482
      https://www.facebook.com/100010432561229
      https://www.facebook.com/100025121057451
      https://www.facebook.com/100029272763083
      https://www.facebook.com/100030713274236
      https://www.facebook.com/100004787314634
      https://www.facebook.com/100011580816661
      https://www.facebook.com/100003947902715
      https://www.facebook.com/100010812394053
      https://www.facebook.com/100006511785643
      https://www.facebook.com/100005602690857
      https://www.facebook.com/100010125405300
      https://www.facebook.com/100007559679025
      https://www.facebook.com/100001760698656
      https://www.facebook.com/100004478676916
      https://www.facebook.com/100024836565364
      https://www.facebook.com/100004055753061
      https://www.facebook.com/100003560258802
      https://www.facebook.com/100005664811609
      https://www.facebook.com/100007851709440
      https://www.facebook.com/100014878896068
      https://www.facebook.com/100012945481943
      database.py 0 → 100644
      View file @ 19646cd9
      import psycopg2
      import pandas as pd
      def database_request(date_start, date_end):
      """
      подключение к базе stm staging используя библиотеуку psycopg2
      """
      conn = psycopg2.connect(
      host="10.30.11.144",
      port="5432",
      database="stm",
      user="postgres",
      password="xjx6nKn4tCXmWMeC",
      )
      cur = conn.cursor()
      """
      запрос для получения определенных колон с таблицы post и post_group
      """
      query = """SELECT post.post_group_id, post.status, post.author_url, post.text,
      post_group.characteristics FROM post_group
      INNER JOIN post ON post_group.id = post.post_group_id
      WHERE post.created_at > %s and post.created_at < %s """
      cur.execute(query, (date_start, date_end))
      """
      переформатирую полученный запрос в dataframe и возвращаю тот же dataframe
      """
      sql_data = pd.DataFrame(cur.fetchall())
      sql_data
      sql_data.columns=[ x.name for x in cur.description ]
      cur.close()
      conn.close()
      return sql_data
      \ No newline at end of file
      identifier.py 0 → 100644
      View file @ 19646cd9
      names_list = open("complainant_list.txt", "r").read().splitlines()
      def is_kazakh_language(text: list) -> bool:
      kaz_translit_words = """жагдай, кайырлы, деп, мен, ол, деген, менин, сол, керек,
      болып, жане, жауап, жумыс, неге, жок, маган, емес, барып, ушин, шара, акша, сагат, жыл, жок, отыр, екен, бирак,
      арыз, жайлы, шагым, емес, калай, куни, ауыр, ешкандай, жатыр, салеметсизбе, саламатсызба, салеметсиздерме,
      саламатсыздарма, саламатсыз ба, салеметсиз бе, калай, кашан, бир, жылы, жумыс, гана, кызмет, сайын, дейн, шейн,
      немесе, негизи, кайта, сиз, бир, еки, уш, алты, жети, сегиз, тогыз, жуз, мын, окиниш, окинишке, орай, сойлемеш,
      жылы, шара, керек, бирак, кайырлы, болып, кайта, кашан, кызмет, жайлы, гана, орай, мен, сол, ушин, жане, неге,
      таяк, жакта, журсын, бершы, кошеде, отыр, уйде, коріп, койып, мына, бармайсын, уйыхтайсын, кирсей"""
      kaz_translit_words = set(kaz_translit_words.split(', '))
      set_of_words = set(text)
      count_of_translit_words = len(set_of_words.intersection(kaz_translit_words))
      count_of_native_words, _ = _get_number_of_kz_words(text=text, threshold=0)
      threshold_of_kz_words = 0.15 * len(set_of_words)
      return threshold_of_kz_words < count_of_translit_words or count_of_native_words > threshold_of_kz_words
      def _get_number_of_kz_words(text: list, threshold=0) -> tuple:
      """
      detect words with kazakh symbols
      Keywords arguments :
      text -- input text
      threshold -- the minimum number of kz symbols in word
      return number of words with kazakh words
      """
      kz_symbols = 'әіңғүұқөһ'
      words_with_kz = [word for word in text if len(set(word).intersection(set(kz_symbols))) > threshold]
      return len(words_with_kz), words_with_kz
      \ No newline at end of file
      main.py 0 → 100644
      View file @ 19646cd9
      from database import database_request
      import pandas as pd
      from identifier import names_list, is_kazakh_language
      """
      будет две даты:
      1. дата начало создание поста в базе
      2. дата окончание создание поста в базе
      например статистика за весь АВГУСТ
      """
      date_start = '2021-08-01'
      date_end = '2021-09-01'
      """
      отправляем даты в функцию для получения данных с базы в виде dataframe
      """
      df = database_request(date_start, date_end)
      ### 1 что нужно отобразить в интерфейсе
      print("общее количество постов: ", len(df))
      df.drop_duplicates(subset='post_group_id', inplace=True)
      ### 2 что нужно отобразить в интерфейсе количество уникальных постов полученные по post_group_id
      print("уникальных постов:", len(df))
      ### 3 что нужно отобразить в интерфейсе количество неразмеченных постов
      print("неразмеченных постов со статусом '-1':", len(df[df['status'] == -1]))
      df['exist'] = df['author_url'].apply(lambda x : True if x.lower() in names_list else False)
      ### 4 что нужно отобразить в интерфейсе количество постов написанные ЛОМами
      print("общее количество постов от ЛОМов", len(df[df['exist'] == True]))
      ### 5 что нужно отобразить в интерфейсе количество постов от ЛОМов которые были размечены
      print("посты от ЛОМов без статуса '-1':", len(df[(df['exist'] == True) & (df['status'] != -1)]))
      df = df[df['exist'] == False]
      df['characteristics'] = df['characteristics'].astype('str')
      df['lang'] = df['text'].apply(lambda x : is_kazakh_language(x))
      ### 6 что нужно отобразить в интерфейсе количество постов на русском языке
      print("посты на русском языке:", len(df[df['lang'] == False]))
      ### 7 что нужно отобразить в интерфейсе количество НЕразмеченных постов на русском языке
      print("количество постов на русском языке со статусом '-1'", len(df[(df['lang'] == False) & (df['status'] == -1)]))
      ### 8 что нужно отобразить в интерфейсе количество постов на казахском языке
      print("посты на казахском языке:", len(df[df['lang'] == True]))
      ### 9 что нужно отобразить в интерфейсе количество НЕразмеченных постов на казахском языке
      print("количество постов на казахском языке со статусом '-1'", len(df[(df['lang'] == True) & (df['status'] == -1)]))
      df_ru = df[(df['lang'] == False) & (df['status'] != -1)]
      df_kz = df[(df['lang'] == True) & (df['status'] != -1)]
      ### 10 что нужно отобразить в интерфейсе размеченные посты сгруппированные по характеристикам на русском языке
      print("посты на русском по характеристике", "\n", df_ru.groupby("characteristics").count())
      ### 11 что нужно отобразить в интерфейсе размеченные посты сгруппированные по статусам на русском языке
      print("посты на русском по статусу", "\n", df_ru.groupby("status").count())
      ### 12 что нужно отобразить в интерфейсе размеченные посты сгруппированные по характеристикам на казахском языке
      print("посты на казахском по характеристике", "\n", df_kz.groupby("characteristics").count())
      ### 13 что нужно отобразить в интерфейсе размеченные посты сгруппированные по статусам на казахском языке
      print("посты на казахском по статусу", "\n", df_kz.groupby("status").count())
      \ No newline at end of file
      requirements.txt 0 → 100644
      View file @ 19646cd9
      psycopg2==2.8.6
      pandas==0.25.3
      \ No newline at end of file
      Markdown is supported
      0% or
      You are about to add 0 people to the discussion. Proceed with caution.
      Finish editing this message first!
      Please register or to comment