最新消息:20210917 已从crifan.com换到crifan.org

【已解决】Mac中安装Mitmdump和启动服务

Mac crifan 515浏览 0评论
折腾:
【未解决】Mac中Python的uiautomator2报错:tasklist: command not found
期间,需要去mac中安装Mitmdump及启动服务。
安装后,终端中就有如下命令可用了:
mitmproxy, mitmdump or mitmweb
直接去安装:
brew install mitmproxy
详细log
✘ ⚙ xxx@xxx  ~  brew install mitmproxy
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
abcm2ps             bulk_extractor      dive                gatsby-cli          kallisto            mariadb             now-cli             prettier            stanford-corenlp    wtfutil
adwaita-icon-theme  bundletool          dvc                 geoipupdate         kops                mariadb@10.1        nut                 proj                subversion
annie               byacc               dynare              git-archive-all     kubectx             mariadb@10.2        octant              pspg                suite-sparse
arangodb            ceres-solver        envconsul           gradle              lazygit             mariadb@10.3        octave              pulumi              sundials
armadillo           cheat               exploitdb           grpc                lego                mkvtoolnix          onefetch            qbs                 swiftlint
assimp              consul-template     eye-d3              ipmiutil            lerna               monero              petsc               rethinkdb           tektoncd-cli
badtouch            convox              folly               janet               libplist            mysql-client        petsc-complex       scamper             tunnel
blast               dartsim             fop                 jenkins             librealsense        navi                pioneer             sfst                ungit
boot-clj            deployer            frugal              jhipster            libssh              neo4j               plantuml            sparse              vault
broot               dita-ot             futhark             jo                  lmod                nlohmann-json       platformio          sratoolkit          vulkan-headers
==> Deleted Formulae
pound


==> Installing dependencies for mitmproxy: protobuf, gdbm, sqlite, xz and python
==> Installing mitmproxy dependency: protobuf
==> Downloading https://homebrew.bintray.com/bottles/protobuf-3.10.0.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/b1/b13a84b99a3c1655b6c0b1dacd7290b22ef8eb6ad321e5ce29372d4718e736be?__gda__=exp=1573462076~hmac=9876d33fa26012b064d0e0b91c296d89e3c952bbc96e96a79bbb508
######################################################################## 100.0%
==> Pouring protobuf-3.10.0.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/protobuf/3.10.0: 268 files, 19.5MB
==> Installing mitmproxy dependency: gdbm
==> Downloading https://homebrew.bintray.com/bottles/gdbm-1.18.1.mojave.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring gdbm-1.18.1.mojave.bottle.1.tar.gz
🍺  /usr/local/Cellar/gdbm/1.18.1: 20 files, 586.8KB
==> Installing mitmproxy dependency: sqlite
==> Downloading https://homebrew.bintray.com/bottles/sqlite-3.30.1.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/5e/5e6fef2d754e0e4009d502c40ad1846ac46937886b5f3fe89378cb838626d95e?__gda__=exp=1573462089~hmac=c6ff1f8cf01cb3cd353dd998d03e9f47babe61db763b20a53efc77e
######################################################################## 100.0%
==> Pouring sqlite-3.30.1.mojave.bottle.tar.gz
==> Caveats
sqlite is keg-only, which means it was not symlinked into /usr/local,
because macOS provides an older sqlite3.


If you need to have sqlite first in your PATH run:
  echo 'export PATH="/usr/local/opt/sqlite/bin:$PATH"' >> ~/.zshrc


For compilers to find sqlite you may need to set:
  export LDFLAGS="-L/usr/local/opt/sqlite/lib"
  export CPPFLAGS="-I/usr/local/opt/sqlite/include"


For pkg-config to find sqlite you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/sqlite/lib/pkgconfig"


==> Summary
🍺  /usr/local/Cellar/sqlite/3.30.1: 11 files, 3.9MB
==> Installing mitmproxy dependency: xz
==> Downloading https://homebrew.bintray.com/bottles/xz-5.2.4.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/01/010667293df282c8bceede3bcd36953dd57c56cef608d09a5b50694ab7d4b96b?__gda__=exp=1573462093~hmac=b953edf17dfad0dd8b1baeac87edf64431969905338e6bb621d7526
######################################################################## 100.0%
==> Pouring xz-5.2.4.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/xz/5.2.4: 92 files, 1MB
==> Installing mitmproxy dependency: python
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.5.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/bc/bcbdc3612a10d06facca7ec9499265731d530c8273b1a2dc28f16258d4410552?__gda__=exp=1573462097~hmac=bc3254129ae5a40b58d49576ae404987d65006ea5179250d2960e00
######################################################################## 100.0%
==> Pouring python-3.7.5.mojave.bottle.tar.gz
==> /usr/local/Cellar/python/3.7.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.5/bin --install-lib=/usr/local/lib/python3.7/site-pac
==> /usr/local/Cellar/python/3.7.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.5/bin --install-lib=/usr/local/lib/python3.7/site-pac
==> /usr/local/Cellar/python/3.7.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/3.7.5/bin --install-lib=/usr/local/lib/python3.7/site-pac
==> Caveats
Python has been installed as
  /usr/local/bin/python3


Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin


If you need Homebrew's Python 2.7 run
  brew install python@2


You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages


See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python/3.7.5: 3,972 files, 60.7MB
==> Installing mitmproxy
==> Downloading https://homebrew.bintray.com/bottles/mitmproxy-4.0.4_3.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/7d/7d6eb3b85bbab030dd305f8df023df97b15eab5dbb176ee9868ca56a26531c50?__gda__=exp=1573462116~hmac=ae22309e2aac8607648c9e77e440533b7566bcc050674340dd325f5
######################################################################## 100.0%
==> Pouring mitmproxy-4.0.4_3.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/mitmproxy/4.0.4_3: 1,623 files, 22.3MB
==> Caveats
==> sqlite
sqlite is keg-only, which means it was not symlinked into /usr/local,
because macOS provides an older sqlite3.


If you need to have sqlite first in your PATH run:
  echo 'export PATH="/usr/local/opt/sqlite/bin:$PATH"' >> ~/.zshrc


For compilers to find sqlite you may need to set:
  export LDFLAGS="-L/usr/local/opt/sqlite/lib"
  export CPPFLAGS="-I/usr/local/opt/sqlite/include"


For pkg-config to find sqlite you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/sqlite/lib/pkgconfig"


==> python
Python has been installed as
  /usr/local/bin/python3


Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin


If you need Homebrew's Python 2.7 run
  brew install python@2


You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages


See: https://docs.brew.sh/Homebrew-and-Python
然后去看看:
⚙ xxx@xxx  ~  ll /usr/local/bin/mitm*
lrwxr-xr-x  1 xxx  admin    40B 11 11 16:36 /usr/local/bin/mitmdump -> ../Cellar/mitmproxy/4.0.4_3/bin/mitmdump
lrwxr-xr-x  1 xxx  admin    41B 11 11 16:36 /usr/local/bin/mitmproxy -> ../Cellar/mitmproxy/4.0.4_3/bin/mitmproxy
lrwxr-xr-x  1 xxx  admin    39B 11 11 16:36 /usr/local/bin/mitmweb -> ../Cellar/mitmproxy/4.0.4_3/bin/mitmweb
⚙ xxx@xxx  ~  mitmproxy --version
Mitmproxy: 4.0.4
Python:    3.7.5
OpenSSL:   OpenSSL 1.1.1d  10 Sep 2019
Platform:  Darwin-18.7.0-x86_64-i386-64bit
⚙ xxx@xxx  ~  mitmdump --version
Mitmproxy: 4.0.4
Python:    3.7.5
OpenSSL:   OpenSSL 1.1.1d  10 Sep 2019
Platform:  Darwin-18.7.0-x86_64-i386-64bit
⚙ xxx@xxx  ~  mitmweb --version
Mitmproxy: 4.0.4
Python:    3.7.5
OpenSSL:   OpenSSL 1.1.1d  10 Sep 2019
Platform:  Darwin-18.7.0-x86_64-i386-64bit
看看介绍:
  • mitmproxy is an interactive man-in-the-middle proxy for HTTP and HTTPS with a console interface.
  • mitmdump is the command-line version of mitmproxy. Think tcpdump for HTTP.
  • mitmweb is a web-based interface for mitmproxy.
【总结】
mac中安装mitmdump:
先要知道:
  • mitmproxy
    • 是什么:抓包程序
    • =命令行版本的Charles/Fiddler
    • 特点
      • 支持HTTP和HTTPS
  • mitmdump
    • 命令行版的mitmproxy
      • 提供了命令行接口
        • 去操作mitmproxy
      • 类比:
        • tcpdump之于HTTP
  • mitmweb
    • 网页版的mitmproxy
      • 方便以网页形式查看内容
        • 甚至是修改内容
即:需要安装mitmproxy,即可有mitmdump
一般来说,macOS中安装mitmproxy,可以通过brew安装:
brew install mitmproxy
就可以了。
不过对于后续使用期间,调用python脚本包含额外的库,比如yaml:
关于如何去启动服务:
mitmdump -p 8081 -s middleware/Save1.py
其中:
(1)此处的Save1.py是个python脚本
具体内容:
# _*_ coding: utf-8 _*_

import json
import re
import os
import sys
# print("sys.executable=%s" % sys.executable)


try:
    import yaml
except Exception as err:
    print("Failed to import yaml: %s" % err)


class Saver:
    
    def __init__(self):
        self.Allurls = set()
        self.DataFilePath = self.get_DataFilePath()
        self.REMOVED = self.get_NeedSkip()
    
    def get_DataFilePath(self):
        # SavePath = "./middleware/Save.json"
        SavePath = os.path.join("middleware", "Save.json")
        with open(SavePath,"r",encoding="utf-8") as f:
            text = f.read()
            data = json.loads(text)
            return data["1"]
        
    def get_NeedSkip(self):
        # filepath = "./middleware/config.yml"
        filepath = os.path.join("middleware", "config.yml")
        try:
            with open(filepath, "r", encoding="utf-8") as f:
                text = f.read()
        except Exception:
            with open(filepath, "r") as f:
                text = f.read()
        config = yaml.load(text)
        REMOVED = [item.replace('.','\.') for item in config["skip"]]
        return "|".join(REMOVED)
    
    def get_ContentType(self, headers):
        ContentType = "None"
        patten = "b'Accept', b'(.*?)'"
        result = re.search(patten, headers)
        if result:
            ContentType = result.group(1)
            ContentType = ContentType.split(",")[0]
        return ContentType if not "*" in ContentType else "None"
    
    def request(self, flow):
        url = flow.request.url
        ContentType = self.get_ContentType(str(flow.request.headers))
        if not url in self.Allurls and not re.search(self.REMOVED, url):
            self.Allurls.add(url)
            print(url)
            with open(self.DataFilePath, "a", encoding="utf-8") as f:
                f.write(url + "|" + ContentType)
                f.write('\n')

addons = [Saver()]
(2)若想要后台运行,则后面加&
mitmdump -p 8081 -s middleware/Save1.py &
即可。
通过brew安装的mitmproxy,会调用自己内部安装的python(此处是3.7.5)
而不是Mac中自己Python(2.7或3.8),mac中的python中都安装过yaml了
而mitmproxy中python,没有安装过yaml,所以上述脚本会报错
No module named yaml
所以解决办法是:
不要用brew安装,而是用系统中的python的pip去安装mitmproxy
pip install mitmproxy
注:系统中的python是,此处是用的3.8,用pyenv设置全局为3.8
另外此处2.7的python中,pip安装mitmproxy会失败。
之后即可正常调用
mitmdump -p 8081 -s middleware/Save1.py
其中python解析器用的是此处系统的python了。
以及可以正常找到(系统中python中已安装的)yaml了。
详见:
【基本解决】Mac中mitmdump运行命令报错:in script py No module named yaml

转载请注明:在路上 » 【已解决】Mac中安装Mitmdump和启动服务

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
92 queries in 0.183 seconds, using 23.33MB memory