最新消息:20210917 已从crifan.com换到crifan.org

【已解决】pipenv虚拟环境中用pip安装pyspider出错:__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using

pyspider crifan 2429浏览 0评论

折腾:

【已解决】使用PySpider去爬取某网站中的视频

期间,去在pipenv中用pip去安装pyspider出错:

<code>➜  xx pip install pyspider
Collecting pyspider
Collecting cssselect&gt;=0.9 (from pyspider)
  Using cached https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Collecting lxml (from pyspider)
  Using cached https://files.pythonhosted.org/packages/00/fd/5e65f293e366a63198dade275b886e5d24752367c2e67e3993023b0d58ef/lxml-4.2.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting chardet&gt;=2.2 (from pyspider)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting tornado&lt;=4.5.3,&gt;=3.2 (from pyspider)
Collecting Jinja2&gt;=2.7 (from pyspider)
  Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting pycurl (from pyspider)
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    100% |████████████████████████████████| 215kB 117kB/s
    Complete output from command python setup.py egg_info:
    Using curl-config (libcurl 7.54.0)
    Traceback (most recent call last):
      File "&lt;string&gt;", line 1, in &lt;module&gt;
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-dsu2ziyg/pycurl/setup.py", line 913, in &lt;module&gt;
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-dsu2ziyg/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-dsu2ziyg/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-dsu2ziyg/pycurl/setup.py", line 316, in configure_unix
        specify the SSL backend manually.''')
    __main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-dsu2ziyg/pycurl/
</code>

pipenv __main__.ConfigurationError: Curl is configured to use SSL

pipenv install of pycurl produces error with link-time ssl backend vs compile-time ssl backend · Issue #1284 · pypa/pipenv

给此处生成的:

Pipfile

的:

<code>[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.6"

</code>

加上:

<code>[packages]
pycurl = "{ with-openssl }"
</code>

结果:

问题依旧

去换成

<code>[packages]
pycurl = "{ with-openssl }"
pyspider
</code>

结果

<code>➜  xx pipenv install
Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/project.py", line 417, in _parse_pipfile
    return contoml.loads(contents)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/contoml/__init__.py", line 15, in loads
    elements = parse_tokens(tokens)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/prettytoml/parser/__init__.py", line 17, in parse_tokens
    return _parse_token_stream(TokenStream(tokens))
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/prettytoml/parser/__init__.py", line 32, in _parse_token_stream
    raise ParsingError('Failed to parse line {}'.format(pending.head.row))
prettytoml.parser.errors.ParsingError: Failed to parse line 8

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/bin/pipenv", line 11, in &lt;module&gt;
    sys.exit(cli())
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/cli.py", line 366, in install
    selective_upgrade=selective_upgrade,
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/core.py", line 1730, in do_install
    skip_requirements=skip_requirements,
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/core.py", line 608, in ensure_project
    if project.required_python_version:
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/project.py", line 180, in required_python_version
    required = self.parsed_pipfile.get('requires', {}).get(
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/project.py", line 378, in parsed_pipfile
    parsed = self._parse_pipfile(contents)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/project.py", line 420, in _parse_pipfile
    return toml.loads(contents)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/vendor/toml.py", line 156, in loads
    raise TomlDecodeError("Key name found without value."
toml.TomlDecodeError: Key name found without value. Reached end of line.
</code>

好像是语法错误

试试:

pip install pycurl –global-option=”–with-openssl”

问题依旧。

<code>➜  xx pip install pycurl --with-openssl

Usage:
  pip install [options] &lt;requirement specifier&gt; [package-index-options] ...
  pip install [options] -r &lt;requirements file&gt; [package-index-options] ...
  pip install [options] [-e] &lt;vcs project url&gt; ...
  pip install [options] [-e] &lt;local project path&gt; ...
  pip install [options] &lt;archive url/path&gt; ...

no such option: --with-openssl
</code>

pip __main__.ConfigurationError: Curl is configured to use SSL

python – Trouble in installation pycurl – Stack Overflow

<code>➜  xx ll /usr/local/opt/openssl/lib
total 12184
drwxr-xr-x  14 crifan  admin   448B  3 27 21:55 engines
-r--r--r--   1 crifan  admin   1.9M  3 28 11:13 libcrypto.1.0.0.dylib
-r--r--r--   1 crifan  admin   3.2M  3 27 21:55 libcrypto.a
lrwxr-xr-x   1 crifan  admin    21B  3 27 21:55 libcrypto.dylib -&gt; libcrypto.1.0.0.dylib
-r--r--r--   1 crifan  admin   376K  3 28 11:13 libssl.1.0.0.dylib
-r--r--r--   1 crifan  admin   512K  3 27 21:55 libssl.a
lrwxr-xr-x   1 crifan  admin    18B  3 27 21:55 libssl.dylib -&gt; libssl.1.0.0.dylib
drwxr-xr-x   5 crifan  admin   160B  3 28 11:13 pkgconfig
➜  x /usr/local/opt/openssl/include
➜  x ll /usr/local/opt/openssl/include
total 0
drwxr-xr-x  77 crifan  admin   2.4K  3 28 11:13 openssl
➜  x export LDFLAGS=-L/usr/local/opt/openssl/lib
➜  x export CPPFLAGS=-I/usr/local/opt/openssl/include
➜  x pip install pycurl --with-openssl

Usage:
  pip install [options] &lt;requirement specifier&gt; [package-index-options] ...
  pip install [options] -r &lt;requirements file&gt; [package-index-options] ...
  pip install [options] [-e] &lt;vcs project url&gt; ...
  pip install [options] [-e] &lt;local project path&gt; ...
  pip install [options] &lt;archive url/path&gt; ...

no such option: --with-openssl
➜  xx pip install pycurl --global-option="--with-openssl"
/Users/crifan/.local/share/virtualenvs/xx-Ee60YgCa/lib/python3.6/site-packages/pip/_internal/commands/install.py:199: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
  cmdoptions.check_install_build_global(options)
Collecting pycurl
  Using cached https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz
    Complete output from command python setup.py egg_info:
    Using curl-config (libcurl 7.54.0)
    Traceback (most recent call last):
      File "&lt;string&gt;", line 1, in &lt;module&gt;
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-l53panp6/pycurl/setup.py", line 913, in &lt;module&gt;
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-l53panp6/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-l53panp6/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-l53panp6/pycurl/setup.py", line 316, in configure_unix
        specify the SSL backend manually.''')
    __main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pip-install-l53panp6/pycurl/
</code>

问题依旧。

Error install with new version 7.43.0.2 · Issue #526 · pycurl/pycurl

pycurl安装问题 – CSDN博客

然后看到的办法,和之前自己的:

【已解决】pyspider运行出错:ImportError pycurl libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)

一样:

<code>➜  x pip uninstall pycurl
Skipping pycurl as it is not installed.
➜  x export PYCURL_SSL_LIBRARY=openssl
➜  x export LDFLAGS=-L/usr/local/opt/openssl/lib;export CPPFLAGS=-I/usr/local/opt/openssl/include;pip install pycurl --compile --no-cache-dir
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    100% |████████████████████████████████| 215kB 6.0kB/s
Installing collected packages: pycurl
  Running setup.py install for pycurl ... done
Successfully installed pycurl-7.43.0.2
</code>

然后继续去:

<code>pipenv install pyspider
</code>

即可继续正常安装:

【总结】

(在Mac的pipenv中)安装pyspider期间,会用到pycurl,然后:

安装期间出错:

<code>__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using
</code>

或使用期间出错:

<code>ImportError pycurl libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
</code>

的话,则解决办法都是:

注意,如果用的是pipenv,则需要先进入pipenv环境:

<code>pipenv shell
</code>

然后再去运行下面的命令。

<code>pip uninstall pycurl
export PYCURL_SSL_LIBRARY=openssl
export LDFLAGS=-L/usr/local/opt/openssl/lib;export CPPFLAGS=-I/usr/local/opt/openssl/include;pip install pycurl --compile --no-cache-dir
</code>

完整log供参考:

<code>➜  xx git:(master) ✗ pipenv shell
Launching subshell in virtual environment…
. /Users/crifan/.local/share/virtualenvs/x-BXR06pHN/bin/activate
➜  x git:(master) ✗  . /Users/crifan/.local/share/virtualenvs/x-BXR06pHN/bin/activate
➜  x git:(master) ✗ pip uninstall pycurl
Skipping pycurl as it is not installed.
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
➜  x git:(master) ✗ export PYCURL_SSL_LIBRARY=openssl
➜  x git:(master) ✗ export LDFLAGS=-L/usr/local/opt/openssl/lib;export CPPFLAGS=-I/usr/local/opt/openssl/include;pip install pycurl --compile --no-cache-dir
Collecting pycurl
  Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    100% |████████████████████████████████| 215kB 146kB/s
Installing collected packages: pycurl
  Running setup.py install for pycurl ... done
Successfully installed pycurl-7.43.0.2
➜  x git:(master) ✗ pipenv install
Installing dependencies from Pipfile.lock (0f5b56)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 26/26 — 00
</code>

【后记】

【已解决】Mac中pipenv中运行PySpider出错:ImportError pycurl libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)

转载请注明:在路上 » 【已解决】pipenv虚拟环境中用pip安装pyspider出错:__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
93 queries in 0.187 seconds, using 23.47MB memory