Python Scrapy和Great Expectations(great_expectations)—不合作

发布于04月05日

我正在try 在同一虚拟环境中使用包scrapy和great_expectations.这两个包之间的兼容性似乎有问题，这取决于我导入它们的顺序.

示例:

我创建了一个虚拟环境，pip安装了每个软件包的最新版本.

有效:

import great_expectations
import scrapy

print("done")

Does not work:

import scrapy
import great_expectations

print("done")

Error:

Traceback (most recent call last):
  File 

"/Users/grant/vs_code_projects/grants_projects/test_environment.py", line 2, in <module>
    import great_expectations
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/__init__.py", line 32, in <module>
    register_core_expectations()
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/registry.py", line 187, in register_core_expectations
    from great_expectations.expectations import core  # noqa: F401
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/core/__init__.py", line 1, in <module>
    from .expect_column_distinct_values_to_be_in_set import (
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/core/expect_column_distinct_values_to_be_in_set.py", line 12, in <module>
    from great_expectations.expectations.expectation import (
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 2350, in <module>
    class BatchExpectation(Expectation, ABC):
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 287, in __new__
    newclass._register_renderer_functions()
  File "/Users/grant/Envs/test_env/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 369, in _register_renderer_functions
    attr_obj: Callable = getattr(cls, candidate_renderer_fn_name)
AttributeError: __provides__

编辑:

我试过Scrapy的所有主要版本(当前版本，2.0.0，1.0.0)与Great Expectations的所有主要版本，结果是一样的.

import scrapy def use_great_expectations(): import great_expectations # Your Great Expectations code here print("Great Expectations used") print("Scrapy and other initializations") # Other Scrapy or general code here # Now, call the function to use Great Expectations use_great_expectations()

import scrapy import lazy_import great_expectations = lazy_import.lazy_module("great_expectations") def use_great_expectations(): # Assuming `great_expectations` functionalities are required here # Now, the Great Expectations module will be actually loaded print("Using Great Expectations") # Example Great Expectations code here

为什么？

When a module is imported, Python executes all top-level statements in the module.
If these statements modify global state, or if the module's initialization logic is complex (involving dynamic imports or modifications to the module's attributes), it can lead to bugs or conflicts when combined with other modules that have their own complex initialization procedures.

Scrapy依赖于zope.interface，它可以修改类和对象的行为，其方式可能与其他库期望Python对象模型的工作方式不完全兼容.

如果zope.interface或由Scrapy的导入改变的其他组件改变了属性访问的行为(通过__getattr__、__getattribute__或__dir__等机制)，它可能会导致您观察到的AttributeError.

Python Scrapy和Great Expectations(great_expectations)—不合作

推荐答案

为什么？

Python相关问答推荐

Python plt.text中重叠，包adjust_text不起作用，如何修复？

Python在tuple上操作不会通过整个单词匹配

重新匹配{ }中包含的文本，其中文本可能包含{{var}

从dict的列中分钟

Python库：可选地支持numpy类型，而不依赖于numpy

C#使用程序从Python中执行Exec文件

我如何使法国在 map 中完全透明的代码？

Python，Fitting into a System of Equations

OR—Tools中CP—SAT求解器的IntVar设置值

对象的`call`方法的setattr在Python中不起作用'

mypy无法推断类型参数.List和Iterable的区别

Django—cte给出：QuerySet对象没有属性with_cte''''

合并帧，但不按合并键排序

需要帮助重新调整python fill_between与数据点

基于形状而非距离的两个numpy数组相似性

并行编程：同步进程

语法错误：文档. evaluate：表达式不是合法表达式

Python—在嵌套列表中添加相同索引的元素，然后计算平均值

Seaborn散点图使用多个不同的标记而不是点

如何在基于时间的数据帧中添加计算值