TOC

转载:为什么 Python 这么 “慢” 却还这么流行?

来自 Quora,原文标题:Why is Python so popular despite being so slow?

说明:这里谈的 Python,很大程度上说的是 CPython 这个标准实现,而不是 Python 这门语言。

I love this question. I really do. Mostly because I asked myself this question over, and over, and over some more throughout my time in high school, college, and at my first job as a software engineer. And I have poured countless hours doing research, watching informative and in depth presentations on things like the GIL, etc. etc. etc. just to figure out when/how/if I would run into a major problem because Python was "slow".

花费大量时间和精力就是为了弄明白大家说的这个 GIL 之类 Python 的所谓性能问题究竟在什么时候会拖累我的程序。

So before I start writing out this huge answer (mostly for my own benefit since it has been thoroughly answered by many others already) let's examine the two most common reasons that lead people to (rightfully) claim Python is slow and/or it is not high performance:

  • Interpreted
  • The GIL

大家说的最多的就这两个原因:解释型语言、GIL

The first is fairly straight forward, but at a high level compilers translate higher level languages into lower-level (faster) languages so a compiled language will almost always execute faster than non-compiled ones. There are some exceptions to this rule-of-thumb (such as situations where JIT can be faster than AOT compiling), but they are distraction to this discussion.

第一条,解释型语言通常比较慢,这大家都认可(JIT 技术不是讨论范围)。

The second is a bit more notorious, but Python has something called the Global Interpreter Lock
which basically prevents multi-threading by mandating the interpreter only execute a single thread within a single process (instance of the Python interpreter) at a time. How this works can be pretty interesting

as well, but also a tangential rabbit hole like compilers.

第二条,GIL 限制真正的并发。

9/10 times the slower performance of Python does not matter.

Over time I have come to the opinion that there are two main reasons for this.

The first is that what matters is not code execution time, but end-user experience. It doesn't usually matter if a function takes 0.001 seconds or 0.01 to execute. In this vein, for most problems horizontal scaling can be used to solve many bottlenecks that would have been created by Python.

大多数情况下,架构上的横向拓展可以抵消 Python 的性能缺陷。

Take for example these benchmarks for popular web frameworks. The best one using Python came in at 650.5K req/s while the best number was 2.2M. Purely from a performance perspective, you are probably wondering why you wouldn't just pick the fastest, but then you look at that #1 spot and realize it is using C. C is a great language (IMO), but Python is a lot more expressive and has a larger eco-system with pre-built tools you could choose to use. So instead of squeezing every last bit of computing power out of your server while sacrificing development time/scope, you could instead get 4 servers for Python for every 1 needing C and save many times that in developer productivity and development time. This is obviously a dramatized and drastically simplified example, but the point being illustrated I think is sound.

根据一些基准测试,基于 Python 某些框架能承受 650K req/s,其他的产品则最好能达到 2200K。如果你要选择那些性能更好的技术去尽可能压榨机器计算能力,你需要放弃的是 Python 代码的编码效率和庞大的生态。
或者,你也可以选择 Python,牺牲掉一部分程序运行效率,节约开发者的时间和精力,然后增加机器提升产品性能。
相信大家都会选择

Which brings us to the second realization surrounding the GIL and my conclusion that it really isn't that bad.

我们再回头看看 GIL 是否真的这么不堪。

By not allowing multi-threading (or simultaneous execution within the same process) Python drastically simplifies programming complexity faced by developers. Commonissues and optimizations for multi-threaded processes can just be ignored en-masse by the developer because the Python interpreter will only ever execute a single piece of logic at a time.

这个设计是为了简化 Python 多线程的复杂度,多线程并发时常见的那些问题都避开了。

当然有一定代价。

This also doesn't usually matter much for the same horizontal reasoning as point 1. Instead of tackling issues with multi-threading, you can choose to tackle issues with multi-processing. Instead of managing multiple threads in a single process you can spin up multiple processes and communicate between them. The difference is subtle, but again, for 9/10 of those cases the performance overhead of multi-processing vs. threading doesn't matter.

这点其实没有部份人说的那么严重,影响不大。就像官方推荐的一样,如果希望利用多核处理器,你可以使用多进程来实现,不同进程之间也可以很方便地通信。还是那句话,多线程和多进程之间的那点性能开销差异在绝大多数情况下可以忽略。

In the cases where the performance over head does matter, you can always "glue" a different language into your Python logic. The classic example is how Numpy brings high performance array data structures to Python by dropping into C.

而且,Python 是著名的胶水语言,在真的非常在乎性能的场景下,可以通过其他高效的语言实现逻辑,然后将其黏合到我们的程序中。Numpy 对 C 的使用,就是一个很好的例子。

So what's the take away from all of this?

回头总结一下:

With computational power (# of processor cores, speed of individual cores, $ cost of server hardware, etc.) getting cheaper all the time, most performance issues can usually be solved somewhat by horizontal scaling.

横向扩展成本很低,大多数问题都可以通过增加计算资源得到解决。

For the things you can't really solve horizontally, you can write something in a different language and "glue it" into your Python logic (play on the saying that Python is a "glue" language).

对于 Python 不擅长的,执行效率敏感的,我们可以通过其他语言来实现,最后很方便的嵌入 Python 逻辑。

So if ultimately performance can be augmented/handled, you are left with the main reasons Python is desirable as a language:

  • Simple
  • Productive
  • Readable (and thereby more maintainable)

剩下就是 Python 流行的原因了:语法简单,编码效率高,代码可读性强。

There is countless more we could talk about with regards to the strengths and weaknesses of Python and we could go into ever more detail, but I think this is a good start at tackling the question of why Python interpreter performance hasn't affected its popularity.

讨论

  1. C 生态更好
    这个讨论没有意义。我的观点,Python 在某些地方流行而已,有些外国杠精...不提了。
  2. Python 不算解释型。
    似乎也没有意义,不能用几十年前的观念来对现在的语言进行简单归类。
  3. 关于 GIL 的讨论:
    GIL 阻止多线程同时执行,导致 Python 多线程对外呈现出单核多线程的特性。
    我的这个理解有没有问题?为什么有人提出相同的看法时,作者说这是灰色地带。

    Tony Flury:
    I would argue that the GIL doesn’t prevent multi-threading - what the GIL does is prevent multicore multi-threading. If you have an application where threads are a mix of CPU and I/O then single core multi-threading will be sufficient for most applications.
    Grant Steffen Hulegaard:
    You start getting into grey areas since GIL locked multi-threading has a sort of amplifying effect on context switches.

    我想了下,可能作者的意思是 GIL 不止是限制了并行,而且由于需要频繁加锁解锁会给程序带来额外的性能损失。这个应该很好就能验证出来。
    Python:关于 GIL

  4. 关于 Python 效率低下的问题:在大型任务面前,积累的损失让人无法忽略其存在,可能会导致程序完全不可用。
    还是那个观点,不同应用场景应该采用不同的方案,不要局限于一种语言。

  5. 有人提出其他缺点:不同版本的 Python 运行时之间有差异(这总是难以避免的)
  6. 作者提到两种提升效率的方式:
    1. Sprinkling C routines throughout my Python logic with CFFI.
      通过 CFFI 对接 C 语言程序。
    2. Dropping in another highly parallel language (Scala would probably be my choice).
      选择其他高并发语言实现部分逻辑,比如 Scala。