码厩技术博客

#50 聊一聊众 Webdrivers

WebDriver 是浏览器自动化工具。

主要的用途：

1. 自动化测试
2. 爬虫
3. 网页截图

知名的：

• Selenium
• Puppeteer

#49 Python DKIM 签名

• pydkim
  • https://pypi.org/project/pydkim/
  • https://hewgill.com/pydkim/
  • NOTE: This page describes the last release of pydkim from 2008. The latest version is a fork found at dkimpy in Launchpad and is under active development.
  • 最新版本是 2008/06 发布的 v0.3
• dkimpy
  • https://pypi.org/project/dkimpy/
  • https://launchpad.net/dkimpy
  • 最新版本是昨天发布的 v0.9.2

说明

默认签名字段（28 个）：

cc, content-description, content-id, content-transfer-encoding, content-type,
date,
from,
in-reply-to,
list-archive, list-help, list-id, list-owner, list-post, list-subscribe, list-unsubscribe,
message-id, mime-version,
references, reply-to, resent-cc, resent-date, resent-from, resent-message-id, resent-sender, resent-to,
sender, subject,
to

d = dkim.DKIM('')
print(b', '.join(d.should_sign | d.frozen_sign))
b'list-unsubscribe, content-id, list-id, mime-version, resent-date, sender, cc, reply-to, content-type, list-owner, resent-message-id, resent-cc, resent-from, to, content-description, date, list-post, in-reply-to, content-transfer-encoding, from, references, list-help, subject, list-archive, resent-sender, list-subscribe, message-id, resent-to'
print(b', '.join(sorted(d.should_sign | d.frozen_sign)))
b'cc, content-description, content-id, content-transfer-encoding, content-type, date, from, in-reply-to, list-archive, list-help, list-id, list-owner, list-post, list-subscribe, list-unsubscribe, message-id, mime-version, references, reply-to, resent-cc, resent-date, resent-from, resent-message-id, resent-sender, resent-to, sender, subject, to'

示例

准备实验用的密钥对。

openssl genpkey -algorithm RSA -out /tmp/private_key.pem
openssl rsa -in /tmp/private_key.pem -check
openssl rsa -pubout -in /tmp/private_key.pem -out /tmp/public_key.pem

import dkim

domain = 'mail.markjour.com'
selector = 's20190416'

with open('/tmp/private_key.pem', 'rb') as f:
    privkey = f.read().strip()
# dkim.parse_pem_private_key(privkey)

message = """
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=utf-8
Date: Mon, 24 Sep 2018 12:31:21 +0000 (UTC)
From: Admin <no-reply@mail.markjour.com>
Mime-Version: 1.0
Subject: Hello World
Message-ID: <n4F5zz24LXvYqPHVrZLPJokasT7MlLxYQx6g>
Reply-To: sender@mail.markjour.com
To: kwicoo@gmail.com
List-Unsubscribe: <mailto:unsubscribe@mail.markjour.com?p=Ahi2DRmdOnTdpsDzPClCPqbpwmFyjvGJV2xfJGWqw6eFEKRwI402QeoSsFrArTw1s48A59f60pLl0x71ojsQSWERnp3aMZA6YvEw>
X-SMTP-ID: c89cf6a5-22b7-4d1a-9bce-9f91a6be1bfb

HELLO WORLD
""".strip().encode()
# dkim.rfc822_parse(message)
# print(dkim.DKIM(message).default_sign_headers())
# [b'Content-Transfer-Encoding', b'Content-Type', b'Date', b'From', b'Mime-Version', b'Subject', b'Message-ID', b'Reply-To', b'To', b'List-Unsubscribe', b'From']

signature = dkim.sign(message, selector.encode(), domain.encode(), privkey)
print(signature.decode())
# DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.markjour.com;
#  i=@mail.markjour.com; q=dns/txt; s=s20190416; t=1555392275;
#  h=content-transfer-encoding : content-type : date : from :
#  mime-version : subject : message-id : reply-to : to : list-unsubscribe
#  : from; bh=qg03cTlGc4OH4uPv7BGgoUyhgh23r+o1O6qzYOLixvA=;
#  b=sJ09G6hHPaP6AMp2mqUXjEZ+BfUFz0o6nbpXWxJ4/OG0o9ZwPSj8aJibZtJjTKP3k/TR/
#  6SD543V8iNw+JwwM+XLOUZa0iduK+QkedccqNl5Hcfc9UI/U11NoHz76B3csL9KE9tb40jF
#  mlLCuVUjci4HlOfEoKF8Ame8yWDHXVoNS/YT9/OSSc5q5q+qp6OX6PvzzxDomCHC6kbhOdv
#  Yc/KEXrMQ1JQ971pRUBNQK3eN7bV7g1BwXuMEuhdwDa4aZ4YYcakKywo4Oey7bIy1E7evZN
#  5rUitRExLH4dQNrhxoZd4c3QOjd4ROTwseAaMN10U/egzDXjcw2q0UUC1UKQ==

append_headers = [b'x-smtp-id']

d = dkim.DKIM(message)
include_headers = d.default_sign_headers()
include_headers.extend(append_headers)
signature = d.sign(selector.encode(), domain.encode(), privkey, include_headers=include_headers)
print(signature.decode())
# DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.markjour.com;
#  i=@mail.markjour.com; q=dns/txt; s=s20190416; t=1555392275;
#  h=content-transfer-encoding : content-type : date : from :
#  mime-version : subject : message-id : reply-to : to : list-unsubscribe
#  : from : x-smtp-id; bh=qg03cTlGc4OH4uPv7BGgoUyhgh23r+o1O6qzYOLixvA=;
#  b=MTSeE8X3R+8bn+kkJaX5j/OKPMe+sdombmmwK5zME3SHBqiOLbxCwOGyh3qJKXdLpJlEg
#  pBnsDmNEjgC/rtBoclvnlCsaN7OFcZIe6ehfjwGeaw41r38Y8IgUQCkuN+IiL8FN1IiMI2f
#  kSayumwcOCAwmA4yJfu8n1v4W416jXt775YKR+1bt2Df1fNA6FnfoSMTqZl7rHn9zo76Efg
#  yvm7M0uT3uz0NZbJtqOnMFzRri9TEj4jYiCgsNaBYA9prbZlA02svoJx9qIJ2mKA+EcVpxK
#  IsEAY4ZXzXfhynKLeOYGK786ghiZrtsQYGbP6c1fAzTNy+fLJzRFozsV/wEQ==

headers = set(dkim.DKIM.SHOULD) | set(dkim.DKIM.FROZEN) | append_headers - set(dkim.DKIM.SHOULD_NOT)
signature = dkim.sign(message, selector.encode(), domain.encode(), privkey, include_headers=headers)
print(signature.decode())
# DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mail.markjour.com;
#  i=@mail.markjour.com; q=dns/txt; s=s20190416; t=1555392275;
#  h=in-reply-to : x-smtp-id : content-type : cc : content-id : list-post
#  : date : resent-from : list-owner : list-id : to : content-description
#  : list-subscribe : message-id : sender : mime-version :
#  resent-message-id : list-help : content-transfer-encoding : resent-cc
#  : resent-date : list-unsubscribe : references : resent-sender : from :
#  list-archive : subject : resent-to : reply-to;
#  bh=qg03cTlGc4OH4uPv7BGgoUyhgh23r+o1O6qzYOLixvA=;
#  b=lLJafHJ8B/DoO4FncLp+BIHaPy4xsq7dRAWjzAvkRoDSwjcg3EloW0FsCXS45EkmwmBZC
#  Vks7zeOR1CS8oxcpauhxj1XnlwfcwLWtAQ3pogQTzNh4EEUFiNfgJTdXefAh7cpGHolQmy7
#  w2TBXDPx+Ikynw2tNnGOBduLWi+BH3Et8KGaskR4D9QHWSrk4pqeaNannNhDPUfE98d2fS3
#  kKBvqiEaTubQBdi8VXcl8J4R1SfdJZR2NfBJkPjJejlwTJaSytF2zyberpgflj0sEc8iHvM
#  2UQRcpxqm8GMRyzzKAXBSTzQhmaTOHntGokDTunNlUc/izFFRJm9SFiVq64g==

OpenDKIM

使用 opendkim-genkey 生成签名私钥和 DNS 配置文件：

$ opendkim-genkey --verbose --domain=mail.markjour.com --selector=s20190416 --directory=/tmp/
opendkim-genkey: generating private key
opendkim-genkey: private key written to s20190416.private
opendkim-genkey: extracting public key
opendkim-genkey: DNS TXT record written to s20190416.txt

域名配置好之后，可以使用 opendkim-testkey 检查：

$ opendkim-testkey -d test.markjour.com -k /tmp/private_key.pem -s s20190416 -v
opendkim-testkey: 's20190416._domainkey.test.markjour.com' record not found

#48 Python Supervisor 的简单配置

#47 Python 解析 UserAgent

#46 Python3 中的 Bytes 索引取值

#45 Python 定时任务的简单部署

#44 Python 与 HTTP/2

HTTP/2 都出来三年半了，还没有一个好用的 Python 客户端库，真是的。

#43 PEP

主要是了解一下 PEP（Python Enhancement Proposal）的状态和类型。

#42 Python 打包上传到 PyPI

首先你得有 PyPI 的账号，没有注册的话，不用搞了。
然后你肯定要先准备 setup.py 了，如果这个都没弄就不用搞了。

算了，先贴一个简单的样板吧。

setup.py

# -*- coding: utf-8 -*-

from setuptools import setup

setup(
    name='PageageName',
    version='0.0.1',
    author='YourName',
    author_email='YourEmail',
    url='PackageSite (e.g. GitHub)',
    description='....',
    long_description=open('README.md').read(),
    long_description_content_type='text/markdown',
    packages=['PackageDir'],
    install_requires=[
        'requests>=2.18.4',
        'balabala',
    ],
    extras_require={  # 可选
        'dev': [
            'balabala...',
        ],
        'test': [
            'balabala...',
        ],
    },
    classifiers=[
        'Development Status :: 3 - Alpha',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'License :: OSI Approved :: MIT License',
    ],
)

流程

python3 -m pip install twine

rm -rf dist

python3 setup.py sdist bdist_wheel
python2 setup.py sdist bdist_wheel

twine upload dist/*

twine 会询问账号密码，如果不想每次这么麻烦，可以创建 ~/.pypirc 文件。

.pypirc 文件

[distutils]
index-servers =
    pypi

[pypi]
repository = https://upload.pypi.org/legacy/
username = your_username
password = your_password

参考资料

• PyPI, twine
• ReadTheDocs, twine documentation
• Python, The .pypirc file

#41 Python 应用: 简易 SMTP 服务器

#40 Python 应用: 简易 FTP 服务器

FTP 需要 pyftpdlib 包的支持。

#39 Python 应用: 简易 HTTP 服务器

Python2

在当前目录起 HTTP 服务，可以用于测试和临时性的文件下载服务。

# Default bind to 0.0.0.0:8000
python -m SimpleHTTPServer

# Maybe you want to use port 8080
python -m SimpleHTTPServer 8080

Python3

除了可以指定端口，还可以指定绑定地址、工作目录。

# Also bind to 0.0.0.0:8000
python -m http.server

python -m http.server -h
# usage: server.py [-h] [--cgi] [--bind ADDRESS] [--directory DIRECTORY] [port]
#
# positional arguments:
#   port                  Specify alternate port [default: 8000]
#
# optional arguments:
#   -h, --help            show this help message and exit
#   --cgi                 Run as CGI Server
#   --bind ADDRESS, -b ADDRESS
#                         Specify alternate bind address [default: all interfaces]
#   --directory DIRECTORY, -d DIRECTORY
#                         Specify alternative directory [default:current directory]

python -m http.server 9999
python -m http.server --bind=127.0.0.1
python -m http.server --bind=127.0.0.1 9999
python -m http.server -d ~/Pictures

#38 Python Redis Debug

如果可以的话，使用 redis-cli monitor 命令来输出所有 Redis 命令也很方便。

有时，条件不允许，或者 Redis 需要处理其他的连接，我希望将自己代码调用的 Redis 命令输出到日志中，方便调试。

#37 Python SMTP

import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart

SMTP_HOST = "smtp.example.com"
SMTP_PORT = 587
SMTP_USERNAME = "your_username"
SMTP_PASSWORD = "your_password"
SMTP_STARTTLS = True

sender = "sender@example.com"
recipient = "recipient@example.com"
subject = "Test Email"
message = "This is a test email."

msg = MIMEMultipart()
msg["From"] = sender
msg["To"] = recipient
msg["Subject"] = subject
msg.attach(MIMEText(message))

smtp = smtplib.SMTP(SMTP_HOST, SMTP_PORT)
smtp.ehlo()
if SMTP_STARTTLS:
    smtp.starttls()
    smtp.ehlo()
if SMTP_USERNAME and SMTP_USERNAME:
    smtp.login(SMTP_USERNAME, SMTP_PASSWORD)

smtp.sendmail(sender, recipient, msg.as_string())
smtp.quit()

#36 Python 项目结构图

常用的代码风格检查工具 pylint 中集成了 pyreverse, 这是一个用于生成 Python 的类图的工具。