#74 服务丧尸?为什么

2022-10-31

发现一个问题,暂时没有任何思路:

一个线上执行了 66 天的服务,突然在前天凌晨 1:30 没有日志输出了,从外网连接不上,内网可以连上。

等后续有了进展再更新。

#73 代码中的 TODO 注释

2022-08-19

Stop Using TODO for Everything》建议采用更加明确的标记来注释代码:

  • FIXME/BUG 需要修复(缺陷)
  • CHECKME/REVIEW 需要审查
  • DOCME 需要文档
  • TESTME 需要测试
  • HACK/OPTIMIZE 需要优化

我就是喜欢在代码中写 TODO,以后可以照着这个建议搞些花样。

#72 应该如何做 Code Review

2022-06-17

Review 的三点好处

  1. 帮忙发现问题
  2. 相互学习
  3. 统一编码风格

应该怎么做

  1. 找出相应的变更(提交),开发者做简单说明,约好评审时间
  2. 参与评审的人尽快找时间从头到尾检查每一处变更,提出意见发出来
  3. 不可能一点意见都没有,每个人都应该能发出几条
  4. 编码风格也行
  5. 然后约一个会,在会上讲解整体的逻辑流程,每人提出评审意见

参考资料与拓展阅读

#71 一些微不足道的开发经验

2022-06-17

设计

  1. 方案设计要留有余地,在满足需求的前提下,尽量更加通用一点。
    前提是要认真理解需求,认真理解现有的架构,多想想,再多想想。
  2. 但是也要避免过早地、过度地优化。
    我们应该了解对团队来说更有价值的事情是什么,然后把主要的精力放在更加有价值的业务上。
    基于这个认知,我们就能更清楚需求到底是什么,应该如何去设计。

编码

  1. 遵守业界通用的编码规范非常重要。
  2. 代码可读性非常重要,简洁、美观。
  3. 如果一个逻辑有好多步骤,尽量分成几个更小的单元实现。
  4. 避免写重复的代码,将重复的部分抽离出来(有时需要考验设计能力)。
  5. 关键的点打印日志很重要,尤其是功能刚上线的时候,嫌多的话可以后面调整。

协作

  1. 沟通非常重要,沟通非常重要,沟通非常重要。
  2. 要通知到相关人,不只是消息发送出去就行,要确保他们真的理解你的意思。
    有些时候,作为经手人,还需要继续跟踪,了解他们的后续操作以及整件事情的进展。
  3. 对接时须对细节再三确认,然后形成约定,比如接口提供方将小数统一整理成保留两位,接口提供方应该保证默认排序规则。
    如果不确认,形成约定,日后有变更,可能就是意想不到的坑。
    如果是一些简单的工作,两边都处理一下也未尝不可。

工程

  1. 开发的预估时间往往是比较保守,及时你认为这个时间绝对没问题,也往往免不了延期。
  2. 可能是需求会变化,可能是方案的实现中会遇到一些变数,还可能是有别的事情会占用时间。
  3. 即便是在公开会议上也应该顶住压力,留有余地。如果真的是非常紧急,至少需要确保优先级真的比较高。

#69 常见的程序设计语言

2022-06-03

根据最新的 TIOBE 数据, 分析前面 50 中语言。
具体排名的意义对我们来说到不大,只是用来做一个比较热门语言的清单。

#68 思考:代码优先还是产品优先

2022-05-22

阮一峰最新的《科技爱好者周刊》中提到一个观点:代码优先和产品优先。大概是说代码优先程序员比较注重技术和代码,产品优先程序员则是更注重解决的问题。阮一峰明显是认为应该产品优先。

我看后感觉我就是文章的批判对象,从而进行了一个反思。但是我思考之后认为阮一峰说的并不全对,我的做法没有什么问题。

公司的目标,或者说研发中心的目标,就是用技术解决问题。而研发中心的开发人员的职责就是设计可行方案,然后通过代码来实现这个方案。
研发人员并不直接面向市场,面向用户,大多数时候也不负责产品的设计。甚至有些时候,研发人员都不参与产品设计方案评审。

理论上,如果我们能够吃透产品的设计,那么对开发工作肯定是有利的。但是我们的时间精力是非常有限的,平衡工作和生活之余,还有挤出时间来自学。
而且,对产品的思考更多的是产品设计人员的工作,让开发者把精力放在对产品的思考上有点不切实际。

确实,“如果产品不好用,代码也不会好”。但是,我们所处的位置真的很难左右公司的产品策略。让产品好用这个目标,在我们这边的操作空间就是写好自己代码。
所以对于开发者来说,代码优先几乎等同于产品优先,并不冲突,更好的代码等于更好的产品。

开发者根据项目背景和工期安排设计方案就行了。需要在代码和产品上做取舍的,是技术部门负责人的工作

我知道很多人都说打工人要有更高层次的思维,思考更多,做得更多,才能有更好的发展。我自己确实也是这么想的,积极主动的承担起份外的工作,将能创造更大的影响力,然后决定你在组织中的角色。但我觉得这是工作之余的一些额外的投入,和日常开发工作无关,没有什么产品优先的问题,开发者还是应该把主要精力都放在自己的代码上。

  1. 我们应该投入一些精力去了解公司的产品设计,以及各项重点数据。
    产品不是一成不变的,所以这是一个需要持续投入精力去做的事情。
  2. 我们还应该关注其他公司同事的工作,尤其是任务相关的其他同事。

#67 GitHub RPC 相关项目

2022-04-26

C++

  1. protocolbuffers/protobuf shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    Protocol Buffers - Google's data interchange format
  2. aria2/aria2 shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    aria2 is a lightweight multi-protocol & multi-source, cross platform download utility operated in command-line. It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink.
  3. google/flatbuffers shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    FlatBuffers: Memory Efficient Serialization Library
  4. apache/incubator-brpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Industrial-grade RPC framework used throughout Baidu, with 1,000,000+ instances and thousands kinds of services. "brpc" means "better RPC".
  5. RPCS3/rpcs3 shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    PS3 emulator/debugger
  6. TarsCloud/Tars shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Tars is a high-performance RPC framework based on name service and Tars protocol, also integrated administration platform, and implemented hosting-service via flexible schedule.
  7. capnproto/capnproto shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    Cap'n Proto serialization/RPC system - core tools and C++ library
  8. idealvin/cocoyaxi shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A go-style coroutine library in C++11 and more.
  9. baidu/sofa-pbrpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A light-weight RPC implement of google protobuf RPC framework.
  10. zeroc-ice/ice shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    Comprehensive RPC framework with support for C++, C#, Java, JavaScript, Python and more.
  11. Tencent/phxrpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A simple C++ based RPC framework.
  12. FISCO-BCOS/FISCO-BCOS shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    FISCO BCOS 是由微众牵头的金链盟主导研发、对外开源、安全可控的企业级金融区块链底层技术平台。 单链配置下,性能 TPS 可达万级。提供群组架构、并行计算、分布式存储、可插拔的共识机制、隐私保护算法、支持全链路国密算法等诸多特性。 经过多个机构、多个应用,长时间在生产环境中的实践检验,具备金融级的高性能、高可用性及高安全性。FISCO BCOS is a secure and reliable financial-grade open-source blockchain platform. The platform provides rich features including group architecture, cross-chain communication protocols, pluggable consensus mechanisms, privacy protection algorithms, OSCCA-approved (Office of State Commercial Cryptography Administration) cryptography algorithms, and distributed storage. Its performance, usability, and security have been testified by many institutional users and successful business applications in a live production environment.
  13. rpclib/rpclib shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    rpclib is a modern C++ msgpack-RPC server and client library
  14. sogou/srpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    High performance, low latency, lightweight enterprise-level RPC system. Supports Baidu bRPC, Tencent tRPC, thrift protocols.

Go

  1. asim/go-micro shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A Go microservices framework
  2. chai2010/advanced-go-programming-book shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    :books: 《Go 语言高级编程》开源图书,涵盖 CGO、Go 汇编语言、RPC 实现、Protobuf 插件实现、Web 框架实现、分布式系统等高阶主题(完稿)
  3. zeromicro/go-zero shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A web and RPC framework written in Go. It's born to ensure the stability of the busy sites with resilient design. Builtin goctl greatly improves the development productivity.
  4. grpc/grpc-go shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    The Go language implementation of gRPC. HTTP/2 based RPC
  5. geektutu/7days-golang shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7 天用 Go 动手写/从零实现系列
  6. smallnest/rpcx shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚 有 𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠 有 𝐫𝐩𝐜𝐱! build for clound!
  7. roadrunner-server/roadrunner shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    🤯 High-performance PHP application server, load-balancer and process manager written in Golang
  8. twitchtv/twirp shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A simple RPC framework with protobuf service definitions
  9. cloudwego/kitex shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A high-performance and strong-extensibility Golang RPC framework that helps developers build microservices.
  10. davyxu/cellnet shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    High performance, simple, extensible golang open source network library
  11. hashicorp/go-plugin shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Golang plugin system over RPC.
  12. TarsCloud/TarsGo shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A high performance microservice framework in golang. A linux foundation project.

Java

  1. doocs/advanced-java shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
  2. apache/dubbo shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Apache Dubbo is a high-performance, java based, open source RPC framework.
  3. redisson/redisson shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Redisson - Redis Java client with features of In-Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, local cache ...
  4. dianping/cat shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    CAT 作为服务端项目基础组件,提供了 Java, C/C++, Node.js, Python, Go 等多语言客户端,已经在美团点评的基础架构中间件框架(MVC 框架,RPC 框架,数据库框架,缓存框架等,消息队列,配置系统等)深度集成,为美团点评各业务线提供系统丰富的性能指标、健康状况、实时告警等。
  5. grpc/grpc-java shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    The Java gRPC implementation. HTTP/2 based RPC
  6. nathanmarz/storm shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
  7. weibocom/motan shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A cross-language remote procedure call(RPC) framework for rapid development of high performance distributed services.
  8. line/armeria shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Your go-to microservice framework for any situation, from the creator of Netty et al. You can build any type of microservice leveraging your favorite technologies, including gRPC, Thrift, Kotlin, Retrofit, Reactive Streams, Spring Boot and Dropwizard.
  9. sofastack/sofa-rpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    SOFARPC is a high-performance, high-extensibility, production-level Java RPC framework.

Python

  1. crossbario/autobahn-python shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    WebSocket and WAMP in Python for Twisted and asyncio
  2. quantmind/pulsar shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Event driven concurrent framework for Python
  3. lgandx/PCredz shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    This tool extracts Credit card numbers, NTLM(DCE-RPC, HTTP, SQL, LDAP, etc), Kerberos (AS-REQ Pre-Auth etype 23), HTTP Basic, SNMP, POP, SMTP, FTP, IMAP, etc from a pcap file or from a live interface.
  4. Thriftpy/thriftpy shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Thriftpy has been deprecated, please migrate to https://github.com/Thriftpy/thriftpy2
  5. tomerfiliba-org/rpyc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    RPyC (Remote Python Call) - A transparent and symmetric RPC library for python
  6. samuelcolvin/arq shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Fast job queuing and RPC in python with asyncio and redis.
  7. arskom/spyne shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A transport agnostic sync/async RPC library that focuses on exposing services with a well-defined API using popular protocols.
  8. jgarzik/python-bitcoinrpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Python interface to bitcoin's JSON-RPC API
  9. Pyrlang/Pyrlang shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Erlang node implemented in Python 3.5+ (Asyncio-based)
  10. Thriftpy/thriftpy2 shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Pure python approach of Apache Thrift.
  11. kevinhwang91/rnvimr shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Make Ranger running in a floating window to communicate with Neovim via RPC
  12. hakril/PythonForWindows shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    A codebase aimed to make interaction with Windows and native execution easier
  13. joshmarshall/jsonrpclib shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A Python JSON-RPC over HTTP that mirrors xmlrpclib syntax.
  14. CiscoDevNet/yang-explorer shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    An open-source Yang Browser and RPC Builder Application
  15. phunt/avro-rpc-quickstart shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Apache Avro RPC Quick Start.
  16. pavlov99/json-rpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    🔁 JSON-RPC 1/2 transport implementation. Supports python 2/3 and pypy.
  17. progrium/duplex shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Full duplex modern RPC
  18. maxcutler/python-wordpress-xmlrpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Python library for WordPress XML-RPC integration
  19. studio-ousia/mprpc shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w
    A fast Python RPC library
  20. Ananto30/zero shields.io:github/stars shields.io:github/languages/code-size shields.io:github/commit-activity/w shields.io:github/license
    Zero: A simple, fast, high performance and low latency Python framework (RPC + PubSub) for building microservices or distributed servers

#66 墨菲安全 CLI

2022-04-25

OSCHINA 上听说墨菲安全, 是一个 Golang 写的 cli 工具, 收集项目的依赖信息提交到服务器端, 查询可能的安全风险。
根据官方文档,目前支持 Java(maven,gradle),Golang(go mod),Node(npm,yarn),Python(pip)语言。

本文档就是简单的试用一下。

总结:我感觉挺好的,一般来说对于依赖我们都不上心,其实是有很大安全风险在里面。我觉得应该对线上业务的依赖都应该有严格的审查。
如果是安全方面的需求没有那么强烈,用这个工具可以为依赖的管理提供一些帮助。
可以考虑和 CI/CD 继承,如果有安全风险就发出告警信息。

#65 给新人的目标

2022-04-19

一些阶段性目标:

  1. 参照网上查到的博客和文档,先列出大纲以及制定学习计划
    大概以下五个主题的内容:

  2. Python 标准库

  3. Tornado
  4. Redis
  5. RabbitMQ (pika)
  6. MySQL (SQLAlchemy/pymysql)

  7. 如果资源方面允许,可以实现一个内部系统开发,进行编码和设计方面的指导

  8. 阅读项目文档,理解邮件和短信的处理流程,项目结构,以及每个项目的核心逻辑
  9. 给定一个周边项目进行开发维护,小任务开发(Code Review)
  10. 从写文档和单元测试开始,逐渐参与核心项目的开发和维护

其他:Linux 基础,Git,我们的开发流程和规范

标准库

  1. 常用基础库: os, sys, shutil, re, time, datetime, ramdom, json, pickle ...
  2. 日志: logging
  3. HTTP 相关: urllib, http
  4. 邮件相关: email, smtplib, smtpd
  5. 编码和加密相关: md5, sha, base64, hmac, binascii
  6. 单元测试: unitest 非常重要

Tornado

  1. Web 框架先走起来
  2. 基本用法,比如 add_timeout, call_later, add_callback ...
  3. 了解 IOLoop/IOStream
  4. 项目开发过程中的一些实践

Redis

  1. 安装
  2. redis-cli
  3. 数据类型
  4. 命令清单
  5. redis

RabbitMQ

  1. 基本概念 (Exchange / Queue / Binding / Channel / Connection)
  2. 安装和使用
  3. pika
  4. 发布/订阅 (生产, 消费)

MySQL

  1. 数据类型
  2. 连接
  3. 索引
  4. pymysql / mysqlclient
  5. SQLAlchemy