Golang (9) - 码厩

最简模式

package main

import (
    "fmt"
    "net/http"
)

func hello(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello world\n")
}

func main() {
    http.HandleFunc("/", hello)

    http.ListenAndServe(":8080", nil)
}

重点是记住这两点：

func hello(w http.ResponseWriter, req *http.Request)
http.HandleFunc 注册一个函数到指定路径上

`ListenAndServe` 流程分析

https://github.com/golang/go/blob/master/src/net/http/server.go#L3240
https://github.com/golang/go/blob/master/src/net/http/server.go#L2976

func ListenAndServe(addr string, handler Handler) error {
    server := &Server{Addr: addr, Handler: handler}
    return server.ListenAndServe()
}

func (srv *Server) ListenAndServe() error {
    if srv.shuttingDown() {
        return ErrServerClosed
    }
    addr := srv.Addr
    if addr == "" {
        addr = ":http"
    }
    ln, err := net.Listen("tcp", addr)
    if err != nil {
        return err
    }
    return srv.Serve(ln)
}

主要的逻辑：

net.Listen("tcp", addr) -> net.Listener
for 循环：
net.Listener.Accept() -> net.Conn
http.Server.newConn(conn) -> http.Conn
go http.Conn.serve(ctx)
goroutine 中又是一个 for 循环：
http.Conn.readRequest(ctx) -> http.response
serverHandler{c.server}.ServeHTTP(w, w.req)
最后这个 serverHandler.ServeHTTP 就比较关键了：
优先采用 http.Server 上的 Handler；
如果没有，则用默认的 http.DefaultServeMux 实现简单的 URL 路由。
这也是为什么 ListenAndServe 传了个 nil 进来，srv.Handler 为 nil 就用 DefaultServeMux。

type ServeMux struct {
    mu    sync.RWMutex
    m     map[string]muxEntry
    es    []muxEntry // slice of entries sorted from longest to shortest.
    hosts bool       // whether any patterns contain hostnames
}

func (mux *ServeMux) match(path string) (h Handler, pattern string)
func (mux *ServeMux) redirectToPathSlash(host, path string, u *url.URL) (*url.URL, bool)
func (mux *ServeMux) shouldRedirectRLocked(host, path string) bool
func (mux *ServeMux) Handler(r *Request) (h Handler, pattern string)
func (mux *ServeMux) handler(host, path string) (h Handler, pattern string)
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request)
func (mux *ServeMux) Handle(pattern string, handler Handler)
func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request))

DefaultServeMux

// ==================================================================
// 封装 handler，添加到路由表 =========================================
// ==================================================================
func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
    DefaultServeMux.HandleFunc(pattern, handler)
}
func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {
    if handler == nil {
        panic("http: nil handler")
    }
    // https://github.com/golang/go/blob/master/src/net/http/server.go#L2505
    // func (mux *ServeMux) Handle(pattern string, handler Handler)
    // 把 handler 注册到 mux.m (map[string]muxEntry) 上
    // muxEntry{h: handler, pattern: pattern}
    mux.Handle(pattern, HandlerFunc(handler))
}

// 封装普通函数为 Handler
type HandlerFunc func(ResponseWriter, *Request)
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
    f(w, r)
}

type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

// ==================================================================
// 路由匹配 =========================================================
// ==================================================================
ServeHTTP -> Handler -> handler -> match
          -> redirectToPathSlash -> shouldRedirectRLocked -> 跳转(http.RedirectHandler)

默认路由规则：

优先完整匹配
URL 前缀匹配，最长的路由规则优先

另一种方式 (`http.Handle`)

不用 http.HandleFunc，而是传入一个 Handler（实现 ServeHTTP 方法的结构体）。

package main

import (
    "fmt"
    "net/http"
)

type HelloHandler struct {
    content string
}

func (h *HelloHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, h.content)
}

func main() {
    http.Handle("/", &HelloHandler{"hello world\n"})

    http.ListenAndServe(":8080", nil)
}

func Handle(pattern string, handler Handler) { DefaultServeMux.Handle(pattern, handler) }

http.Handle 和 http.HandleFunc 都只是给默认路由（DefaultServeMux）上注册一个规则，知道这一点就可以了。

重点是 mux.Handle 和 Handler 接口。

还可以这样

package main

import (
    "fmt"
    "net/http"
)

type HelloHandler struct {
    content string
}

func (h *HelloHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, h.content)
}

func main() {
    http.ListenAndServe(":8080", &HelloHandler{"hello world\n"})
}

知道背后的流程就豁然开朗了。

当然，这样也是可以的：

func hello(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello world\n")
}
func main() {
    http.ListenAndServe(":8080", http.HandlerFunc(hello))
}

整理总结

http.Conn
http.Server
到处都是 Hander（接口），路由（mux）也是 Handler
func(w http.ResponseWriter, req *http.Request)

实现正则路由

GitHub 上一搜索 ~~golang mux~~ golang router 就有，这是几个可以参考的方案：

我自己实现一个简单的路由机制，参考 Django：

<name> OR <converter:name>

其中：converter 内置支持 str, int, slug, uuid, path, 可以自定义，默认是 str; name 可以是合法的 Go 变量名。

中间件

通过 Handler 套娃实现中间件：

type LogMiddleware struct {
    handler Handler
}
func (this LogMiddleware) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    start := time.Now()
    this.handler.ServeHTTP(w, r)
    fmt.Printf("%s %s %v\n", r.Method, r.URL.RequestURI(), time.Since(start))
}
mux := ...
http.ListenAndServe(":8000", LogMiddleware{mux})

系统内置 Handler：

grep -E "func.+ServeHTTP" src/net/http -R | grep -Fv "_test.go"
src/net/http/cgi/host.go:func (h *Handler) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
src/net/http/fs.go:func (f *fileHandler) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/httputil/reverseproxy.go:func (p *ReverseProxy) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
src/net/http/pprof/pprof.go:func (name handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
src/net/http/server.go:func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/server.go:func (rh *redirectHandler) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/server.go:func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/server.go:func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
src/net/http/server.go:func (h *timeoutHandler) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/server.go:func (globalOptionsHandler) ServeHTTP(w ResponseWriter, r *Request) {
src/net/http/server.go:func (h initALPNRequest) ServeHTTP(rw ResponseWriter, req *Request) {
src/net/http/triv.go:func (ctr *Counter) ServeHTTP(w http.ResponseWriter, req *http.Request) {
src/net/http/triv.go:func (ch Chan) ServeHTTP(w http.ResponseWriter, req *http.Request) {

有待分析。

优雅重启

参考资料与拓展阅读

Eli Bendersky's website, Life of an HTTP request in a Go server
通过分析gin、beego源码，读懂web框架对http请求处理流程的本质

#24 转载：Golang 自举

Golang 2021-01-30

Google最近公布了实现Go 1.5自举（Bootstrap）的计划。相关文档的作者是Go核心开发者Russ Cox，他在Go语言上已经耕耘了接近6年。据Russ介绍，Google就“如何从Go源码树中去除所有的C程序”已经酝酿了一年。

#23 Golang SMTP Login 认证

Golang 2021-01-16

官方库里面只看到 PlainAuth 和 CramMd5Auth 两种认证方法。

type loginAuth struct {
    username, password string
}

func LoginAuth(username, password string) smtp.Auth {
    return &loginAuth{username, password}
}

func (a *loginAuth) Start(server *smtp.ServerInfo) (proto string, toServer []byte, err error) {
    return "LOGIN", []byte{}, nil
}

func (a *loginAuth) Next(fromServer []byte, more bool) (toServer []byte, err error) {
    if more {
        switch strings.ToUpper(string(fromServer)) {
        case "USERNAME:":
            return []byte(a.username), nil
        case "PASSWORD:":
            return []byte(a.password), nil
        default:
            return nil, fmt.Errorf("unknown command %v", fromServer)
        }
    }
    return nil, nil
}

#22 Go SMTP 发送邮件

Golang SMTP 2021-01-15

package main

import (
    "crypto/tls"
    "errors"
    "fmt"
    "net"
    "net/smtp"
)

// ============================================================================

type Transaction struct {
    Host      string
    Port      uint16
    LocalName string
    TlsEnable bool
    Username  string
    Password  string
    MailFrom  string
    RcptTo    []string
    Data      []byte
}

func NewTransaction() Transaction {
    trans := Transaction{}
    return trans
}

func (trans Transaction) Send() error {
    addr := fmt.Sprintf("%s:%d", trans.Host, trans.Port)

    // SendMail(addr string, a Auth, from string, to []string, msg []byte) error
    c, err := smtp.Dial(addr)
    if err != nil {
        return err
    }
    defer c.Close()

    c.Hello(trans.LocalName)

    if ok, _ := c.Extension("STARTTLS"); ok {
        serverName, _, _ := net.SplitHostPort(addr)
        config := &tls.Config{
            InsecureSkipVerify: true,
            ServerName:         serverName,
        }
        if err = c.StartTLS(config); err != nil {
            return err
        }
    } else {
        fmt.Printf("smtp: server doesn't support STARTTLS\n")
    }

    if trans.Username != "" {
        if ok, _ := c.Extension("AUTH"); !ok {
            return errors.New("smtp: server doesn't support AUTH")

        }
        auth := smtp.PlainAuth("", trans.Username, trans.Password, trans.Host)
        if err = c.Auth(auth); err != nil {
            fmt.Println("smtp: authentication failed")
            return err
        }
    }

    if err = c.Mail(trans.MailFrom); err != nil {
        return err
    }

    for _, addr := range trans.RcptTo {
        if err = c.Rcpt(addr); err != nil {
            return err
        }
    }

    w, err := c.Data()
    if err != nil {
        return err
    }

    _, err = w.Write(trans.Data)
    if err != nil {
        return err
    }

    err = w.Close()
    if err != nil {
        return err
    }

    return c.Quit()
}

// ============================================================================

func main() {
    from := "ninedoors@126.com"
    to := "ninedoors@qq.com"
    msg := []byte{}
    fmt.Println("============================================================")
    trans := Transaction{
        "smtp.126.com", 25, "peach", false,
        from, "password",
        from, []string{to},
        msg,
    }
    trans.Send()
}

net/smtp 没有发现有好的日志实现，我只能定制了一个版本实现了日志
smtp.Client -> textproto.Conn -> textproto.Writer

#21 Golang `make`

Golang 2020-12-28

m := make(map[string]int, 5)

发现 len(m) = 0 之后我觉得有必要重新复习一下 make 函数了。

make 的作用：分配内存，初始化，返回值（不是指针）。
PS：返回值的说明：返回的是这三种引用类型本身（指针），而不是指向这三个引用的指针。
PS：new 就会返回指针。还有，new 不限类型，不初始化。

slice -> len, cap
map -> 预留大约 n 个元素的空间（具体分配空间的规则不清楚）
chan -> buffer size

The built-in function make takes a type T, which must be a slice, map or channel type, optionally followed by a type-specific list of expressions. It returns a value of type T (not *T). The memory is initialized as described in the section on initial values.

Call             Type T     Result

make(T, n)       slice      slice of type T with length n and capacity n
make(T, n, m)    slice      slice of type T with length n and capacity m

make(T)          map        map of type T
make(T, n)       map        map of type T with initial space for approximately n elements

make(T)          channel    unbuffered channel of type T
make(T, n)       channel    buffered channel of type T, buffer size n

Each of the size arguments n and m must be of integer type or an untyped constant. A constant size argument must be non-negative and representable by a value of type int; if it is an untyped constant it is given type int. If both n and m are provided and are constant, then n must be no larger than m. If n is negative or larger than m at run time, a run-time panic occurs.

s := make([]int, 10, 100)       // slice with len(s) == 10, cap(s) == 100
s := make([]int, 1e3)           // slice with len(s) == cap(s) == 1000
s := make([]int, 1<<63)         // illegal: len(s) is not representable by a value of type int
s := make([]int, 10, 0)         // illegal: len(s) > cap(s)
c := make(chan int, 10)         // channel with a buffer size of 10
m := make(map[string]int, 100)  // map with initial space for approximately 100 elements

Calling make with a map type and size hint n will create a map with initial space to hold n map elements. The precise behavior is implementation-dependent.

#20 Golang: 命令行参数解析（`flag`）

Golang 2020-12-25

Go 自带 flag 包，可以用于解析命令行参数。但用起来非常繁琐，更谈不上优雅，远不如 Python 的 argparse 包简洁明了，易于使用。

语法

类型

flag.Flag
flag.FlagSet
flag.Getter
flag.Value

方法

func Bool(name string, value bool, usage string) *bool
func BoolVar(p *bool, name string, value bool, usage string)

func Duration(name string, value time.Duration, usage string) *time.Duration
func DurationVar(p *time.Duration, name string, value time.Duration, usage string)

func Float64(name string, value float64, usage string) *float64
func Float64Var(p *float64, name string, value float64, usage string)

func Func(name, usage string, fn func(string) error)

func Int(name string, value int, usage string) *int
func IntVar(p *int, name string, value int, usage string)

func Int64(name string, value int64, usage string) *int64
func Int64Var(p *int64, name string, value int64, usage string)

func String(name string, value string, usage string) *string
func StringVar(p *string, name string, value string, usage string)

func Uint(name string, value uint, usage string) *uint
func UintVar(p *uint, name string, value uint, usage string)

func Uint64(name string, value uint64, usage string) *uint64
func Uint64Var(p *uint64, name string, value uint64, usage string)

func Var(value Value, name string, usage string)

带 Var 结尾的方法表示将值绑定到一个变量上。

func Arg(i int) string  // i-th argument
func Args() []string    // non-flag arguments

func NArg() int         // number of arguments remaining after flags have been processed
func NFlag() int        // number of command-line flags that have been set

func Parse()
func Parsed() bool
func Set(name, value string) error

func PrintDefaults()    // 帮助信息
func UnquoteUsage(flag *Flag) (name string, usage string)  // 返回 flag 的名称和用法

func Visit(fn func(*Flag))
func VisitAll(fn func(*Flag))

type ErrorHandling

FlagSet

func NewFlagSet(name string, errorHandling ErrorHandling) *FlagSet

上面的方法 FlagSet 也都有一份。

示例：Todo

package main

import (
    "flag"
    "fmt"
)

var (
    addFlag       = flag.String("add", "", "Add a new task")
    listFlag      = flag.Bool("list", false, "List all tasks")
    removeFlag    = flag.Int("remove", -1, "Remove a task by its index")
    removeAllFlag = flag.Bool("remove-all", false, "Remove all tasks")
    doneFlag      = flag.Int("done", -1, "Mark a task as done by its index")
    undoneFlag    = flag.Int("undone", -1, "Mark a task as undone by its index")
)

func main() {
    flag.Parse()
    args := make(map[string]interface{}, 10)
    args["add"] = *addFlag
    args["list"] = *listFlag
    args["remove"] = *removeFlag
    args["remove"] = *removeAllFlag
    args["done"] = *doneFlag
    args["undone"] = *undoneFlag
    fmt.Printf("%#v\n", args)
    // map[string]interface {}{"add":"", "done":-1, "list":false, "remove":false, "undone":-1}
    fmt.Println("Other arguments:", flag.Args())
    // Other arguments: []
}

参考资料与拓展阅读

pkg.go.dev, flag package
https://zhuanlan.zhihu.com/p/619004569

#19 Go 知名项目

Golang 2020-12-21

按照 stars 排序，就取头部的一些项目分析一下：

golang/go
The Go programming language
跳过
kubernetes/kubernetes
Production-Grade Container Scheduling and Management
K8S，跳过
avelino/awesome-go
A curated list of awesome Go frameworks, libraries and software
一个重要的学习参考。
moby/moby
Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Docker，跳过
gin-gonic/gin
Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.
Gin，跳过
gohugoio/hugo
The world’s fastest framework for building websites.
Hugo，知名静态网站生成工具
fatedier/frp
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
frp, 知名网络工具
syncthing/syncthing
Open Source Continuous File Synchronization
知名文件同步工具
junegunn/fzf
:cherry_blossom: A command-line fuzzy finder
fzf，知名 find 命令替代品
prometheus/prometheus
The Prometheus monitoring system and time series database.
知名监控平台
caddyserver/caddy
Fast, multi-platform web server with automatic HTTPS
知名 HTTP 服务器
astaxie/build-web-application-with-golang
A golang ebook intro how to build a web with golang
《Go Web 编程》
gogs/gogs
Gogs is a painless self-hosted Git service
知名 Git 服务器
etcd-io/etcd
Distributed reliable key-value store for the most critical data of a distributed system
知名 KV 存储服务
v2ray/v2ray-core
A platform for building proxies to bypass network restrictions.
知名网络工具
traefik/traefik
The Cloud Native Application Proxy
知名代理服务
ethereum/go-ethereum
Official Go implementation of the Ethereum protocol
知名区块链项目
FiloSottile/mkcert
A simple zero-config tool to make locally trusted development certificates with any names you'd like.
minio/minio
Multi-Cloud Object Storage
rclone/rclone
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
hashicorp/terraform
Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
wagoodman/dive
A tool for exploring each layer in a docker image
evanw/esbuild
An extremely fast JavaScript and CSS bundler and minifier
知名前端构建工具
pingcap/tidb
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try free: https://tidbcloud.com/signup
知名国产数据库
istio/istio
Connect, secure, control, and observe services.
知名服务网格组件
go-gitea/gitea
Git with a cup of tea, painless self-hosted git service
知名 Git 服务（gogs fork）
unknwon/the-way-to-go_ZH_CN
《The Way to Go》中文译本，中文正式名《Go 入门指南》
cli/cli
GitHub’s official command line tool
GitHub 官方命令行工具
go-gorm/gorm
The fantastic ORM library for Golang, aims to be developer friendly
知名 Go ORM 库
beego/beego
beego is an open-source, high-performance web framework for the Go programming language.
知名 Go web 框架
jesseduffield/lazygit
simple terminal UI for git commands
Git 终端界面
Dreamacro/clash
A rule-based tunnel in Go.
网络工具
spf13/cobra
A Commander for modern Go CLI interactions
docker/compose
Define and run multi-container applications with Docker
halfrost/LeetCode-Go
✅ Solutions to LeetCode by Go, 100% test coverage, runtime beats 100% / LeetCode 题解
harness/drone
Drone is a Container-Native, Continuous Delivery Platform
知名 CI/CD 服务
hashicorp/consul
Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
知名服务发现和配置管理服务
cockroachdb/cockroach
CockroachDB - the open source, cloud-native distributed SQL database.
数据库
nektos/act
Run your GitHub Actions locally 🚀
开发工具：GitHub Action
hashicorp/vault
A tool for secrets management, encryption as a service, and privileged access management
安全相关服务
kubernetes/minikube
Run Kubernetes locally
知名 K8S 项目
influxdata/influxdb
Scalable datastore for metrics, events, and real-time analytics
知名时序数据库
go-kit/kit
A standard library for microservices.
微服务框架
mattermost/mattermost-server
Mattermost is an open source platform for secure collaboration across the entire software development lifecycle.
研发团队协作工具
jesseduffield/lazydocker
The lazier way to manage everything docker
Docker 管理工具
labstack/echo
High performance, minimalist Go web framework
知名 HTTP 框架
kataras/iris
The fastest HTTP/2 Go Web Framework. A true successor of expressjs and laravel. Supports AWS Lambda, gRPC, MVC, Unique Router, Websockets, Sessions, Test suite, Dependency Injection and more. Thank you / 谢谢 https://github.com/kataras/iris/issues/1329
知名 HTTP 框架
portainer/portainer
Making Docker and Kubernetes management easy.
知名容器管理平台
helm/helm
The Kubernetes Package Manager
知名 K8S 项目
github/hub
A command-line tool that makes git easier to use with GitHub.
inconshreveable/ngrok
Introspected tunnels to localhost
openfaas/faas
OpenFaaS - Serverless Functions Made Simple
知名 FaaS 项目
ehang-io/nps
一款轻量级、高性能、功能强大的内网穿透代理服务器。支持 tcp、udp、socks5、http 等几乎所有流量转发，可用来访问内网网站、本地支付接口调试、ssh 访问、远程桌面，内网 dns 解析、内网 socks5 代理等等……，并带有功能强大的 web 管理端。a lightweight, high-performance, powerful intranet penetration proxy server, with a powerful web management terminal.
nsqio/nsq
A realtime distributed messaging platform
知名 MQ 服务
coreybutler/nvm-windows
A node.js version management utility for Windows. Ironically written in Go.
开发工具：前端相关
photoprism/photoprism
AI-Powered Photos App for the Decentralized Web 🌈💎✨
照片管理系统
sirupsen/logrus
Structured, pluggable logging for Go.
重要的日志库
yeasy/docker_practice
Learn and understand Docker&Container technologies, with real DevOps practice!
电子书：Docker 技术入门与实战
gofiber/fiber
⚡️ Express inspired web framework written in Go
知名 Web 框架
k3s-io/k3s
Lightweight Kubernetes
知名 K8S 项目
tsenart/vegeta
HTTP load testing tool and library. It's over 9000!
HTTP 压测工具
schollz/croc
Easily and securely send things from one computer to another :crocodile: :package:
工具：文件传输
zyedidia/micro
A modern and intuitive terminal-based text editor
工具：编辑器
spf13/viper
Go configuration with fangs
配置管理库
rancher/rancher
Complete container management platform
知名容器管理平台
tmrts/go-patterns
Curated list of Go design patterns, recipes and idioms
设计模式
go-delve/delve
Delve is a debugger for the Go programming language.
调试工具
urfave/cli
A simple, fast, and fun package for building command line apps in Go
命令行库
asim/go-micro
A Go microservices framework
知名 Go 微服务框架
dgraph-io/dgraph
Native GraphQL Database with graph backend
GraphQL 数据库（？）
dapr/dapr
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.
知名微服务平台
iawia002/lux
👾 Fast and simple video download library and CLI tool written in Go
工具：视频下载
go-kratos/kratos
Your ultimate Go microservices framework for the cloud-native era.
微服务框架
zeromicro/go-zero
A cloud-native Go microservices framework with cli tool for productivity.
微服务框架
valyala/fasthttp
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
HTTP 库
quii/learn-go-with-tests
Learn Go with test-driven development
资料：TDD
goharbor/harbor
An open source trusted cloud native registry project that stores, signs, and scans content.
gorilla/websocket
A fast, well-tested and widely used WebSocket implementation for Go.
chai2010/advanced-go-programming-book
:books: 《Go 语言高级编程》开源图书，涵盖 CGO、Go 汇编语言、RPC 实现、Protobuf 插件实现、Web 框架实现、分布式系统等高阶主题(完稿)
restic/restic
Fast, secure, efficient backup program
fyne-io/fyne
Cross platform GUI in Go inspired by Material Design
gorilla/mux
A powerful HTTP router and URL matcher for building Go web servers with 🦍
知名 HTTP 路由库
gocolly/colly
Elegant Scraper and Crawler Framework for Golang
爬虫框架
yudai/gotty
Share your terminal as a web application
终端分享
grafana/k6
A modern load testing tool, using Go and JavaScript - https://k6.io
测试工具
stretchr/testify
A toolkit with common assertions and mocks that plays nicely with the standard library
derailed/k9s
🐶 Kubernetes CLI To Manage Your Clusters In Style!
K8S 相关
joewalnes/websocketd
Turn any program that uses STDIN/STDOUT into a WebSocket server. Like inetd, but for WebSockets.
matryer/xbar
Put the output from any script or program into your macOS Menu Bar (the BitBar reboot)
grpc/grpc-go
The Go language implementation of gRPC. HTTP/2 based RPC
v2fly/v2ray-core
A platform for building proxies to bypass network restrictions.
网络工具
grafana/loki
Like Prometheus, but for logs.
日志服务
filebrowser/filebrowser
📂 Web File Browser
Web 项目
uber-go/zap
Blazing fast, structured, leveled logging in Go.
知名日志库
jaegertracing/jaeger
CNCF Jaeger, a Distributed Tracing Platform
buger/goreplay
GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
helm/charts
⚠️(OBSOLETE) Curated applications for Kubernetes
hoanhan101/ultimate-go
The Ultimate Go Study Guide
cloudreve/Cloudreve
🌩 支持多家云存储的云盘系统 (Self-hosted file management and sharing system, supports multiple storage providers)
chrislusf/seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
go-redis/redis
Type-safe Redis client for Golang
antonmedv/fx
Terminal JSON viewer
cayleygraph/cayley
An open-source graph database
vitessio/vitess
Vitess is a database clustering system for horizontal scaling of MySQL.
julienschmidt/httprouter
A high performance HTTP request router that scales well
kubernetes/kops
Kubernetes Operations (kOps) - Production Grade k8s Installation, Upgrades and Management
containers/podman
Podman: A tool for managing OCI containers and pods.
hashicorp/packer
Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
docker-slim/docker-slim
DockerSlim (docker-slim): Don't change anything in your Docker container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)
hyperledger/fabric
Hyperledger Fabric is an enterprise-grade permissioned distributed ledger framework for developing solutions and applications. Its modular and versatile design satisfies a broad range of industry use cases. It offers a unique approach to consensus that enables performance at scale while preserving privacy.
ipfs/go-ipfs
IPFS implementation in Go
rakyll/hey
HTTP load generator, ApacheBench (ab) replacement
wtfutil/wtf
The personal information dashboard for your terminal
greyireland/algorithm-pattern
算法模板，最科学的刷题方式，最快速的刷题路径，你值得拥有~
google/cadvisor
Analyzes resource usage and performance characteristics of running containers.
grpc-ecosystem/grpc-gateway
gRPC to JSON proxy generator following the gRPC HTTP spec
inancgumus/learngo
1000+ Hand-Crafted Go Examples, Exercises, and Quizzes
authelia/authelia
The Single Sign-On Multi-Factor portal for web apps
flipped-aurora/gin-vue-admin
基于 vite+vue3+gin 搭建的开发基础平台（已完成 setup 语法糖版本），集成 jwt 鉴权，权限管理，动态路由，显隐可控组件，分页封装，多点登录拦截，资源权限，上传下载，代码生成器，表单生成器等开发必备功能。
golang/dep
Go dependency management tool experiment (deprecated)
ahmetb/kubectx
Faster way to switch between clusters and namespaces in kubectl
txthinking/brook
A cross-platform network tool designed for developers. 一个为开发者设计的跨平台网络工具.
xtaci/kcptun
A Stable & Secure Tunnel based on KCP with N:M multiplexing and FEC. Available for ARM, MIPS, 386 and AMD64。KCP プロトコルに基づく安全なトンネル。KCP 프로토콜을 기반으로 하는 보안 터널입니다。
GoogleContainerTools/skaffold
Easy and Repeatable Kubernetes Development
bcicen/ctop
Top-like interface for container metrics
boltdb/bolt
An embedded key/value database for Go.
kubernetes/ingress-nginx
NGINX Ingress Controller for Kubernetes
google/gvisor
Application Kernel for Containers
dutchcoders/transfer.sh
Easy and fast file sharing from the command-line.
revel/revel
A high productivity, full-stack web framework for the Go language.
ory/hydra
OpenID Certified™ OpenID Connect and OAuth Provider written in Go - cloud native, security-first, open source API security for your infrastructure. SDKs for any language. Works with Hardware Security Modules. Compatible with MITREid.
pulumi/pulumi
Pulumi - Universal Infrastructure as Code. Your Cloud, Your Language, Your Way 🚀
charmbracelet/bubbletea
A powerful little TUI framework 🏗
CodisLabs/codis
Proxy based Redis cluster solution supporting pipeline and scaling dynamically
go-sql-driver/mysql
Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package
snail007/goproxy
🔥 Proxy is a high performance HTTP(S) proxies, SOCKS5 proxies,WEBSOCKET, TCP, UDP proxy server implemented by golang. Now, it supports chain-style proxies,nat forwarding in different lan,TCP/UDP port forwarding, SSH forwarding.Proxy 是 golang 实现的高性能 http,https,websocket,tcp,socks5 代理服务器,支持内网穿透,链式代理,通讯加密,智能 HTTP,SOCKS5 代理,黑白名单,限速,限流量,限连接数,跨平台,KCP 支持,认证 API。
aquasecurity/trivy
Scanner for vulnerabilities in container images, file systems, and Git repositories, as well as for configuration issues and hard-coded secrets
Netflix/chaosmonkey
Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
cilium/cilium
eBPF-based Networking, Security, and Observability
casbin/casbin
An authorization library that supports access control models like ACL, RBAC, ABAC in Golang
jmoiron/sqlx
general purpose extensions to golang's database/sql
gravitational/teleport
Certificate authority and access plane for SSH, Kubernetes, web apps, databases and desktops
qax-os/excelize
Go language library for reading and writing Microsoft Excel™ (XLAM / XLSM / XLSX / XLTM / XLTX) spreadsheets
fogleman/primitive
Reproducing images with geometric primitives.
gizak/termui
Golang terminal dashboard
peterq/pan-light
百度网盘不限速客户端, golang + qt5, 跨平台图形界面
AdguardTeam/AdGuardHome
Network-wide ads & trackers blocking DNS server
influxdata/telegraf
The plugin-driven server agent for collecting & reporting metrics.
emirpasic/gods
GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more
PuerkitoBio/goquery
A little like that j-thing, only in Go.
go-chi/chi
lightweight, idiomatic and composable router for building Go HTTP services
go-martini/martini
Classy web framework for Go
知名 Go Web 框架
golang/groupcache
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
知名 Go 缓存库
bettercap/bettercap
The Swiss Army knife for 802.11, BLE, IPv4 and IPv6 networks reconnaissance and MITM attacks.
getlantern/lantern
Lantern 官方版本下载蓝灯翻墙代理科学上网外网加速器梯子路由 lantern proxy vpn censorship-circumvention censorship gfw accelerator
kubernetes/dashboard
General-purpose web UI for Kubernetes clusters
micro/micro
API first development platform
gopherjs/gopherjs
A compiler from Go to JavaScript for running Go code in a browser
halfrost/Halfrost-Field
✍🏻 这里是写博客的地方 —— Halfrost-Field 冰霜之地
containerd/containerd
An open and reliable container runtime
argoproj/argo-workflows
Workflow engine for Kubernetes
hashicorp/nomad
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
elastic/beats
:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
nats-io/nats-server
High-Performance server for NATS.io, the cloud and edge native messaging system.
json-iterator/go
A high-performance 100% compatible drop-in replacement of "encoding/json"
tomnomnom/gron
Make JSON greppable!
dgraph-io/badger
Fast key-value DB in Go.
ent/ent
An entity framework for Go
containrrr/watchtower
A process for automating Docker container base image updates.
dolthub/dolt
Dolt – It's Git for Data
talkgo/night
Weekly Go Online Meetup via Bilibili ｜ Go 夜读｜通过 bilibili 在线直播的方式分享 Go 相关的技术话题，每天大家在微信/telegram/Slack 上及时沟通交流编程技术话题。
geektutu/7days-golang
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7 天用 Go 动手写/从零实现系列
FiloSottile/age
A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability.
ardanlabs/gotraining
Go Training Class Material :
google/grumpy
Grumpy is a Python to Go source code transcompiler and runtime.
go-playground/validator
:100:Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving
rqlite/rqlite
The lightweight, distributed relational database built on SQLite
milvus-io/milvus
An open-source vector database for scalable similarity search and AI applications.
GoogleContainerTools/kaniko
Build Container Images In Kubernetes
thanos-io/thanos
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
dgrijalva/jwt-go
ARCHIVE - Golang implementation of JSON Web Tokens (JWT). This project is now maintained at:
golangci/golangci-lint
Fast linters Runner for Go
tidwall/gjson
Get JSON values quickly - JSON parser for Go
git-lfs/git-lfs
Git extension for versioning large files
sqshq/sampler
Tool for shell commands execution, visualization and alerting. Configured with a simple YAML file.
OpenDiablo2/OpenDiablo2
An open source re-implementation of Diablo 2
goreleaser/goreleaser
Deliver Go binaries as fast and easily as possible
mailhog/MailHog
Web and API based SMTP testing
SMTP 工具
github/gh-ost
GitHub's Online Schema Migrations for MySQL

#18 Go 练手小项目

Golang 2020-12-21

开始在学习 Go 语言了，先在 GitHub 上找一批小项目用来学习。

julienschmidt/httprouter
A high performance HTTP request router that scales well
golang/groupcache
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
robfig/cron
a cron library for go
jroimartin/gocui
Minimalist Go package aimed at creating Console User Interfaces.
tylertreat/comcast
Simulating shitty network connections so you can build better systems.
pkg/errors
Simple error handling primitives
schachmat/wego
weather app for the terminal
teh-cmc/go-internals
A book about the internals of the Go programming language.
simeji/jid
json incremental digger
patrickmn/go-cache
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.
wercker/stern
⎈ Multi pod and container log tailing for Kubernetes
allegro/bigcache
Efficient cache for gigabytes of data written in Go.
microsoft/ethr
Ethr is a Comprehensive Network Measurement Tool for TCP, UDP & ICMP.
samber/lo
💥 A Lodash-style Go library based on Go 1.18+ Generics (map, filter, contains, find...)
goproxyio/goproxy
A global proxy for Go modules.
eranyanay/1m-go-websockets
handling 1M websockets connections in Go
fogleman/nes
NES emulator written in Go.

inconshreveable/ngrok
Introspected tunnels to localhost
sirupsen/logrus
Structured, pluggable logging for Go.
tmrts/go-patterns
Curated list of Go design patterns, recipes and idioms
gorilla/websocket
A fast, well-tested and widely used WebSocket implementation for Go.
gorilla/mux
A powerful HTTP router and URL matcher for building Go web servers with 🦍
julienschmidt/httprouter
A high performance HTTP request router that scales well
rakyll/hey
HTTP load generator, ApacheBench (ab) replacement
ahmetb/kubectx
Faster way to switch between clusters and namespaces in kubectl
charmbracelet/bubbletea
A powerful little TUI framework 🏗
jmoiron/sqlx
general purpose extensions to golang's database/sql
emirpasic/gods
GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more
PuerkitoBio/goquery
A little like that j-thing, only in Go.
go-martini/martini
Classy web framework for Go
golang/groupcache
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
FiloSottile/age
A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability.
dgrijalva/jwt-go
ARCHIVE - Golang implementation of JSON Web Tokens (JWT). This project is now maintained at:
tidwall/gjson
Get JSON values quickly - JSON parser for Go

#17 gvm: Golang 版本管理

Golang 2020-12-19

https://github.com/moovweb/gvm stars license language

安装

bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)

使用

source ~/.gvm/scripts/gvm

gvm listall

# gvm 默认从 github 克隆代码到本地，然后进行编译
# 可以采用 gitee 仓库加速
git clone git@gitee.com:mirrors/go ~/.gvm/archive/go

gvm install go1.16

gvm list
gvm use go1.16

go version
# go version go1.16 linux/amd64

#16 Golang 结构体

Golang 2020-12-18

package main

import (
    "encoding/json"
    "fmt"
    "reflect"
)

type Address struct {
    City    string `json:"city"`
    Country string `json:"country"`
}

type User struct {
    Username string `json:"username"`
    Password string `json:"password"`
}

type Person struct {
    Name    string  `json:"name"`
    Age     int     `json:"age"`
    Address Address `json:"address"`
    // 这么写，JSON 会多一层
    // User    `json:"user"`
    User
}

func (p Person) SayHello() {
    fmt.Println("Hello, my name is", p.Name)
}

func main() {
    person := Person{
        Name: "Bob",
        Age:  30,
        Address: Address{
            City:    "London",
            Country: "UK",
        },
        User: User{
            Username: "bob1999",
            Password: "pa55w0rd",
        },
    }

    // 属性操作
    fmt.Println("Name:", person.Name)
    fmt.Println("Age:", person.Age)
    fmt.Println("City:", person.Address.City)
    fmt.Println("Country:", person.Address.Country)
    fmt.Println("Username:", person.Username)
    fmt.Println("Password:", person.Password)
    fmt.Printf("%+v\n", person)
    fmt.Printf("%#v\n", person)

    // 方法调用
    person.SayHello()

    // 通过反射获取标签内容
    t := reflect.TypeOf(person)
    for i := 0; i < t.NumField(); i++ {
        field := t.Field(i)
        tag := field.Tag.Get("json")
        fmt.Println(field.Name, ":", tag)
    }

    // JSON 序列化
    jsonData, err := json.Marshal(person)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("JSON:", string(jsonData))
}

// Name: Bob
// Age: 30
// City: London
// Country: UK
// Username: bob1999
// Password: pa55w0rd
// main.Person{name:"Bob", age:30, address:main.Address{city:"London", country:"UK"}, User:main.User{username:"bob1999", password:"pa55w0rd"}}

其他知识点：

字段名大小写问题：首字母小写的话，只有本包可以访问
Zero-value Struct

type Fruit struct {
    name string
}
var apple Fruit
fmt.Println(apple) // {}

new 关键字

package main

import "fmt"

type Employee struct {
    Name string
    Age  int
}

func main() {
    p := new(Employee)
    fmt.Println(p) // &{ 0}

    var p2 Employee
    fmt.Println(p2) // { 0}

    p3 := Employee{"Me", 30}
    fmt.Println(p3) // {Me 30}
}

struct 字面量

type Person struct {
    Name  string
    Age   int
    Email string
}

p1 := Person{
    Name:  "Alice",
    Age:   25,
    Email: "alice@example.com",
}
p2 := Person {"Alice", 25, "alice@example.com"}

指针
匿名结构体

package main

import (
    "fmt"
)

func main() {
    user := struct {
        username string
    }{
        username: "admin",
    }
    fmt.Printf("%+v\n", user)
    // {username:admin}
    fmt.Printf("%#v\n", user)
    // struct { username string }{username:"admin"}
    fmt.Printf("%v\n", user)
    // {admin}
    fmt.Println(user)
    // {admin}
}

场景：

定义一个临时结构体接收反序列化数据
内嵌结构体
匿名属性（或者应该叫啥，我也不知道）
内嵌类型不能重复，
前面的例子可以看出，可以直接访问匿名属性（结构体）内部属性，
但是如果同名就只能老老实实通过类型名字来访问了

package main

import (
    "fmt"
)

type Book struct {
    string // book name (anno field)

    // syntax error: unexpected [, expected field name or embedded type
    // []Author
    Authors []Author // slice of authors

    Publisher
}

type Author struct {
    string // author name (anno field)
}

type Publisher struct {
    string // publisher name (anno field)
}

func main() {
    p := Book{
        string: "Learning Python",
        Authors: []Author{
            {string: "Stanley B.Lippman"},
            {string: "Josée LaJoie"},
            {"Barbara E.Moo"},
        },
        Publisher: Publisher{"人民邮电出版社"},
    }
    p.string = "C++ Primer" // change attr

    fmt.Println(p)
    fmt.Printf("%v\n", p)
    fmt.Printf("%+v\n", p)
    fmt.Printf("%#v\n", p)
    fmt.Printf("%v\n", p.string)
    fmt.Printf("%v\n", p.Authors[0].string)
    fmt.Printf("%v\n", p.Publisher)
    fmt.Printf("%v\n", p.Publisher.string)
}

// {C++ Primer [{Stanley B.Lippman} {Josée LaJoie} {Barbara E.Moo}] {人民邮电出版社}}
// {C++ Primer [{Stanley B.Lippman} {Josée LaJoie} {Barbara E.Moo}] {人民邮电出版社}}
// {string:C++ Primer Authors:[{string:Stanley B.Lippman} {string:Josée LaJoie} {string:Barbara E.Moo}] Publisher:{string:人民邮电出版社}}
// main.Book{string:"C++ Primer", Authors:[]main.Author{main.Author{string:"Stanley B.Lippman"}, main.Author{string:"Josée LaJoie"}, main.Author{string:"Barbara E.Moo"}}, Publisher:main.Publisher{string:"人民邮电出版社"}}
// C++ Primer
// Stanley B.Lippman
// {人民邮电出版社}
// 人民邮电出版社

将函数通过属性的方式定义

type FoodNameGetter func(string) string

type Food struct {
    name   string
    getter FoodNameGetter // declare function
}

pizza := Food{
    name: "Pizza",
    getter: func(name string) string { // declare function body
        return "This is " + name + "."
    },
}

结构体比较

如果类型和值相同，就等于。

package main

import "fmt"

type Teacher struct {
    name string
}

type Student struct {
    name string
}

func main() {
    s1 := Student{"John"}
    s2 := Student{"John"}
    fmt.Println(s1 == s2) // true

    // invalid operation: s1 == t1 (mismatched types Student and Teacher)
    // t1 := Teacher{"John"}
    // fmt.Println(s1 == t1)
}

参考资料与拓展阅读

https://golangdocs.com/structs-in-golang

一	二	三	四	五	六	日

码厩技术博客

#25 Golang HTTP Server

最简模式

`ListenAndServe` 流程分析

DefaultServeMux

另一种方式 (`http.Handle`)

还可以这样

整理总结

实现正则路由

中间件

优雅重启

参考资料与拓展阅读

#24 转载：Golang 自举

#23 Golang SMTP Login 认证

#22 Go SMTP 发送邮件

#21 Golang `make`

#20 Golang: 命令行参数解析（`flag`）

语法

类型

方法

FlagSet

示例：Todo

参考资料与拓展阅读

#19 Go 知名项目

#18 Go 练手小项目

#17 gvm: Golang 版本管理

安装

使用

#16 Golang 结构体

参考资料与拓展阅读

#25 Golang HTTP Server

最简模式

ListenAndServe 流程分析

DefaultServeMux

另一种方式 (http.Handle)

还可以这样

整理总结

实现正则路由

中间件

优雅重启

参考资料与拓展阅读

#24 转载：Golang 自举

#23 Golang SMTP Login 认证

#22 Go SMTP 发送邮件

#21 Golang make

#20 Golang: 命令行参数解析（flag）

语法

类型

方法

FlagSet

示例：Todo

参考资料与拓展阅读

#19 Go 知名项目

#18 Go 练手小项目

#17 gvm: Golang 版本管理

安装

使用

#16 Golang 结构体

参考资料与拓展阅读

`ListenAndServe` 流程分析

另一种方式 (`http.Handle`)

#21 Golang `make`

#20 Golang: 命令行参数解析（`flag`）