--- title: Databend 开源周报第 82 期 description: Databend 上线表数据缓存和查询结果缓存，运行缓存友好型工作负载时，性能显著提高！另外 Databend 本周也发布官方 Deb 软件源并提供 systemd 服务管理支持。 slug: category-weekly/databend-weekly-82 date: 2023-02-27 tags: [weekly] category: Weekly image: databend-weekly.png authors: - DatabendLabs --- # Databend 开源周报 第 82 期 > 英文版移步：https://www.databend.com/blog/2023-02-22-databend-weekly > > [Databend](https://github.com/databendlabs/databend) 是一款现代云数仓。专为弹性和高效设计，为您的大规模分析需求保驾护航。自由且开源。即刻体验云服务：[https://app.databend.cn](https://app.databend.cn) 。 {/* truncate */} ## What's New 探索 Databend 本周新进展，遇到更贴近你心意的 Databend。 ### Features & Improvements :sparkles: **AST** - select from stage 支持带连接选项的 uri ([#10066](https://github.com/databendlabs/databend/pull/10066)) **Catalog** - 支持创建 Iceberg catalog ([#9017](https://github.com/databendlabs/databend/pull/9017)) **Expression** - decimal 类型支持聚合函数 min/max ([#10085](https://github.com/databendlabs/databend/pull/10085)) - decimal 类型支持聚合函数 sum/avg ([#10059](https://github.com/databendlabs/databend/pull/10059)) **Pipeline** - 增强 pipelines processors 能力 ([#10098](https://github.com/databendlabs/databend/pull/10098)) **Query** - create stage, select stage, copy, infer_schema 支持具名文件类型 ([#10084](https://github.com/databendlabs/databend/pull/10084)) - 查询结果缓存 ([#10042](https://github.com/databendlabs/databend/pull/10042)) **Storage** - 表数据缓存 ([#9772](https://github.com/databendlabs/databend/pull/9772)) - 在 `drop all` 中使用 `drop_table_by_id` API ([#10054](https://github.com/databendlabs/databend/pull/10054)) - 原生存储格式支持嵌套数据类型 ([#9798](https://github.com/databendlabs/databend/pull/9798)) ### Code Refactoring :tada: **Meta** - 为升级实现兼容层 ([#10082](https://github.com/databendlabs/databend/pull/10082)) - 更优雅的错误处理 ([#10112](https://github.com/databendlabs/databend/pull/10112), [#10114](https://github.com/databendlabs/databend/pull/10114), etc.) **Cluster** - 支持交换排序 ([#10149](https://github.com/databendlabs/databend/pull/10149)) **Executor** - 检查 processor graph 完成情况 ([#10166](https://github.com/databendlabs/databend/pull/10166)) **Planner** - 为物理计划构建器执行常量折叠 ([#9889](https://github.com/databendlabs/databend/pull/9889)) **Query** - 使用 AccumulatingTransform 实现单状态聚合器 ([#10125](https://github.com/databendlabs/databend/pull/10125)) **Storage** - 适配 OpenDAL 批删除支持 ([#10150](https://github.com/databendlabs/databend/pull/10150)) - 适配 OpenDAL 基于查询的元数据缓存 ([#10162](https://github.com/databendlabs/databend/pull/10162)) ### Build/Testing/CI Infra Changes :electric_plug: - 发布 deb 软件源 ([#10080](https://github.com/databendlabs/databend/pull/10080)) - 支持 systemd units ([#10145](https://github.com/databendlabs/databend/pull/10145)) ### Bug Fixes :wrench: **Expression** - 不再返回 Variant 作为公共超级类型 ([#9961](https://github.com/databendlabs/databend/pull/9961)) - 允许从 string 和 variant 的自动类型转换 ([#10111](https://github.com/databendlabs/databend/pull/10111)) **Cluster** - 在集群模式下修复 limit 查询挂起 ([#10006](https://github.com/databendlabs/databend/pull/10006)) **Storage** - 修复包含 tuple 类型时的列统计错误 ([#10068](https://github.com/databendlabs/databend/pull/10068)) - 增列后 compact 未按预期工作 ([#10070](https://github.com/databendlabs/databend/pull/10070)) - 修复增列的 min/max 统计故障 ([#10137](https://github.com/databendlabs/databend/pull/10137)) ## What's On In Databend 请持续关注 Databend 的最新动态。 ### Query Result Cache 仅仅过去一周，Databend 现已支持查询结果缓存！ ``` ┌─────────┐ 1 ┌─────────┐ 1 │ ├───►│ ├───►Dummy───►Downstream Upstream────►│Duplicate│ 2 │ │ 3 │ ├───►│ ├───►Dummy───►Downstream └─────────┘ │ │ │ Shuffle │ ┌─────────┐ 3 │ │ 2 ┌─────────┐ │ ├───►│ ├───►│ Write │ Upstream────►│Duplicate│ 4 │ │ 4 │ Result │ │ ├───►│ ├───►│ Cache │ └─────────┘ └─────────┘ └─────────┘ ``` **阅读下述材料以了解更多讯息** - [PR | feat(query): query result cache](https://github.com/databendlabs/databend/pull/10042) - [Docs | RFC: Query Result Cache](https://docs.databend.cn/guides/performance/query-result-cache) - [Tracking Issue | RFC: query result cache](https://github.com/databendlabs/databend/issues/10011) ### Table Data Cache Databend 现在支持表数据缓存： - 磁盘缓存：DataBlock 的原始列（压缩）数据。 - 内存缓存（实验性）：DataBlock 的反序列化列对象。 对于缓存友好的工作负载，性能提升显着。 **阅读下述材料以了解更多讯息** - [PR | feat: table data cache](https://github.com/databendlabs/databend/pull/9772) - [Docs | Query Server Configuration - Cache](https://docs.databend.cn/guides/deploy/references/node-config/query-config/#cache-%E9%83%A8%E5%88%86) ### Deb Source & Systemd Support Databend 现在提供官方的 Deb 软件源，并支持使用 `systemd` 管理服务。 对于 DEB822 源格式： ```bash sudo curl -L -o /etc/apt/sources.list.d/databendlabs.sources https://repo.databend.com/deb/databendlabs.sources sudo apt update sudo apt install databend sudo systemctl start databend-meta sudo systemctl start databend-query ``` **阅读下述材料以了解更多讯息** - [PR | chore(ci): release with systemd units](https://github.com/databendlabs/databend/pull/10145) - [Docs | Installing Databend - Package Manager](https://docs.databend.cn/guides/deploy/deploy/download) ## What's Up Next 我们始终对前沿技术和创新理念持开放态度，欢迎您加入社区，为 Databend 注入活力。 ### 服务启动进度报告 启动 Query / Meta 节点时，有必要执行检查并将结果显式输出，以帮助用户诊断故障并确认节点 / 集群状态。 **示例：** ```bash storage check succeed meta check failed: timeout, no response. endpoints: xxxxxxxx . status check failed: address already in use. ``` [Issue 10193: Feature: output the necessary progress when starting a query/meta node](https://github.com/databendlabs/databend/issues/10193) 如果你对这个主题感兴趣，可以尝试解决其中的部分问题或者参与讨论和 PR review。或者，你可以点击 https://link.databend.com/i-m-feeling-lucky 来挑选一个随机问题，祝好运！ ## Changelogs 前往查看 Databend 每日构建的变更日志，以了解开发的最新动态。 地址：https://github.com/databendlabs/databend/releases ## Contributors 非常感谢贡献者们在本周的卓越工作。 | [andylokandy](https://github.com/andylokandy) | [ariesdevil](https://github.com/ariesdevil) | [b41sh](https://github.com/b41sh) | [Big-Wuu](https://github.com/Big-Wuu) | [BohuTANG](https://github.com/BohuTANG) | [cameronbraid](https://github.com/cameronbraid) | | :-------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------: | | [andylokandy](https://github.com/andylokandy) | [ariesdevil](https://github.com/ariesdevil) | [b41sh](https://github.com/b41sh) | [Big-Wuu](https://github.com/Big-Wuu) | [BohuTANG](https://github.com/BohuTANG) | [cameronbraid](https://github.com/cameronbraid) | | [Chasen-Zhang](https://github.com/Chasen-Zhang) | [ClSlaid](https://github.com/ClSlaid) | [dantengsky](https://github.com/dantengsky) | [drmingdrmer](https://github.com/drmingdrmer) | [everpcpc](https://github.com/everpcpc) | [johnhaxx7](https://github.com/johnhaxx7) | | :----------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: | | [Chasen-Zhang](https://github.com/Chasen-Zhang) | [ClSlaid](https://github.com/ClSlaid) | [dantengsky](https://github.com/dantengsky) | [drmingdrmer](https://github.com/drmingdrmer) | [everpcpc](https://github.com/everpcpc) | [johnhaxx7](https://github.com/johnhaxx7) | | [lichuang](https://github.com/lichuang) | [mergify[bot]](https://github.com/apps/mergify) | [PsiACE](https://github.com/PsiACE) | [RinChanNOWWW](https://github.com/RinChanNOWWW) | [soyeric128](https://github.com/soyeric128) | [sundy-li](https://github.com/sundy-li) | | :-------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------: | | [lichuang](https://github.com/lichuang) | [mergify[bot]](https://github.com/apps/mergify) | [PsiACE](https://github.com/PsiACE) | [RinChanNOWWW](https://github.com/RinChanNOWWW) | [soyeric128](https://github.com/soyeric128) | [sundy-li](https://github.com/sundy-li) | | [suyanhanx](https://github.com/suyanhanx) | [TCeason](https://github.com/TCeason) | [Xuanwo](https://github.com/Xuanwo) | [xudong963](https://github.com/xudong963) | [youngsofun](https://github.com/youngsofun) | [zhang2014](https://github.com/zhang2014) | | :----------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------: | | [suyanhanx](https://github.com/suyanhanx) | [TCeason](https://github.com/TCeason) | [Xuanwo](https://github.com/Xuanwo) | [xudong963](https://github.com/xudong963) | [youngsofun](https://github.com/youngsofun) | [zhang2014](https://github.com/zhang2014) | | [zhyass](https://github.com/zhyass) | | :----------------------------------------------------------------------------------------------------------------------------: | | [zhyass](https://github.com/zhyass) | ## Connect With Us Databend 是一款开源、弹性、低成本，基于对象存储也可以做实时分析的新式数仓。期待您的关注，一起探索云原生数仓解决方案，打造新一代开源 Data Cloud。 - [Databend Website](https://docs.databend.cn) - [GitHub Discussions](https://github.com/databendlabs/databend/discussions) - [Twitter](https://twitter.com/Datafuse_Labs) - [Slack Channel](https://link.databend.com/join-slack)