![]() In short, you need to accept the default settings for almost everything until you have basic settings for Target Global Configuration, Portals and Initiator Groups until you have something that resembles these screenshots. Look under Sharing / Block Shares (iSCSI). The iSCSI service needs a little bit more setting up than NFS, and the iSCSI settings are in a different place, too. You must select Enable NFSv4, NFSv3 ownership model for NFSv4 and Allow non-root mount. Access the NFS settings by clicking on the pencil icon in the Services menu. The NFS service requires a little tweaking to make it work properly with Kubernetes. To set the shell for your user, go to Accounts / Users and click on the user you’ll be using. The author of democratic-csi informs me it should be set to bash or sh, and on recent deployments of TrueNAS it defaults to csh, which won’t work. ![]() You will also need to ensure that the user account used by Kubernetes to SSH to TrueNAS has a supported shell. This guide uses the root user for simplicity but in a production environment you should create a separate user with either a strong password, or a certificate. Kubernetes will need access to the TrueNAS API with a privileged user. I’m using both iSCSI and NFS, and I’ve started them running and also set them to start automatically (e.g. With your storage pools in place, it’s time to enable the services you need. ![]() Here’s what my dataset structure looks like – ignore backup and media: TrueNAS pools Brevity is important here, as we’ll see later. I wanted to provide both iSCSI and NFS, so under k8s I created more sub datasets iscsi and nfs. I’ve got various things on my TrueNAS box so under hdd I created a dataset k8s. In my case, this top-level pool is called hdd. This is configured in the Storage / Pools menu. TrueNAS PoolsĪ complete guide to TrueNAS is outside the scope of this article, but basically you’ll need a working pool. I thought it would be helpful to document and share my experience, so here’s my rough guide on how to set up storage on TrueNAS Core 12 with MicroK8s and democratic-csi. I was unfamiliar with Kubernetes storage and TrueNAS, but I found it quite easy to get started and the lead developer was super helpful while answering my questions. After some searching, I found a project called democratic-csi which describes itself asĭemocratic-csi implements the csi (container storage interface) spec providing storage for various container orchestration systems (ie: Kubernetes). It’s quite easy to mount an existing NFS or iSCSI share into a Kubernetes pod but the hard part is automating the creation of these storage resources with a provisioner. ![]() The key part is how to integrate Kubernetes with TrueNAS. In the end I settled upon TrueNAS (which used to be called FreeNAS but was recently renamed) as it is simple to set up and provides a number of storage options that Kubernetes can consume, both as block storage via iSCSI and file storage via NFS. It works, but I never really liked the solution so I decided to look at dedicated storage solutions for my home lab and a small number of production sites, which would escape the single-node limitation of the MicroK8s storage addon and allow me to scale to more than one node. A while ago I blogged about the possibilities of using Ceph to provide hyperconverged storage for Kubernetes. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |