Solaris User

ZFSが使用するメモリを制限する (zfs_arc_maxによるARCの上限設定)

ZFSが使用する物理メモリを制限する方法について紹介します。

ZFSはアクティブなストレージプールのデータをキャッシュするために物理メモリを使用します。このキャッシュはARC(Adaptive Replacement Cache)と呼ばれ、システムにフリーメモリが十分ある間はどんどん増えていきます。

しかし、ARCのメモリ使用に対する優先度は高く設定されておらず、他のアプリケーションが使用するメモリが足りなくなった場合には、ARCの領域は解放されますし、ARCが他のアプリケーションからメモリ領域を奪う設計にはなっていませんので、一般にARCのサイズを制限する必要はないと言われております。

とは言っても、リソースマネージメントをしっかり行いたいというニーズはあります。その場合には以下のように/etc/systemファイルでzfs:zfs_arc_maxパラメータを設定してZFSが使用するメモリを制限することができます。以下では上限を1GB ( 1x 1024 x 1024 x 1024 = 1073741824バイト)に設定しています。

bash-3.00# vi /etc/system
...
set zfs:zfs_arc_max = 1073741824

上記のように変更したら再起動をすれば、設定が反映します。

またARC(キャッシュ)の上限の設定や利用状況を確認したい場合は以下のようにkstatコマンドで確認できます。

bash-3.00# kstat -n arcstats
module: zfs                             instance: 0
name:   arcstats                        class:    misc
        c                               1073741824
        c_max                           1073741824
        c_min                           134217728
        crtime                          52.45855435
        deleted                         0
        demand_data_hits                0
        demand_data_misses              0
        demand_metadata_hits            0
        demand_metadata_misses          0
        evict_skip                      0
        hash_chain_max                  0
        hash_chains                     0
        hash_collisions                 0
        hash_elements                   0
        hash_elements_max               0
        hdr_size                        0
        hits                            0
        l2_abort_lowmem                 0
        l2_cksum_bad                    0
        l2_evict_lock_retry             0
        l2_evict_reading                0
        l2_feeds                        0
        l2_free_on_write                0
        l2_hdr_size                     0
        l2_hits                         0
        l2_io_error                     0
        l2_misses                       0
        l2_rw_clash                     0
        l2_size                         0
        l2_writes_done                  0
        l2_writes_error                 0
        l2_writes_hdr_miss              0
        l2_writes_sent                  0
        memory_throttle_count           0
        mfu_ghost_hits                  0
        mfu_hits                        0
        misses                          0
        mru_ghost_hits                  0
        mru_hits                        0
        mutex_miss                      0
        p                               536870912
        prefetch_data_hits              0
        prefetch_data_misses            0
        prefetch_metadata_hits          0
        prefetch_metadata_misses        0
        recycle_miss                    0
        size                            0
        snaptime                        147.92725895

また、リアルタイムでZFSのキャッシュの使用状況を確認するのにarcstat.plという便利なツールが公開されています。
⇒ arcstat.plのダウンロード
上記ファイルを適当なディレクトリにダウンロードして、実行権を与えて以下のように実行します。

bash-3.00# ./arcstat.pl
    Time  read  miss  miss%  dmis  dm%  pmis  pm%  mmis  mm%  arcsz     c
11:50:25    40     2      5     2    5     0    0     2    5   338K    1G
11:50:26     0     0      0     0    0     0    0     0    0   338K    1G
11:50:27     3     0      0     0    0     0    0     0    0   488K    1G
11:50:28     9     0      0     0    0     0    0     0    0     8M    1G
11:50:29     0     0      0     0    0     0    0     0    0    14M    1G
11:50:30    10     0      0     0    0     0    0     0    0    22M    1G
11:50:31     0     0      0     0    0     0    0     0    0    30M    1G
11:50:32     0     0      0     0    0     0    0     0    0    41M    1G
11:50:33     0     0      0     0    0     0    0     0    0    47M    1G
11:50:34     1     0      0     0    0     0    0     0    0    51M    1G
11:50:35     5     0      0     0    0     0    0     0    0    57M    1G
11:50:36     3     0      0     0    0     0    0     0    0    65M    1G
11:50:37     0     0      0     0    0     0    0     0    0    77M    1G
...

上記の例では、別の端末でZFSのファイルシステムに新しいデータをコピーしています。右端のcカラム(1G)がARCの上限で、その隣のarcszカラムが現在使用中のキャッシュサイズです。どんどんキャッシュのサイズが増えているのがわかります。

ちなみにarcstat.plの出力の各フィールドの説明や使用方法は-vオプションで確認できます。

bash-3.00# ./arcstat.pl -v
Arcstat version 0.1
Usage: arcstat.pl [-hvx] [-f fields] [-o file] [interval [count]]
Field definitions are as follows
 dread : Demand data accesses per second
  Time : Time
  pmis : Prefetch misses per second
   pm% : Prefetch miss percentage
mtxmis : mutex_miss per second
 arcsz : Arc Size
   mm% : Metadata miss percentage
  mrug : MRU Ghost List hits per second
  hits : Arc reads per second
   mfu : MFU List hits per second
   mh% : Metadata hit percentage
  read : Total Arc accesses per second
  Hit% : Arc Hit percentage
  rmis : recycle_miss per second
  mmis : Metadata misses per second
  mhit : Metadata hits per second
  dmis : Demand Data misses per second
   mru : MRU List hits per second
   ph% : Prefetch hits percentage
 eskip : evict_skip per second
     c : Arc Target Size
  mfug : MFU Ghost List hits per second
 miss% : Arc miss percentage
  miss : Arc misses per second
   dm% : Demand Data miss percentage
   dh% : Demand Data hit percentage
  dhit : Demand Data hits per second
  phit : Prefetch hits per second
 mread : Metadata accesses per second
 pread : Prefetch accesses per second

Note: K=10^3 M=10^6 G=10^9 and so on

念のためarcstat.plのコードも載せておきます。

bash-3.00# cat arcstat.pl
#!/bin/perl -w
#
# Print out ZFS ARC Statistics exported via kstat(1)
# For a definition of fields, or usage, use arctstat.pl -v
#
# Author: Neelakanth Nadgir http://blogs.sun.com/realneel
# Comments/Questions/Feedback to neel_sun.com or neel_gnu.org
#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License, Version 1.0 only
# (the "License").  You may not use this file except in compliance
# with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Fields have a fixed width. Every interval, we fill the "v"
# hash with its corresponding value (v[field]=value) using calculate().
# @hdr is the array of fields that needs to be printed, so we
# just iterate over this array and print the values using our pretty printer.

use strict;
use POSIX qw(strftime);
use Sun::Solaris::Kstat;
use Getopt::Long;
use IO::Handle;

my %cols = (# HDR => [Size, Description]
        "Time"  =>[8, "Time"],
        "hits"  =>[4, "Arc reads per second"],
        "miss"  =>[4, "Arc misses per second"],
        "read"  =>[4, "Total Arc accesses per second"],
        "Hit%"  =>[4, "Arc Hit percentage"],
        "miss%" =>[5, "Arc miss percentage"],
        "dhit"  =>[4, "Demand Data hits per second"],
        "dmis"  =>[4, "Demand Data misses per second"],
        "dh%"   =>[3, "Demand Data hit percentage"],
        "dm%"   =>[3, "Demand Data miss percentage"],
        "phit"  =>[4, "Prefetch hits per second"],
        "pmis"  =>[4, "Prefetch misses per second"],
        "ph%"   =>[3, "Prefetch hits percentage"],
        "pm%"   =>[3, "Prefetch miss percentage"],
        "mhit"  =>[4, "Metadata hits per second"],
        "mmis"  =>[4, "Metadata misses per second"],
        "mread" =>[5, "Metadata accesses per second"],
        "mh%"   =>[3, "Metadata hit percentage"],
        "mm%"   =>[3, "Metadata miss percentage"],
        "arcsz" =>[5, "Arc Size"],
        "c"     =>[4, "Arc Target Size"],
        "mfu"   =>[4, "MFU List hits per second"],
        "mru"   =>[4, "MRU List hits per second"],
        "mfug"  =>[4, "MFU Ghost List hits per second"],
        "mrug"  =>[4, "MRU Ghost List hits per second"],
        "eskip" =>[5, "evict_skip per second"],
        "mtxmis"=>[6, "mutex_miss per second"],
        "rmis"  =>[4, "recycle_miss per second"],
        "dread" =>[5, "Demand data accesses per second"],
        "pread" =>[5, "Prefetch accesses per second"],
);
my %v=();
my @hdr = qw(Time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c);
my @xhdr = qw(Time mfu mru mfug mrug eskip mtxmis rmis dread pread read);
my $int = 1;            # Print stats every 1 second by default
my $count = 0;          # Print stats forever
my $hdr_intr = 20;      # Print header every 20 lines of output
my $opfile = "";
my $sep = "  ";         # Default seperator is 2 spaces
my $version = "0.1";
my $cmd = "Usage: arcstat.pl [-hvx] [-f fields] [-o file] [interval [count]]\n";
my %cur;
my %d;
my $out;
my $kstat = Sun::Solaris::Kstat->new();
STDOUT->autoflush;

sub detailed_usage {
        print STDERR "Arcstat version $version\n$cmd";
        print STDERR "Field definitions are as follows\n";
        foreach my $hdr (keys %cols) {
                print STDERR sprintf("%6s : %s\n", $hdr, $cols{$hdr}[1]);
        }
        print STDERR "\nNote: K=10^3 M=10^6 G=10^9 and so on\n";
        exit(1);

}

sub usage {
        print STDERR "Arcstat version $version\n$cmd";
        print STDERR "\t -x : Print extended stats\n";
        print STDERR "\t -f : Specify specific fields to print (see -v)\n";
        print STDERR "\t -o : Print stats to file\n";
        print STDERR "\t -s : Specify a seperator\n\nExamples:\n";
        print STDERR "\tarcstat -o /tmp/a.log 2 10\n";
        print STDERR "\tarcstat -s , -o /tmp/a.log 2 10\n";
        print STDERR "\tarcstat -v\n";
        print STDERR "\tarcstat -f Time,Hit%,dh%,ph%,mh%\n";
        exit(1);
}

sub init {
        my $desired_cols;
        my $xflag = '';
        my $hflag = '';
        my $vflag;
        my $res = GetOptions('x' => \$xflag,
                'o=s' => \$opfile,
                'help|h|?' => \$hflag,
                'v' => \$vflag,
                's=s' => \$sep,
                'f=s' => \$desired_cols);
        $int = $ARGV[0] || $int;
        $count = $ARGV[1] || $count;
        usage() if !$res or $hflag or ($xflag and $desired_cols);
        detailed_usage() if $vflag;
        @hdr = @xhdr if $xflag;         #reset headers to xhdr
        if ($desired_cols) {
                @hdr = split(/[ ,]+/, $desired_cols);
                # Now check if they are valid fields
                my @invalid = ();
                foreach my $ele (@hdr) {
                        push(@invalid, $ele) if not exists($cols{$ele});
                }
                if (scalar @invalid > 0) {
                        print STDERR "Invalid column definition! -- "
                                . "@invalid\n\n";
                        usage();
                }
        }
        if ($opfile) {
                open($out, ">$opfile") ||die "Cannot open $opfile for writing";
                $out->autoflush;
                select $out;
        }
}

# Capture kstat statistics. We maintain 3 hashes, prev, cur, and
# d (delta). As their names imply they maintain the previous, current,
# and delta (cur - prev) statistics.
sub snap_stats {
        my %prev = %cur;
        if ($kstat->update()) {
                printf("\n");
        }
        my $hashref_cur = $kstat->{"zfs"}{0}{"arcstats"};
        %cur = %$hashref_cur;
        foreach my $key (keys %cur) {
                next if $key =~ /class/;
                if (defined $prev{$key}) {
                        $d{$key} = $cur{$key} - $prev{$key};
                } else {
                        $d{$key} = $cur{$key};
                }
        }
}

# Pretty print num. Arguments are width and num
sub prettynum {
        my @suffix=(' ','K', 'M', 'G', 'T', 'P', 'E', 'Z');
        my $num = $_[1] || 0;
        my $sz = $_[0];
        my $index = 0;
        return sprintf("%s", $num) if not $num =~ /^[0-9\.]+$/;
        while ($num > 1000 and $index < 8) {
                $num = $num/1000;
                $index++;
        }
        return sprintf("%*d", $sz, $num) if ($index == 0);
        return sprintf("%*d%s", $sz - 1, $num,$suffix[$index]);
}

sub print_values {
        foreach my $col (@hdr) {
                printf("%s%s", prettynum($cols{$col}[0], $v{$col}), $sep);
        }
        printf("\n");
}

sub print_header {
        foreach my $col (@hdr) {
                printf("%*s%s", $cols{$col}[0], $col, $sep);
        }
        printf("\n");
}

sub calculate {
        %v=();
        $v{"Time"} = strftime("%H:%M:%S", localtime);
        $v{"hits"} = $d{"hits"}/$int;
        $v{"miss"} = $d{"misses"}/$int;
        $v{"read"} = $v{"hits"} + $v{"miss"};
        $v{"Hit%"} = 100*$v{"hits"}/$v{"read"} if $v{"read"} > 0;
        $v{"miss%"} = 100 - $v{"Hit%"} if $v{"read"} > 0;

        $v{"dhit"} = ($d{"demand_data_hits"} + $d{"demand_metadata_hits"})/$int;
        $v{"dmis"} = ($d{"demand_data_misses"}+$d{"demand_metadata_misses"})/$int;
        $v{"dread"} = $v{"dhit"} + $v{"dmis"};
        $v{"dh%"} = 100*$v{"dhit"}/$v{"dread"} if $v{"dread"} > 0;
        $v{"dm%"} = 100 - $v{"dh%"} if $v{"dread"} > 0;

        $v{"phit"}=($d{"prefetch_data_hits"} + $d{"prefetch_metadata_hits"})/$int;
        $v{"pmis"}=($d{"prefetch_data_misses"}
                +$d{"prefetch_metadata_misses"})/$int;
        $v{"pread"} = $v{"phit"} + $v{"pmis"};
        $v{"ph%"} = 100*$v{"phit"}/$v{"pread"} if $v{"pread"} > 0;
        $v{"pm%"} = 100 - $v{"ph%"} if $v{"pread"} > 0;

        $v{"mhit"}=($d{"prefetch_metadata_hits"}+$d{"demand_metadata_hits"})/$int;
        $v{"mmis"}=($d{"prefetch_metadata_misses"}
                +$d{"demand_metadata_misses"})/$int;
        $v{"mread"} = $v{"mhit"} + $v{"mmis"};
        $v{"mh%"} = 100*$v{"mhit"}/$v{"mread"} if $v{"mread"} > 0;
        $v{"mm%"} = 100 - $v{"mh%"} if $v{"mread"} > 0;

        $v{"arcsz"} = $cur{"size"};
        $v{"c"} = $cur{"c"};
        $v{"mfu"} = $d{"hits"}/$int;
        $v{"mru"} = $d{"mru_hits"}/$int;
        $v{"mrug"} = $d{"mru_ghost_hits"}/$int;
        $v{"mfug"} = $d{"mru_ghost_hits"}/$int;
        $v{"eskip"} = $d{"evict_skip"}/$int;
        $v{"rmiss"} = $d{"recycle_miss"}/$int;
        $v{"mtxmis"} = $d{"mutex_miss"}/$int;
}

sub main {
        my $i = 0;
        my $count_flag = 0;

        init();
        if ($count > 0) { $count_flag = 1; }
        while (1) {
                print_header() if ($i == 0);
                snap_stats();
                calculate();
                print_values();
                last if ($count_flag == 1 && $count-- <= 1);
                $i = ($i == $hdr_intr) ? 0 : $i+1;
                sleep($int);
        }
        close($out) if defined $out;
}

&main;

オラクルユニバーシティSolaris系技術トレーニング一覧

• OpenStack Administration Using Oracle Solaris 11 NEW
• Oracle ZFS Storage Appliance Administration NEW
• Oracle Solaris 11 新機能
• Oracle Solaris 11 システム管理 I
• Oracle Solaris 11 システム管理 II
• Oracle Solaris 11 パフォーマンス管理
• Oracle Solaris 11 セキュリティ管理
• Oracle Solaris 11 ZFS 管理
• Oracle Solaris 11 ゾーン管理
• Oracle Solaris 11 トラブルシューティング
• Oracle VM Server for SPARC: Installation and Configuration

Solarisお勧め書籍

Solaris 11.2 システムハンドブック

Oracle Solaris 11 試験対策本(OCA)