From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:47:47 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00EF7106564A for ; Sat, 19 Jun 2010 23:47:47 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 4AEE48FC22 for ; Sat, 19 Jun 2010 23:47:46 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 599AB45FC0; Sun, 20 Jun 2010 01:21:20 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5120D45F83 for ; Sun, 20 Jun 2010 01:21:15 +0200 (CEST) Date: Sun, 20 Jun 2010 01:21:01 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619232101.GI6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: abc X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:47:47 -0000 abc -- Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:47:47 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0966A106566B for ; Sat, 19 Jun 2010 23:47:47 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 4E4A88FC23 for ; Sat, 19 Jun 2010 23:47:46 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 20B6845E86; Sun, 20 Jun 2010 01:17:51 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0D34945DF4 for ; Sun, 20 Jun 2010 01:17:45 +0200 (CEST) Date: Sun, 20 Jun 2010 01:17:31 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619231731.GH6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="17/8oYur5Y32USnW" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Test. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:47:47 -0000 --17/8oYur5Y32USnW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Test. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --17/8oYur5Y32USnW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwdUAsACgkQForvXbEpPzR3XwCgp3pIBvSM1iiC78tZ/OO2srG+ RjUAoN7MiaqchFSakl5XSuh9xvvgLuPe =VGr/ -----END PGP SIGNATURE----- --17/8oYur5Y32USnW-- From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:55:39 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 768D91065670 for ; Sat, 19 Jun 2010 23:55:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id BB7E08FC1C for ; Sat, 19 Jun 2010 23:55:38 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id BEC4A45CDC; Sun, 20 Jun 2010 01:55:36 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 26E22456B1 for ; Sun, 20 Jun 2010 01:55:31 +0200 (CEST) Date: Sun, 20 Jun 2010 01:55:17 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619235517.GJ6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gTY1JhLGodeuSBqf" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Welcome! X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:55:39 -0000 --gTY1JhLGodeuSBqf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello:) Ok, so we now have a mailing list. It is not private, archives can be read through mailman's web interface, but subscriptions are moderated by me for now. Currently we have the following guys here: Josh Carter / Spectra Logic Randy Chou / Panzura Pawel Jakub Dawidek / FreeBSD Samir Desai / Spectra Logic Justin Gibbs / Spectra Logic Steve Jung / Panzura Xin Li / iXsystems, FreeBSD Michael Lin / Panzura Martin Matuska / FreeBSD Ravi Mulam / Panzura Matt Olander / iXsystems Steve Pate / Spectra Logic Tushar Tambay / Spectra Logic John Taylor / Panzura Ganesh Varadarajan / Spectra Logic I hope I got all the names and companies right. If you would like to add someone, let me know privately. To speed things up I went ahead and just subscribed all of you, instead of sending invitiations. If you don't want to be here, please forgive me and let me know (also privately), I'll remove you from the list ASAP. The idea of this list is to coordinate efforts of companies and individuals directly involved in FreeBSD/ZFS development. I must say that after spending few years on FreeBSD/ZFS alone and failing to get more people involved, I'm very happy to see so many names here:) --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --gTY1JhLGodeuSBqf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwdWOQACgkQForvXbEpPzS5ZQCfYSvpuswzh7IXolplrNCyVfXG O4AAnAqcQWmyqUzfdRbYgc446IosLtI2 =m4z5 -----END PGP SIGNATURE----- --gTY1JhLGodeuSBqf-- From owner-zfs-devel@FreeBSD.ORG Sun Jun 20 09:19:03 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5099E1065672 for ; Sun, 20 Jun 2010 09:19:03 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 887298FC16 for ; Sun, 20 Jun 2010 09:19:02 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A3C69CC69C for ; Sun, 20 Jun 2010 11:01:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id VdAbqzF1D51R for ; Sun, 20 Jun 2010 11:01:11 +0200 (CEST) Received: from [10.9.8.1] (chello085216227216.chello.sk [85.216.227.216]) by mail.vx.sk (Postfix) with ESMTPSA id 53191CC695 for ; Sun, 20 Jun 2010 11:01:11 +0200 (CEST) Message-ID: <4C1DD8D9.5090404@FreeBSD.org> Date: Sun, 20 Jun 2010 11:01:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS development strategy X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jun 2010 09:19:03 -0000 Hi Pawel, Xin and all members of this list. First of all, I would like to thank Pawel for all the work on porting ZFS to FreeBSD and for creating this list. I hope this list will help us to exchange new ideas, help each other with existing issues or discuss how things should be implemented. As the very first topic, I would like to discuss our ZFS development strategy. To coordinate our work as best as possible, it would be very nice to have a common set of goals and targets or at least some basic strategy guidelines we should follow. I understand everybody has some kind of "own" agenda, but we should e.g. avoid reimplementing things 2 times (and sometimes both in different ways). At current state, we have quite well-tested post-v14 in head, stable/8, post-v13 in stable/7, and latest work (post-v26) in pjd's perforce tree. My main effort until now has been finding and importing bugfixes and/or important improvements (e.g. parallel traversal, l2arc) from OpenSolaris that do or may affect FreeBSD users into head and stable. If I continue this effort I could easily land in v15 as a pre-step for v26 (v15 introduces group and user quotas and splits the python code as well). What I also plan to do is creating a python dummy port for the python code part. So I would maily like to ask the following questions about ourt ZFS: 1. Are we interested in setting up some goals / guidelines for our ZFS in FreeBSD to help coordinating our work? 2. Is there a (very rough) estimate when v26 might find its way into wider testing and head? 3. If the answer for 2. lies in far future, does it make sense to go v15 as a step inbetween (including or not-including the python dummy port)? Thank you very much, mm From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 17:00:43 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A05E1065670; Tue, 22 Jun 2010 17:00:43 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.211.181]) by mx1.freebsd.org (Postfix) with ESMTP id E621A8FC13; Tue, 22 Jun 2010 17:00:42 +0000 (UTC) Received: by ywh11 with SMTP id 11so3657076ywh.7 for ; Tue, 22 Jun 2010 10:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=fz/GRygvqQbJzOtg5eMUJV3qT9/ZWfJTgDOes4p7AOk=; b=rSasFyYsVsJbW15owLpHg7f2qWPdfY2v6Km/QE5EJBapAA/2Q5UxQhGaOBceYJtoBl h9T03OzGWNI5821bLqrhJV02V981mLM4M3an0Mp/pvjQ+hly9EGIlySgFtzrCU66I2V2 WI7taY3MwMpwozgJ7fULf5Sp9z5Fe7SZfE9io= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=M6gItA8LYA0l6StcoMiaUI2hOfbOYnRfyxl1Giz5UNrIGpzIl7p+ydtgvmY7ptltGr kvnhy1fuZ9LoDyRA3CMwpEUrQXH2y3OhqWoxGFJVxY1zTU/3zjhEQsUKum80tdKnV7Rp 6noMczzdR7HgOfL0hOKH9GuC0fCf++1JjxpLc= MIME-Version: 1.0 Received: by 10.150.172.1 with SMTP id u1mr6082693ybe.240.1277224300482; Tue, 22 Jun 2010 09:31:40 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.150.191.5 with HTTP; Tue, 22 Jun 2010 09:31:40 -0700 (PDT) In-Reply-To: <4C1DD8D9.5090404@FreeBSD.org> References: <4C1DD8D9.5090404@FreeBSD.org> Date: Tue, 22 Jun 2010 09:31:40 -0700 X-Google-Sender-Auth: LAoe_36N7Xnliedqa1qsLZq6YUM Message-ID: From: Tushar Tambay To: Martin Matuska , zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: ZFS development strategy X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 17:00:43 -0000 Hi all, I haven't got everyones skype ids yet and I do not have a conf call # that has a toll free dial-in from poland and india. So first question - does anyone have a conf call number we can use ? If not, just as a backup, can you folks send out you skype ids please ? On 6/20/10, Martin Matuska wrote: > Hi Pawel, Xin and all members of this list. > > First of all, I would like to thank Pawel for all the work on porting > ZFS to FreeBSD and for creating this list. > I hope this list will help us to exchange new ideas, help each other > with existing issues or discuss how things should be implemented. > > As the very first topic, I would like to discuss our ZFS development > strategy. > > To coordinate our work as best as possible, it would be very nice to > have a common set of goals and targets or at least some basic strategy > guidelines we should follow. I understand everybody has some kind of > "own" agenda, but we should e.g. avoid reimplementing things 2 times > (and sometimes both in different ways). > > At current state, we have quite well-tested post-v14 in head, stable/8, > post-v13 in stable/7, and latest work (post-v26) in pjd's perforce tree. > > My main effort until now has been finding and importing bugfixes and/or > important improvements (e.g. parallel traversal, l2arc) from OpenSolaris > that do or may affect FreeBSD users into head and stable. If I continue > this effort I could easily land in v15 as a pre-step for v26 (v15 > introduces group and user quotas and splits the python code as well). > What I also plan to do is creating a python dummy port for the python > code part. > > So I would maily like to ask the following questions about ourt ZFS: > 1. Are we interested in setting up some goals / guidelines for our ZFS > in FreeBSD to help coordinating our work? > 2. Is there a (very rough) estimate when v26 might find its way into > wider testing and head? > 3. If the answer for 2. lies in far future, does it make sense to go v15 > as a step inbetween (including or not-including the python dummy port)? > > Thank you very much, > mm > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Sent from my mobile device tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 20:23:23 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D07D1065670 for ; Tue, 22 Jun 2010 20:23:23 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.211.181]) by mx1.freebsd.org (Postfix) with ESMTP id 174448FC1C for ; Tue, 22 Jun 2010 20:23:22 +0000 (UTC) Received: by ywh11 with SMTP id 11so3777536ywh.7 for ; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=1B3d5kKzLyDlhOf3MZPxzjeHYHLB/hL06Ljmuk1PQhc=; b=jIDOIMoBb2kpJCiSnF6yMFUfe7Pc9wLpZO0lhGSTS29UkUxV9McaGcKNBnt3sqxb+l VYYY6yi1kIguUqzmwDfWv0Je/EoZKAb+DjPRoUdqW++z9N7N0i9mHfWWrTUqMcyvFp9Z mlScJ7/SdafOTJ0tS3qRC6GbLPKDHnJTFh/k4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=jfTv3PxfHvAF1sg3sNyA/FT6FZ+5kWIm7fIE7wmNC1C6g+47TXsKeLZKkCx/C0YPnh xF+y9Zz5TaXLTiV9taMbRSH6tyO7BBUh9cjhYaaIFehvnNOjPbN2geqLQytNuQfDNjLb 3OQpxz61GNCdgAnoyvO7nBPtZuMmgH6RuVyqs= MIME-Version: 1.0 Received: by 10.150.208.19 with SMTP id f19mr6761359ybg.208.1277238202175; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.150.191.5 with HTTP; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) Date: Tue, 22 Jun 2010 13:23:22 -0700 X-Google-Sender-Auth: JeZXc8J3q-h9eT8G05Kl1JFeGw4 Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs/bsd conf call minutes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 20:23:23 -0000 date : 7/22/10 attendees : justin, pawel, xin, ganesh, samir, steve, tushar, john development branch & checkin logistics - /project/zfs/stable/8 will be our primary collaborative branch - everyone should have checkin privileges for this already - justin will do a weekly merge of changes/fixes from the bsd/head and bsd/stable branches to /project/zfs/stable/8 and /project/zfs/current - pawel will request martin to do a weekly merge of changes/fixes from /user/pjd/zfs branch to the /project/zfs/stable/8 and /project/zfs/current - ganesh will send out a description of zfs test suite layout to zfs-devel@freebsd.org alias and run the discussion to decide where the test suite will be checked in - samir will establish a checkin criteria based on zfs test suite - we will delay discussing the details of translation layer required to support zfs v24 on the official 8.x BSD release until we get a better feel for stability/performance of zfs zfs / python - on opensolaris, zfs utilities have been re-written in python (earlier in C) - python is not available in base bsd disribution (only in ports) - we will keep zfs utilities in python and include zfs in base bsd distribution but users will need to configure their systems with the python port to get all the zfs functionality zfs/dedupe - suspected dedupe performance issue needs to be quantified with further testing - dedupe performance primarily tied to how much of the index can be cached. - zfs has the facility to introduce a second level cache using (say) SSD storage but this needs testing/verification. - pawel suspects a priority/scheduling issue with dedupe - under heavy dedupe load, system becomes unresponsive/deadlocked. - no dedupe tests exist in recently ported test suite zfs/memory issue - KVA exhaustion is a problem but limited to 32bit platforms - besides KVA, there are other memory usage issues (though not very severe). pawel's recent fix to reclaim memory used by name-cache should address some of these - BSD memory tuning is primarily for ufs. so assumptions about inode and other sizes need to be revisited for zfs. zfs / io - (pawel) currently zfs commits changes in transaction groups every 10 (earlier 30) seconds and flushes whenever ARC is full. - (justin) zfs flushing to disk is v. bursty even when applied i/o load is very uniform. - zfs needs better flush behind algorithms for (application) i/o - justin will fix broken disk elevator code. he has seen data corruption / unmountable zfs file systems that are caused by this. zfs / zvols - bioflush has no sync/async flag. a big problem for zfs/zvol - currently all i/o is treated as async and callers need to call another interface to force flush - ... pretty sure i've got some details wrong on this. justing - can you please correct this ? next conf call - 7/20/2010 cheers, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 21:47:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 567F91065690 for ; Tue, 22 Jun 2010 21:47:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id A83DE8FC2F for ; Tue, 22 Jun 2010 21:47:20 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 6F8A3D4415 for ; Tue, 22 Jun 2010 23:47:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SO7xvB23ru+v for ; Tue, 22 Jun 2010 23:47:17 +0200 (CEST) Received: from [10.9.8.1] (chello085216227216.chello.sk [85.216.227.216]) by mail.vx.sk (Postfix) with ESMTPSA id 232E0D4408 for ; Tue, 22 Jun 2010 23:47:17 +0200 (CEST) Message-ID: <4C212F68.5060701@FreeBSD.org> Date: Tue, 22 Jun 2010 23:47:20 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS v15 patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 21:47:21 -0000 Dear ZFS developers, As the user/group quotas feature is too much attractive for my needs, I couldn't resist and have created (and debugged + tested) a ZFS v15 patch for head (applies cleanly against stable/8 as well). It is a backport of several onnv-revisions, always consulting pjd's p4 tree and includes four post-9396 related user/groupquota bugfixes. The bootcode (zfsimpl.h) is properly updated to support v15 as well, the python part is modified (paths, smb support, ioctls). The patch, list of imported revisions and a preliminary but working version of the pyzfs port can be downloaded from this link: http://people.freebsd.org/~mm/patches/zfs/v15/ You can download 8.1-RC1-amd64 with zfsv15 (incl. boot support) mfsBSD ISOs from here: http://mfsbsd.vx.sk/iso/8.1rc1-zfsv15.iso (without symbols, 98.7M) http://mfsbsd.vx.sk/iso/8.1rc1-zfsv15-debug.iso (with symbols, 187.5M) Login/password into the ISOs: root/mfsroot run "zfsinstall" for a zfs-on-root installation (don't forget -V 15 if trying to install) The amd64 ISO's may be tested in virtualbox or real-world systems as well. Regarding new ioctl's: I have added all new ioctl's in the patch as new ones (numbers 49 to 52). This makes the old kernel module work with new library / utilities (tested) or any other binary that references these ioctl's. Regarding the python port: The current port installs the library into PYTHON_SITELIBDIR and pyzfs.py into /usr/lib/zfs/ Here is an idea how the port could work in the future: a) FreeBSD installs /usr/lib/zfs/pyzfs.py (or whatever location desired) by default with #!/usr/bin/env python b) the pyzfs library is installed from ports (dummy or distfile-bound port) Version 15 introduces user and group quota accounting and could be a very good step in preparation towards v26. The code runs impressingly stable (I didn't manage to reproduce any panics or deadlocks yet). It would be very great if this could get widely tested. I guess we could go with this code in direction stable/8 (head first), because it is backwards compatible (zfs and zpool work with older kernel module - only new features don't work, the same for the boot changes, boot from older pools is supported). Any ideas, suggetions, etc. are welcome! Cheers, mm From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 05:18:29 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C582A106566C; Wed, 30 Jun 2010 05:18:29 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6AEA78FC18; Wed, 30 Jun 2010 05:18:29 +0000 (UTC) Received: by gwj16 with SMTP id 16so263313gwj.13 for ; Tue, 29 Jun 2010 22:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=QjJJasVF/rLKcSNDu0oTrF5GPYGB9Kxs+TWnMp0Z6Z0=; b=Yz4illwPRbUDyujN7drSFHZQGCXhpVzDui2WKfS+m2DRcA9V6+Lj31+hWWqdnjK1cs aLL1iC/bEgx2Qj0BXcKtm6c498cK8zlAM41PNn7tHE96hcvyXUol69qC3sZw+FoN+mQS DCjziSo5XRI8TJc4RTTTWd8J7OgkL5x1zsAm4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=v/Fcgt00kDAkFHzRXOjjAB7BGHkb0lwJn7Zg+e/1EMofT/j5ZldYPo3piQ8V5jITkM c/610JhQnhnF1dN1q/W9x1ZH2Y/8atHuJXCIXvMl9kltviPfWTz59rgUoy61K4OC2ZAu bsx7fvXRJXiUn75R1rKETZR1YtlxoyeyG1aSA= MIME-Version: 1.0 Received: by 10.90.107.5 with SMTP id f5mr6880432agc.18.1277875100600; Tue, 29 Jun 2010 22:18:20 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.151.101.5 with HTTP; Tue, 29 Jun 2010 22:18:20 -0700 (PDT) Date: Tue, 29 Jun 2010 22:18:20 -0700 X-Google-Sender-Auth: EfsTd1tIPy91cLayGS_dahuNTK0 Message-ID: From: Tushar Tambay To: pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 05:18:29 -0000 hi pawel, we are ready to check in some changes to the common zfs tree, but before that happens, the zfs bits in /project/zfs/stable/8 will need to be updated to the latest from your tree. my understanding is that you were going to request martin to do this. is there any further update on this ? thanks in advance, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 07:05:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0B3106566B for ; Wed, 30 Jun 2010 07:05:04 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 759C28FC13 for ; Wed, 30 Jun 2010 07:05:03 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 24AEC45E6F; Wed, 30 Jun 2010 09:05:01 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E8E1745E48; Wed, 30 Jun 2010 09:04:55 +0200 (CEST) Date: Wed, 30 Jun 2010 09:04:37 +0200 From: Pawel Jakub Dawidek To: Tushar Tambay Message-ID: <20100630070437.GD1816@garage.freebsd.pl> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YToU2i3Vx8H2dn7O" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 07:05:04 -0000 --YToU2i3Vx8H2dn7O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 29, 2010 at 10:18:20PM -0700, Tushar Tambay wrote: > hi pawel, >=20 > we are ready to check in some changes to the common zfs tree, but before > that happens, the zfs bits in /project/zfs/stable/8 will need to be updat= ed > to the latest from your tree. >=20 > my understanding is that you were going to request martin to do this. is > there any further update on this ? Well, I'm meeting Martin in two days at meetBSD and I'd like to explain the whole thing to him there. My idea is to sync to more or less working changeset (last changeset of user/pjd/zfs/... doesn't even compile atm) and then start to sync only selected changes (bug fixes mostly). What I'm doing in user/pjd/zfs/... is to sync all the changes from OpenSolaris and we don't want that in projects/zfs/..., because changes from OpenSolaris are comming faster than we can test them and fix them, so we have to decide where exactly is the point we want to focus on and don't bring any new features from OpenSolaris to projects/zfs/... - from this point we should only sync bug fixes from usr/pjd/zfs/... I'll follow up after meetBSD. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --YToU2i3Vx8H2dn7O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwq7IUACgkQForvXbEpPzTDAACdEtY5lOlxXrTLHcuHd5cwHCEd YVgAn3yTGhYKL1nL5uxzyAnDP62juetH =9G27 -----END PGP SIGNATURE----- --YToU2i3Vx8H2dn7O-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 07:24:23 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8E86106564A; Wed, 30 Jun 2010 07:24:23 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id D4DC38FC0C; Wed, 30 Jun 2010 07:24:21 +0000 (UTC) Received: by gxk7 with SMTP id 7so292605gxk.13 for ; Wed, 30 Jun 2010 00:24:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=fiUzXI3zCIMUbB/a3SReyQAYb+jpRxLIGLbTp3nZBmM=; b=IvK5JOqfyCgCKkdDxQFNK+Bvn1f9bfPw3Dmg2yJGJiY1mSoQ2KZcosbYSUgWmVfYNu YfTfzju9OvWkNDyeLAFS5fwfx1A7ENFQAdfoSeWi0KMMDu1Ec9kF10qwiAXZ8gh+RU/6 reQ+6XUZjvLcr4tiMBnSehsIlKk1hHLJYdXCc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=ulrNpVVPxiMnP4Fvph3xfmOVrn48T+ORkjM3I2q2XSwyZKUDPQK1NsuoKEFTOBPsRk 1451CJuJGzFok19hnMsqi5j7H0hHcw2Vd7uwU2tb/2CTicnwG4j78AnzJhzRVNOg+Dis e/FJnpDVU7MD9UhNFepeFRfZbrRniPb0NHf3o= MIME-Version: 1.0 Received: by 10.90.63.18 with SMTP id l18mr5695659aga.154.1277882643890; Wed, 30 Jun 2010 00:24:03 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.151.101.5 with HTTP; Wed, 30 Jun 2010 00:24:03 -0700 (PDT) In-Reply-To: <20100630070437.GD1816@garage.freebsd.pl> References: <20100630070437.GD1816@garage.freebsd.pl> Date: Wed, 30 Jun 2010 00:24:03 -0700 X-Google-Sender-Auth: UzeTFJKRBFHG-AM2iXnTee5FU-c Message-ID: From: Tushar Tambay To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 07:24:23 -0000 thanks pawel ! On Wed, Jun 30, 2010 at 12:04 AM, Pawel Jakub Dawidek wrote: > On Tue, Jun 29, 2010 at 10:18:20PM -0700, Tushar Tambay wrote: > > hi pawel, > > > > we are ready to check in some changes to the common zfs tree, but before > > that happens, the zfs bits in /project/zfs/stable/8 will need to be > updated > > to the latest from your tree. > > > > my understanding is that you were going to request martin to do this. is > > there any further update on this ? > > Well, I'm meeting Martin in two days at meetBSD and I'd like to explain > the whole thing to him there. > > My idea is to sync to more or less working changeset (last changeset of > user/pjd/zfs/... doesn't even compile atm) and then start to sync only > selected changes (bug fixes mostly). > > What I'm doing in user/pjd/zfs/... is to sync all the changes from > OpenSolaris and we don't want that in projects/zfs/..., because changes > from OpenSolaris are comming faster than we can test them and fix them, > so we have to decide where exactly is the point we want to focus on and > don't bring any new features from OpenSolaris to projects/zfs/... - from > this point we should only sync bug fixes from usr/pjd/zfs/... > > I'll follow up after meetBSD. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 1 08:07:06 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 292BD106566C for ; Thu, 1 Jul 2010 08:07:06 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 5E2448FC1B for ; Thu, 1 Jul 2010 08:07:05 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 74451DAC56 for ; Thu, 1 Jul 2010 10:07:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 0p22fpFjI33q for ; Thu, 1 Jul 2010 10:07:02 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 8343CDAC48 for ; Thu, 1 Jul 2010 10:07:02 +0200 (CEST) Message-ID: <4C2C4CA6.4050808@FreeBSD.org> Date: Thu, 01 Jul 2010 10:07:02 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------070804070500090504050209" Cc: Subject: Bugfixes backported to Solaris 10 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2010 08:07:06 -0000 This is a multi-part message in MIME format. --------------070804070500090504050209 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Hi all, attached is a list of bugfixes and corresponding onnv-revisions that have been backported to Solaris 10 until today (kernel patch 142901-09). Information if the corresponding bugfix is already in head/stable is included. --------------070804070500090504050209 Content-Type: text/plain; name="zfs_bugfixes_solaris_10.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_bugfixes_solaris_10.txt" List of ZFS v15 bugfixes backported by Solaris 10 142901-09 --------- 9515:d3b739d9d043 (not imported, head-v16-extended-v3.patch) 6586537 async zio taskqs can block out userland commands 9790:e276ee006ff6 (not imported) 6747441 GRUB/vdev_get_bootpath, spa_get_rootconf, zpool_get_physpath should take care of spare vdev 6844158 assertion failed: vd->vdev_ops == &vdev_mirror_ops, file: ../../common/fs/zfs/zvol.c, line: 1054 10896:007ca67ca400 (not imported) 6793877 lockd and Apache can block ZFS force-unmounting on behalf of clients 10801:e0bf032e8673 (not imported, head-v16-extended-v3.patch) 6822816 assertion failed: zap_remove_int(ds_next_clones_obj) returns ENOENT 11546:42ea6be8961b (not imported) 6833999 3-way deadlock in dsl_dataset_hold_ref() and dsl_sync_task_group_sync() 10474:0e96dd3b905a (head: r208130, stable/8: r208288) 6847118 hang in ZFS during rollback 10800:469478b180d9 (not imported, head-v16-extended-v2.patch) 6880764 fsync on ZFS is broken if writes are greater than 32kb on a hard crash and no log attached 10810:b6b161a6ae4a (not imported, head-v16-extended-v3.patch) 6892298 buf->b_hdr->b_state != arc_anon, file: ../../common/fs/zfs/arc.c, line: 2849 142901-10 --------- 11321:506b7043a14c (head: r208131, stable/8: r208288) 6847615 deadlock between zfs_dirent_lock and zfs_rmdir 11454:6e69bacc1a5a (not imported, head-v16-extended-v3.patch) 6898245 suspended zpool should not cause rest of the zfs/zpool commands to hang 142901-12 --------- 12079:13822b941977 (not imported) 6939941 problem with moving files in ZFS 142901-13 --------- 10890:499786962772 (not imported, head-v16-extended-v3.patch) 6807339 spurious checksum errors when replacing a vdev 11249:6c30f7dfc97b (not imported, head-v16-extended.patch) 6906110 bad trap panic in zil_replay_log_record 6906946 ZFS replay isn't handling uid/gid correctly 142901-14 --------- 10839:cf83b553a2ab (head: r209101, stable/8: r209274) 6836714 arc_read_done may try to byteswap undefined data 10575:2a8816c5173b (not imported) 6882227 spa_async_remove() shouldn't do a full clear --------------070804070500090504050209-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 6 21:22:50 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5D6D106566B for ; Tue, 6 Jul 2010 21:22:50 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2E2978FC1A for ; Tue, 6 Jul 2010 21:22:48 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C7EE145E8E; Tue, 6 Jul 2010 23:22:46 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 980C845D8D for ; Tue, 6 Jul 2010 23:22:40 +0200 (CEST) Date: Tue, 6 Jul 2010 23:22:19 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100706212219.GD1724@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mJm6k4Vb/yFcL9ZU" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Some updates. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2010 21:22:50 -0000 --mJm6k4Vb/yFcL9ZU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. During meetBSD in Poland few days ago we had some time to discuss few ZFS-related things. Martin Matuska agreed to work on merging bug fixes from user/pjd/zfs/... to projects/zfs/head/... and to projects/zfs/stable/8/... I'd suggest merging everything from user/pjd/zfs/... except for @179537, which doesn't compile yet to projects/zfs/.... Without this change it should more or less work. Martin is going to upgrade ZFS we have in HEAD to v15. This is also what Solaris is using. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --mJm6k4Vb/yFcL9ZU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwznosACgkQForvXbEpPzQ+qgCg3j+XtM5MN1jWWmMe6tAFmwL+ TGgAnjKHKS12ybyg37OliT4a292yC3Lq =cfsc -----END PGP SIGNATURE----- --mJm6k4Vb/yFcL9ZU-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 7 06:37:38 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B0F41065675 for ; Wed, 7 Jul 2010 06:37:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id BEB788FC18 for ; Wed, 7 Jul 2010 06:37:36 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 688A0F0227 for ; Wed, 7 Jul 2010 08:37:35 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id zwzMfr22hAbJ for ; Wed, 7 Jul 2010 08:37:33 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 5909CF0220 for ; Wed, 7 Jul 2010 08:37:33 +0200 (CEST) Message-ID: <4C3420AD.8040709@FreeBSD.org> Date: Wed, 07 Jul 2010 08:37:33 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2010 06:37:38 -0000 Hi all, I have some update on the status of the v15 upgrade: - crafted a new patch that follows steps of Solaris 10 going v15 (many more revisions added) - released a CFT on freebsd-current@, with very detailed patch information including Solaris 10 patch numbers - positive testing reports from running on i386, amd64 and sparc systems (and cross-importing pools, etc.) - the patch is now ready for head, I will commit it right after 8.1 gets released - an UPDATING entry will give notice about pool upgrading and that the new utilities need a new kernel I have tested the binary compatibility with older utilities as well, and it works properly, here is the matrix: Old kernel, old userland: OK Old kernel, new userland: ERROR New kernel, old userland: OK New kernel, new userland: OK So this fullfills the import constraint into stable/8, I will tag the commit with a 2-month MFC after: Link to the patch information file, including all imported revisions, bug-ids and reference to Solairis 10 patch numbers: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt Direct link to the patch: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch And I repeat: the patch applies cleanly against stable/8 as well (and works). Cheers, mm From owner-zfs-devel@FreeBSD.ORG Wed Jul 7 14:34:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3460106564A; Wed, 7 Jul 2010 14:34:34 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7B0AE8FC1F; Wed, 7 Jul 2010 14:34:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o67EWjcX031759; Wed, 7 Jul 2010 08:32:45 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 07 Jul 2010 08:33:01 -0600 (MDT) Message-Id: <20100707.083301.149607579557318410.imp@bsdimp.com> To: mm@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <4C3420AD.8040709@FreeBSD.org> References: <4C3420AD.8040709@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2010 14:34:34 -0000 In message: <4C3420AD.8040709@FreeBSD.org> Martin Matuska writes: : Hi all, : : I have some update on the status of the v15 upgrade: : : - crafted a new patch that follows steps of Solaris 10 going v15 (many : more revisions added) : - released a CFT on freebsd-current@, with very detailed patch : information including Solaris 10 patch numbers : - positive testing reports from running on i386, amd64 and sparc systems : (and cross-importing pools, etc.) : - the patch is now ready for head, I will commit it right after 8.1 gets : released : - an UPDATING entry will give notice about pool upgrading and that the : new utilities need a new kernel : : I have tested the binary compatibility with older utilities as well, and : it works properly, here is the matrix: : : Old kernel, old userland: OK : Old kernel, new userland: ERROR : New kernel, old userland: OK : New kernel, new userland: OK : : So this fullfills the import constraint into stable/8, I will tag the : commit with a 2-month MFC after: Actually, old kernel + new userland == ERROR does not necessarily fulfill the import constraints. What does ERROR mean here? For -current it might be alright (might here depends on what the consequences are). For -stable, the bar is much higher. Again, it might be OK, but what are the consequences of this error? While the project might not support totally arbitrary movements with kernel and userland, each case is judged based on the consequences of the errors. Warner : Link to the patch information file, including all imported revisions, : bug-ids and reference to Solairis 10 patch numbers: : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt : : Direct link to the patch: : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch : : And I repeat: the patch applies cleanly against stable/8 as well (and : works). : : Cheers, : mm : _______________________________________________ : zfs-devel@freebsd.org mailing list : http://lists.freebsd.org/mailman/listinfo/zfs-devel : To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" : : From owner-zfs-devel@FreeBSD.ORG Thu Jul 8 10:51:48 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D6B8106564A for ; Thu, 8 Jul 2010 10:51:48 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id D0ABC8FC08 for ; Thu, 8 Jul 2010 10:51:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id DE6C4F361A; Thu, 8 Jul 2010 12:51:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 0Z2eVyt1K5Ff; Thu, 8 Jul 2010 12:51:44 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id B6B9EF3610; Thu, 8 Jul 2010 12:51:44 +0200 (CEST) Message-ID: <4C35ADC2.7060900@FreeBSD.org> Date: Thu, 08 Jul 2010 12:51:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: "M. Warner Losh" References: <4C3420AD.8040709@FreeBSD.org> <20100707.083301.149607579557318410.imp@bsdimp.com> In-Reply-To: <20100707.083301.149607579557318410.imp@bsdimp.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2010 10:51:48 -0000 As for good news, I have fixed this as well. Now it is: Old kernel, old userland: OK (both zfs and zpool work) Old kernel, new userland: OK (both zfs and zpool work) New kernel, old userland: OK (both zfs and zpool work) New kernel, new userland: OK (both zfs and zpool work) The problem was that in v15 the check for internal private datasets was moved out of userland to kernel. So it was very easy to introduce a very simple compatibility fix and check for "$" as the first character in appropriate userland parts, too. The patch for the internal private datasets is here: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3-compat.patch (head-v15-v3.patch needs to be applied first) Dňa 7. 7. 2010 16:33, M. Warner Losh wrote / napísal(a): > In message: <4C3420AD.8040709@FreeBSD.org> > Martin Matuska writes: > : Hi all, > : > : I have some update on the status of the v15 upgrade: > : > : - crafted a new patch that follows steps of Solaris 10 going v15 (many > : more revisions added) > : - released a CFT on freebsd-current@, with very detailed patch > : information including Solaris 10 patch numbers > : - positive testing reports from running on i386, amd64 and sparc systems > : (and cross-importing pools, etc.) > : - the patch is now ready for head, I will commit it right after 8.1 gets > : released > : - an UPDATING entry will give notice about pool upgrading and that the > : new utilities need a new kernel > : > : I have tested the binary compatibility with older utilities as well, and > : it works properly, here is the matrix: > : > : Old kernel, old userland: OK > : Old kernel, new userland: ERROR > : New kernel, old userland: OK > : New kernel, new userland: OK > : > : So this fullfills the import constraint into stable/8, I will tag the > : commit with a 2-month MFC after: > > Actually, old kernel + new userland == ERROR does not necessarily > fulfill the import constraints. What does ERROR mean here? For > -current it might be alright (might here depends on what the > consequences are). For -stable, the bar is much higher. Again, it > might be OK, but what are the consequences of this error? > > While the project might not support totally arbitrary movements with > kernel and userland, each case is judged based on the consequences of > the errors. > > Warner > > : Link to the patch information file, including all imported revisions, > : bug-ids and reference to Solairis 10 patch numbers: > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt > : > : Direct link to the patch: > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch > : > : And I repeat: the patch applies cleanly against stable/8 as well (and > : works). > : > : Cheers, > : mm > : _______________________________________________ > : zfs-devel@freebsd.org mailing list > : http://lists.freebsd.org/mailman/listinfo/zfs-devel > : To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > : > : > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Thu Jul 8 19:05:20 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 786D41065670; Thu, 8 Jul 2010 19:05:20 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 212288FC0A; Thu, 8 Jul 2010 19:05:20 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 1D75AA59968; Fri, 9 Jul 2010 03:05:15 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id BlRyfZGc+oNQ; Fri, 9 Jul 2010 03:05:04 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 08A44A59980; Fri, 9 Jul 2010 03:04:54 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=GOaK6S1Bh68IBnPqEKbTa73U4hu5/n1TPjyuIG+akapHEGsUVG8ZQpJRALRrc+7ih XYP0VgVDdp0svhFRVqQOg== Message-ID: <4C362153.5010604@delphij.net> Date: Thu, 08 Jul 2010 12:04:51 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100629 Thunderbird/3.0.5 ThunderBrowse/3.3 MIME-Version: 1.0 To: Martin Matuska References: <4C3420AD.8040709@FreeBSD.org> <20100707.083301.149607579557318410.imp@bsdimp.com> <4C35ADC2.7060900@FreeBSD.org> In-Reply-To: <4C35ADC2.7060900@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2010 19:05:20 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2010/07/08 03:51, Martin Matuska wrote: > As for good news, I have fixed this as well. > > Now it is: > > Old kernel, old userland: OK (both zfs and zpool work) > Old kernel, new userland: OK (both zfs and zpool work) > New kernel, old userland: OK (both zfs and zpool work) > New kernel, new userland: OK (both zfs and zpool work) > > The problem was that in v15 the check for internal private datasets was > moved out of userland to kernel. > So it was very easy to introduce a very simple compatibility fix and > check for "$" as the first character in appropriate userland parts, too. > > The patch for the internal private datasets is here: > http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3-compat.patch > > (head-v15-v3.patch needs to be applied first) That's nice! Thanks for working on this! (Reading the code it seems that '$' is the indicator of hidden dataset regardless whether it's the first character. -CURRENT kernel uses the same strchr() as the patch did.) Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iQEcBAEBCAAGBQJMNiFTAAoJEATO+BI/yjfBcR0IAIsoMjj2Y+0nHNd5lt6/uPVG S+8V0cNj1pTEmg3nqMcLdmTwYrFnS8ueXdJna295Ky2Qkv4KKRtK2cAHQGr5PMPD WF7twJpU7hvBvfO5gduRulZ9yMxchXsWEOezg6nFHvfqjBTIatUrEhWTPM5GHSoD junaAn+McO8p+ysMj1pR0Vwb1YqVOHT+ZQzqt57XH/JsiDD97Qpq+44D+QLltCmd xK8Ht+0tuxbrWgBLoCPbClO3wplpdhi8vdh4GhfUrry9WzHKtHJuhYq3e10K6ZVu yVRrkqb0Z8jFaAtju9CFx2lnExCm1x0+rDu3564C7wLsUWK634wxy0yW3IdEkI8= =2WbQ -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Tue Jul 20 16:41:01 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 744E4106566B for ; Tue, 20 Jul 2010 16:41:01 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 497148FC14 for ; Tue, 20 Jul 2010 16:41:01 +0000 (UTC) Received: by pwj9 with SMTP id 9so2579423pwj.13 for ; Tue, 20 Jul 2010 09:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=eUuaz7lCoi4jfUJyksZZe+jcvCEFwvX/WlPK0fZsDgE=; b=JE+9KJM4Dj6PKyeVKUjzkBivAwkDTlGnrZ660ROb5UeJHgWcuAyCBRLjxVok+uV+Wg URHouP86oc5XUDUa2lw98KCvwzH8EcGAJYYmpzcJAKmc8dSuSMI0Es1+JXvwGcvvfu7K lJaVA8m0c50JcUzL4BEvoFatfnVihdw1gASew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=rdKs/IemXnlQp74Ka6nTpdpTD3ocDeY0Bdtey9pexawihoFhzN19qBh8zmnXYrkZS4 JxY93jg25rXL/zSySK1CKJEYTOa201d6WKB86Al7a3sjVXnRL3vB5Svx9Wz0A/sOmtZc oJXDLwu2Uf/bS/e3lpPiH4Tfda1P2j07o4cOI= MIME-Version: 1.0 Received: by 10.142.148.10 with SMTP id v10mr9535691wfd.105.1279644059397; Tue, 20 Jul 2010 09:40:59 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.218.146 with HTTP; Tue, 20 Jul 2010 09:40:58 -0700 (PDT) Date: Tue, 20 Jul 2010 09:40:58 -0700 X-Google-Sender-Auth: rvn6ZePQa0g2Rhs46yDJJvQ9g4I Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: conf call today X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 16:41:01 -0000 hi all, we are supposed to be having our second monthly call today at 10 AM PST but i am guessing that most people have forgotten that it was today :-) apologies for not sending out a reminder mail yesterday. i meant to but did not get around to it. i propose we move the call this week to thursday 10 AM PST. please respond if that does that not work for you. cheers, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:14:03 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DF1B1065673 for ; Thu, 22 Jul 2010 17:14:03 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 06D9C8FC16 for ; Thu, 22 Jul 2010 17:14:02 +0000 (UTC) Received: by gwj23 with SMTP id 23so258387gwj.13 for ; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=+KJVZMAmFWS/bkiMD+ZLHVFB2RbKtm0dJ80cgY2AkV4=; b=ecLyjWKzCW4burpufKRSK7zeblN8F96lIG9QisT+AhtT+dRNqfNVRmxs3sxLAoOsoG oP8we2vDsFCiE0kwur0Pxo4yI0mJC2Pr5nrEWm3+BoXYPkM1b9bmJmId/YzsTlHZ9Kqv jp4fT9Zzi1ltUmYw0vCS0+iHcwI8H7paRzH6c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=iLxBt0rx5CeTHYd9jhtacSv8/V8ue/67uPHQnJaKlNAdSD4gjuFNag+A3uGJrhdZfx ZiOSMbNXWhstOov0X/TE7BtfyVm9SxCBswu4z5ZP1k7WuKkJMtsZeKWebL0zXltGokg8 B41gB996f6HlSCFzaCSMp2maM3XjC4sqbyIQI= MIME-Version: 1.0 Received: by 10.224.66.167 with SMTP id n39mr1454247qai.391.1279818842342; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.14.97 with HTTP; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) In-Reply-To: References: Date: Thu, 22 Jul 2010 10:14:02 -0700 X-Google-Sender-Auth: Cr18G1JU5q30KRMsawPraUF8dCs Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: conf call today X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:14:03 -0000 hi all, samir, ganesh, pawel and i are on the conference call the call at the moment. if i see anyone else on skype, i will add them. cheers, On Tue, Jul 20, 2010 at 9:40 AM, Tushar Tambay wrote: > hi all, > > we are supposed to be having our second monthly call today at 10 AM PST but > i am guessing that most people have forgotten that it was today :-) > > apologies for not sending out a reminder mail yesterday. i meant to but did > not get around to it. > > i propose we move the call this week to thursday 10 AM PST. > > please respond if that does that not work for you. > > cheers, > > -- > tyt > > phone: 408 203 9736 (c) > - > > > -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:52:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C914A1065670 for ; Thu, 22 Jul 2010 17:52:34 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 1F4F48FC16 for ; Thu, 22 Jul 2010 17:52:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 121EB45E87; Thu, 22 Jul 2010 19:52:22 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 8E74245E82 for ; Thu, 22 Jul 2010 19:52:16 +0200 (CEST) Date: Thu, 22 Jul 2010 19:51:45 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100722175145.GA2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:52:34 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Samir, could you take a look at this change: http://svn.freebsd.org/changeset/base/206051 I think it should fix ioctl structure size limti for good. I'll investigate if the change can be merged to stable/8. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIhTEACgkQForvXbEpPzRI4wCfQoBXbiwEswb0UdgM5iQ8Qt7c CfgAoIanJ7YnYw8eegFHK7fEgoEWVYx+ =wDdW -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:57:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA1A51065678; Thu, 22 Jul 2010 17:57:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 63AE18FC18; Thu, 22 Jul 2010 17:57:21 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 356DCA59F49; Fri, 23 Jul 2010 01:57:20 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id 2cFm2Mp2BNKv; Fri, 23 Jul 2010 01:57:13 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 7A4D2A59F86; Fri, 23 Jul 2010 01:57:10 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=aCPbuxxQyGtew3W5BnL1WhW8MTi/PI4AFi8SHTy0psoFV6JsGoLc/fJcIg31/XUBV JWrmN8r9MXawbMRmGvxFw== Message-ID: <4C488671.7040908@delphij.net> Date: Thu, 22 Jul 2010 10:57:05 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.11) Gecko/20100721 Thunderbird/3.0.6 ThunderBrowse/3.3 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20100722175145.GA2409@garage.freebsd.pl> In-Reply-To: <20100722175145.GA2409@garage.freebsd.pl> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:57:21 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2010/07/22 10:51, Pawel Jakub Dawidek wrote: > Samir, could you take a look at this change: > > http://svn.freebsd.org/changeset/base/206051 > > I think it should fix ioctl structure size limti for good. > I'll investigate if the change can be merged to stable/8. I think this one is Ok (note that we may want to disable the sign-extension warning for non-verbose or even non-DIAGNOSTIC kernels as well). Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMSIZxAAoJEATO+BI/yjfB/AMH/A3NUkmxSOfHFoRJe51LOlj8 GXRs3emfdRQDs63Q+pJz591Ndhbhyg/0+SA2prSWyorBcZN2oVl9+GEEx+YaWPCE DERmaI8wndw1eyKe0Uj8vvMKUU9CSoz6g+qpLUKt7yU1q3yxWRfI52u11lywYufb umJGIWMdXCWa5DtC+k2DH/8RK1gDC/e4bsu3M5KyN8GEXskVanXwRZ5SVr9vqBRR vx6TZEbgycjzd0/xPeTTqj2YpXyKg3R69Dd240KOyTrOsaieyS4bYLkRZcdMi+NX 0Zonik5+6fVtah3eWBSmWe3cW4W81/G02hZZlGdiuT2Z8T2KQmBAAcKfK5ZQS6Y= =b7We -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 18:10:30 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3358106566C for ; Thu, 22 Jul 2010 18:10:30 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 099D88FC1C for ; Thu, 22 Jul 2010 18:10:29 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9470E45E86; Thu, 22 Jul 2010 20:10:28 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A4C7245C99 for ; Thu, 22 Jul 2010 20:10:23 +0200 (CEST) Date: Thu, 22 Jul 2010 20:09:52 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100722180952.GB2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kXdP64Ggrk/fb43R" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: NetApp/ZFS threats. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 18:10:30 -0000 --kXdP64Ggrk/fb43R Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable There was discussion on zfs-discuss@opensolaris mailing list about this. Not sure if there was anything interesting, but: http://mail.opensolaris.org/pipermail/zfs-discuss/2010-July/042931.html --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kXdP64Ggrk/fb43R Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIiW8ACgkQForvXbEpPzQsAQCgy2mrhrZ3azyA4xjGAQOaJklD p6sAoJAsqEx0SQ/bcNcLl5oyo0v9kXlg =d4qn -----END PGP SIGNATURE----- --kXdP64Ggrk/fb43R-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 18:43:12 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25057106566B for ; Thu, 22 Jul 2010 18:43:12 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id ADCFB8FC16 for ; Thu, 22 Jul 2010 18:43:10 +0000 (UTC) Received: by qwg5 with SMTP id 5so73483qwg.13 for ; Thu, 22 Jul 2010 11:43:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=mWfukmRifgPiMqAnEbp3wMuusqRwE0li7gjlZT2T6go=; b=w1VFfIa4Qzm4v3YgPN103JkBuRb7SJSKiUjjh/iruOIqFLX9gsLttqE3TH5y3aP7Nj 1Eylh1pcGB5YSSjyLItRRwvEL0mob+42S2yZzxsAaMCHlp2kS5cJSWnfheG7MWVdjDHz HuSJqF3xHTSVWiFjjJPZ0veET14blIeHYu4Zs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=ie2MWmRgxL6wOG9Tv+Bd19dgSO8aRrNdC2/cvltbUTQuneyipmbM2ZeqCfnF776Kw9 rL5fBu5iMoCk83rEil+86980nEC+ghyawighfYVsPo5d+WDdz8OsAiocMz0AyX+rKJBh XJ+0jr6BPRIL7rxjhbarRWfPP5rgZVU90+mwo= MIME-Version: 1.0 Received: by 10.224.66.8 with SMTP id l8mr1579470qai.306.1279824168826; Thu, 22 Jul 2010 11:42:48 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.14.97 with HTTP; Thu, 22 Jul 2010 11:42:48 -0700 (PDT) Date: Thu, 22 Jul 2010 11:42:48 -0700 X-Google-Sender-Auth: 27XpPfEbsndETpxbd_Zd3RFitnQ Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs/bsd conf call minutes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 18:43:12 -0000 date: 7/22/10 attendees : samir, ganesh, xin, pawel, tushar - martin updated head to zfs v15 and enabled use py-zfs commands - commands that are now enabled (in head) include :- - zfs-allow, zfs-unallow, zfs-groupspace, zfs-userspace - plan (pawel, martin) - bsd head zfs version will match stable solaris (not opensolaris) zfs version. - on perforce we will keep up with latest & greatest version v26 (soon v27) - re. netapp's receent threatened suit against coraid - (pawel) netapp cannot go after freebsd foundation since the foundation does not own freebsd code (this is unlike the situation that exists in netbsd where the netbsd foundation has legal rights over the code) - perhaps justin can share his thoughts on this issue with the group ? action items - samir / ganesh - investigate usefulness of py-zfs enabled commands on v26. then decide whether to port py-zfs - ganesh - send out details of panic in zpool create (using files as devices) - samir - checkin changes to common tree. -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 21:11:54 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0452F1065672 for ; Thu, 22 Jul 2010 21:11:54 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 4E4218FC15 for ; Thu, 22 Jul 2010 21:11:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 527B71117FE for ; Thu, 22 Jul 2010 23:11:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZQnCppujyZjK for ; Thu, 22 Jul 2010 23:11:45 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 430EA1117E6 for ; Thu, 22 Jul 2010 23:11:45 +0200 (CEST) Message-ID: <4C48B418.4000907@FreeBSD.org> Date: Thu, 22 Jul 2010 23:11:52 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 References: <20100722180952.GB2409@garage.freebsd.pl> In-Reply-To: <20100722180952.GB2409@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.org Subject: Re: NetApp/ZFS threats. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 21:11:54 -0000 Here is a link to the ZFS lawsuit information: http://www.sun.com/lawsuit/zfs/ Under "More Information" you find links to many relevant legal documents. The patent 5,819,292 (dealing with "copy on write") and 5,761,662 ("Personalized retrieval using user-defined profile") have been already rejected by final action of the USPTO (United States Patent and Trademark Office). Personalized retrieval using user-defined profile: http://www.freepatentsonline.com/5761662.html http://www.sun.com/lawsuit/zfs/docs/US5761662FinalOfficeAction3-2009.pdf Rejected claims: 1,4-8,11-17,20-32 (rejected as not patentable, these claims cannot be legally enforced) Copy-on-write: http://www.freepatentsonline.com/5819292.html http://www.sun.com/lawsuit/zfs/docs/US5819292FinalOfficeActionJun2009.pdf Rejected claims: 4,8,11-15,20,23,24 (rejected as not patentable, these claims cannot be legally enforced) Confirmed claims: 1, 21, 22 (these claims can be sued and enforced) Snapshots (7,174,352): http://www.freepatentsonline.com/7174352.html http://www.sun.com/lawsuit/zfs/docs/US7174352OfficeAction2-2009.pdf Clones (6,857,001): http://www.freepatentsonline.com/6857001.html Dòa 22. 7. 2010 20:09, Pawel Jakub Dawidek wrote / napísal(a): > There was discussion on zfs-discuss@opensolaris mailing list about this. > Not sure if there was anything interesting, but: > > http://mail.opensolaris.org/pipermail/zfs-discuss/2010-July/042931.html > From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 22:27:31 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 646B11065672; Thu, 22 Jul 2010 22:27:31 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id E033F8FC18; Thu, 22 Jul 2010 22:27:30 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id EFD62BD11A; Fri, 23 Jul 2010 00:27:29 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id qY9O4iMZlDXP; Fri, 23 Jul 2010 00:27:27 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 7C8F6BD10E; Fri, 23 Jul 2010 00:27:27 +0200 (CEST) Message-ID: <4C48C5D7.7070509@FreeBSD.org> Date: Fri, 23 Jul 2010 00:27:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: multipart/mixed; boundary="------------040408050608060809080606" Cc: Xin LI Subject: ZFS patch proposal: zfs_fuid.c fix X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 22:27:31 -0000 This is a multi-part message in MIME format. --------------040408050608060809080606 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit I have examined the opensolaris SMB code and would like to propose the attached patch. It fixes zfs_fuid.c and acts like OpenSolaris with an unresolved IDMAP user: uses nulldomain (no domain) and unknown SMB user (UID_NOBODY). In addition, I would like to allow users to set/unsed the sharesmb property even if it does nothing. It is good for imported pools that have datasets with this property enabled, so that it can be disabled. It should fix kern/145778 and kern/148709. --------------040408050608060809080606 Content-Type: text/plain; name="head-sharesmb.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="head-sharesmb.patch" Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 210319) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -1265,7 +1265,6 @@ case ZFS_PROP_XATTR: case ZFS_PROP_VSCAN: case ZFS_PROP_NBMAND: - case ZFS_PROP_SHARESMB: (void) snprintf(errbuf, sizeof (errbuf), "property '%s' not supported on FreeBSD", propname); ret = zfs_error(hdl, EZFS_PERM, errbuf); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (revision 210319) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (working copy) @@ -410,7 +410,7 @@ domain = zfs_fuid_find_by_idx(zfsvfs, index); ASSERT(domain != NULL); -#ifdef TODO +#ifdef sun if (type == ZFS_OWNER || type == ZFS_ACE_USER) { (void) kidmap_getuidbysid(crgetzone(cr), domain, FUID_RID(fuid), &id); @@ -418,9 +418,9 @@ (void) kidmap_getgidbysid(crgetzone(cr), domain, FUID_RID(fuid), &id); } -#else - panic(__func__); -#endif +#else /* sun */ + id = UID_NOBODY; +#endif /* sun */ return (id); } @@ -514,21 +514,21 @@ if (!zfsvfs->z_use_fuids || !IS_EPHEMERAL(id)) return ((uint64_t)id); -#ifdef TODO +#ifdef sun ksid = crgetsid(cr, (type == ZFS_OWNER) ? KSID_OWNER : KSID_GROUP); VERIFY(ksid != NULL); rid = ksid_getrid(ksid); domain = ksid_getdomain(ksid); - +#else /* sun */ + rid = UID_NOBODY; + domain = nulldomain; +#endif /* sun */ idx = zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); zfs_fuid_node_add(fuidp, kdomain, rid, idx, id, type); return (FUID_ENCODE(idx, rid)); -#else - panic(__func__); -#endif } /* @@ -597,7 +597,7 @@ }; domain = fuidp->z_domain_table[idx -1]; } else { -#ifdef TODO +#ifdef sun if (type == ZFS_OWNER || type == ZFS_ACE_USER) status = kidmap_getsidbyuid(crgetzone(cr), id, &domain, &rid); @@ -606,6 +606,7 @@ &domain, &rid); if (status != 0) { +#endif /* sun */ /* * When returning nobody we will need to * make a dummy fuid table entry for logging @@ -613,10 +614,9 @@ */ rid = UID_NOBODY; domain = nulldomain; +#ifdef sun } -#else - panic(__func__); -#endif +#endif /* sun */ } idx = zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); --------------040408050608060809080606-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 22:47:31 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F26E1065670; Thu, 22 Jul 2010 22:47:31 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6D00F8FC12; Thu, 22 Jul 2010 22:47:19 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8AA1245C9B; Fri, 23 Jul 2010 00:47:17 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 204D345C8C; Fri, 23 Jul 2010 00:47:12 +0200 (CEST) Date: Fri, 23 Jul 2010 00:46:41 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100722224641.GE2409@garage.freebsd.pl> References: <4C48C5D7.7070509@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="udcq9yAoWb9A4FsZ" Content-Disposition: inline In-Reply-To: <4C48C5D7.7070509@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS patch proposal: zfs_fuid.c fix X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 22:47:31 -0000 --udcq9yAoWb9A4FsZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 12:27:35AM +0200, Martin Matuska wrote: > I have examined the opensolaris SMB code and would like to propose the > attached patch. It fixes zfs_fuid.c and acts like OpenSolaris with an > unresolved IDMAP user: uses nulldomain (no domain) and unknown SMB user > (UID_NOBODY). >=20 > In addition, I would like to allow users to set/unsed the sharesmb > property even if it does nothing. It is good for imported pools that > have datasets with this property enabled, so that it can be disabled. >=20 > It should fix kern/145778 and kern/148709. The patch looks good to me. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 210319) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -1265,7 +1265,6 @@ > case ZFS_PROP_XATTR: > case ZFS_PROP_VSCAN: > case ZFS_PROP_NBMAND: > - case ZFS_PROP_SHARESMB: > (void) snprintf(errbuf, sizeof (errbuf), > "property '%s' not supported on FreeBSD", propname); > ret =3D zfs_error(hdl, EZFS_PERM, errbuf); > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (revision 2= 10319) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (working co= py) > @@ -410,7 +410,7 @@ > domain =3D zfs_fuid_find_by_idx(zfsvfs, index); > ASSERT(domain !=3D NULL); > =20 > -#ifdef TODO > +#ifdef sun > if (type =3D=3D ZFS_OWNER || type =3D=3D ZFS_ACE_USER) { > (void) kidmap_getuidbysid(crgetzone(cr), domain, > FUID_RID(fuid), &id); > @@ -418,9 +418,9 @@ > (void) kidmap_getgidbysid(crgetzone(cr), domain, > FUID_RID(fuid), &id); > } > -#else > - panic(__func__); > -#endif > +#else /* sun */ > + id =3D UID_NOBODY; > +#endif /* sun */ > return (id); > } > =20 > @@ -514,21 +514,21 @@ > if (!zfsvfs->z_use_fuids || !IS_EPHEMERAL(id)) > return ((uint64_t)id); > =20 > -#ifdef TODO > +#ifdef sun > ksid =3D crgetsid(cr, (type =3D=3D ZFS_OWNER) ? KSID_OWNER : KSID_GROUP= ); > =20 > VERIFY(ksid !=3D NULL); > rid =3D ksid_getrid(ksid); > domain =3D ksid_getdomain(ksid); > - > +#else /* sun */ > + rid =3D UID_NOBODY; > + domain =3D nulldomain; > +#endif /* sun */ > idx =3D zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); > =20 > zfs_fuid_node_add(fuidp, kdomain, rid, idx, id, type); > =20 > return (FUID_ENCODE(idx, rid)); > -#else > - panic(__func__); > -#endif > } > =20 > /* > @@ -597,7 +597,7 @@ > }; > domain =3D fuidp->z_domain_table[idx -1]; > } else { > -#ifdef TODO > +#ifdef sun > if (type =3D=3D ZFS_OWNER || type =3D=3D ZFS_ACE_USER) > status =3D kidmap_getsidbyuid(crgetzone(cr), id, > &domain, &rid); > @@ -606,6 +606,7 @@ > &domain, &rid); > =20 > if (status !=3D 0) { > +#endif /* sun */ > /* > * When returning nobody we will need to > * make a dummy fuid table entry for logging > @@ -613,10 +614,9 @@ > */ > rid =3D UID_NOBODY; > domain =3D nulldomain; > +#ifdef sun > } > -#else > - panic(__func__); > -#endif > +#endif /* sun */ > } > =20 > idx =3D zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --udcq9yAoWb9A4FsZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIylAACgkQForvXbEpPzTVzgCg2vhW98aOD+ahLlpOB/b4AhYK fl8AoIJ8YmvBZ8OeWMBYXmn1b7LaIrTJ =Ghn6 -----END PGP SIGNATURE----- --udcq9yAoWb9A4FsZ-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 23:13:27 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 967BC1065678; Thu, 22 Jul 2010 23:13:27 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 51A638FC1D; Thu, 22 Jul 2010 23:13:26 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 37CF6113834; Fri, 23 Jul 2010 01:13:26 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Dbq6dK4peSQy; Fri, 23 Jul 2010 01:13:24 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 0AA0F11382B; Fri, 23 Jul 2010 01:13:24 +0200 (CEST) Message-ID: <4C48D09C.8040704@FreeBSD.org> Date: Fri, 23 Jul 2010 01:13:32 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI Subject: More bugfixes from OpenSolaris X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 23:13:27 -0000 Hi, I have the following changesets from OpenSolaris upstream that I would like to propose to import to head and after v15 merge to stable/8. The code resulting from these bugfixes has not been changed in OpenSolaris by today (matches pjd p4 tree). 1.) revision 9788:f660bc44f2e8 (possible memory corruption on zfs umount) zfs_znode_move() does not ensure valid file system pointer http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6843700 http://mfsbsd.vx.sk/zfs/head-9788.patch 2.) revision 9909:aa280f585a3e (locking bugfix - affects zfs rollback) zfs related assertion failure: vp->v_filocks == 0L, file: ../../common/fs/vnode.c, line: 2344 http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6790232 http://mfsbsd.vx.sk/zfs/head-9909.patch 3.) revision 9847:2f3ba86e857a (feature) snapshots should be considered as descendants via zfs allow -d http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6809340 http://mfsbsd.vx.sk/zfs/head-9847.patch Patches are direct OpenSolaris imports and apply cleanly. From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 07:34:57 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A25B31065678; Fri, 23 Jul 2010 07:34:57 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 5C9748FC22; Fri, 23 Jul 2010 07:34:56 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 28BD211236B; Fri, 23 Jul 2010 09:34:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 2GapBKHA88r6; Fri, 23 Jul 2010 09:34:53 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 580AF1122F7; Fri, 23 Jul 2010 09:34:53 +0200 (CEST) Message-ID: <4C494624.5000208@FreeBSD.org> Date: Fri, 23 Jul 2010 09:35:00 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 07:34:57 -0000 Hi Pawel and Xin, I have a user report and I can confirm myself of ZFS v15 acting slower than before the import on several systems. I would therefore like to propose the combined stat() and rrwlock speedup patch as this has noticeable impact on this issue in my tests and the overall impression of system responsiveness. http://mfsbsd.vx.sk/zfs/head-9981-10143-10232-10250-10269.patch The patch code remained in OpenSolaris like that until today and it was part of the original zfs v16 patch I posted so it has already been tested by the users. A small part of the 10143 patch is not imported, see notes below. The patch covers the following opensolaris revisions and bug IDs: 9981:b4907297e740 ----------------- stat() performance on files on zfs should be improved http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6775100 rrwlock is overly protective of its counters http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6827779 10143:d2d432dfe597 (small part not included, see below) ----------------- memory leaks found at: zfs_acl_alloc/zfs_acl_node_alloc http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6857433 truncate() on zfsroot succeeds when file has a component of its path set without access permission http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6860318 10232:f37b85f7e03e ----------------- zfs sometimes incorrectly giving search access to a dir http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6865875 10250:b179ceb34b62 ----------------- zpool_upgrade_007_pos testcase panic'd with BAD TRAP: type=e (#pf Page fault) http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6867395 10269:2788675568fd ----------------- zfs_rezget() can be hazardous when znode has a cached ACL http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6868276 ---------------------------------- If the import of a small part of 10143 (http://mfsbsd.vx.sk/zfs/head-10143-addon.patch) is possible only after importing ABE: http://mfsbsd.vx.sk/zfs/head-9749-9866.patch 9749:105f407a2680 ----------------- Support for Access Based Enumeration http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6802734 A test driver for some ZFS features would be beneficial (this is the yet unused zlook program) http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6826760 inconsistent xattr readdir behavior with too-small buffer http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6844861 9866:ddc5f1d8eb4e ----------------- zfs with rstchown=0 or file_chown_self privilege allows user to "take" ownership http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6848431 From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 09:18:26 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9266C1065677; Fri, 23 Jul 2010 09:18:26 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2B6218FC15; Fri, 23 Jul 2010 09:18:18 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3FAC645CDC; Fri, 23 Jul 2010 11:18:17 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 527AD45C89; Fri, 23 Jul 2010 11:18:12 +0200 (CEST) Date: Fri, 23 Jul 2010 11:17:42 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100723091742.GB1756@garage.freebsd.pl> References: <4C494624.5000208@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Y7xTucakfITjPcLV" Content-Disposition: inline In-Reply-To: <4C494624.5000208@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 09:18:26 -0000 --Y7xTucakfITjPcLV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 09:35:00AM +0200, Martin Matuska wrote: > Hi Pawel and Xin, >=20 > I have a user report and I can confirm myself of ZFS v15 acting slower > than before the import on several systems. > I would therefore like to propose the combined stat() and rrwlock > speedup patch as this has noticeable impact on this issue in my tests > and the overall impression of system responsiveness. Of course speed up is very welcome, but as I read it v15 is slower than v14 for some unknown reason. So of course getting more optimizations is desirable, but not really related to why v15 is slower than v14. Is my understanding correct? What kind of performance problems do you observe? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Y7xTucakfITjPcLV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxJXjUACgkQForvXbEpPzRezACgqdX6YdrTfKG5Fh3v4UqUsNem fl8AoJES/M5vxF2viDxzf7/TXlfc7EaJ =hOwW -----END PGP SIGNATURE----- --Y7xTucakfITjPcLV-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 10:47:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08431106564A; Fri, 23 Jul 2010 10:47:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id B72AB8FC14; Fri, 23 Jul 2010 10:47:20 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 54987114A71; Fri, 23 Jul 2010 12:47:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 8g7wKn-0S20g; Fri, 23 Jul 2010 12:47:17 +0200 (CEST) Received: from [10.0.3.171] (188-167-71-217.dynamic.chello.sk [188.167.71.217]) by mail.vx.sk (Postfix) with ESMTPSA id 563F6114A66; Fri, 23 Jul 2010 12:47:17 +0200 (CEST) Message-ID: <4C497331.5040304@FreeBSD.org> Date: Fri, 23 Jul 2010 12:47:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> In-Reply-To: <20100723091742.GB1756@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 10:47:21 -0000 To my experience, processes spend more time in zio (as of top). When there is a lot of FS activity the system responsiveness goes down (filesystem reacts slowly or gets stallish). A good example is my tinderbox system - when tinderbox does a lot of small ports (heavy ZFS activity, untarring of many files and compiling, searching for changed files, etc.) the system shows a stallish behaviour in disk reads and my shell reacts slowly. With the stat and rrwlock patch it responds slightly better as it did before v15. The question is, why. The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in a ZFS mirror. vm.kmem_size="6G" vfs.zfs.arc_max="4G" Anyway, I would vote for these patches, maybe including ABE (it doesn't hurt). Dòa 23. 7. 2010 11:17, Pawel Jakub Dawidek wrote / napísal(a): > On Fri, Jul 23, 2010 at 09:35:00AM +0200, Martin Matuska wrote: >> Hi Pawel and Xin, >> >> I have a user report and I can confirm myself of ZFS v15 acting slower >> than before the import on several systems. >> I would therefore like to propose the combined stat() and rrwlock >> speedup patch as this has noticeable impact on this issue in my tests >> and the overall impression of system responsiveness. > Of course speed up is very welcome, but as I read it v15 is slower than > v14 for some unknown reason. So of course getting more optimizations is > desirable, but not really related to why v15 is slower than v14. > Is my understanding correct? What kind of performance problems do you > observe? > From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 14:01:18 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89B421065674; Fri, 23 Jul 2010 14:01:18 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id CC3B58FC16; Fri, 23 Jul 2010 14:01:17 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 042B245E87; Fri, 23 Jul 2010 16:01:16 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E64BF45C9F; Fri, 23 Jul 2010 16:01:10 +0200 (CEST) Date: Fri, 23 Jul 2010 16:00:40 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100723140040.GD1756@garage.freebsd.pl> References: <4C48D09C.8040704@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+JUInw4efm7IfTNU" Content-Disposition: inline In-Reply-To: <4C48D09C.8040704@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: More bugfixes from OpenSolaris X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 14:01:18 -0000 --+JUInw4efm7IfTNU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 01:13:32AM +0200, Martin Matuska wrote: > Hi, I have the following changesets from OpenSolaris upstream that I > would like to propose to import to head and after v15 merge to stable/8. > The code resulting from these bugfixes has not been changed in > OpenSolaris by today (matches pjd p4 tree). >=20 > 1.) revision 9788:f660bc44f2e8 (possible memory corruption on zfs umount) > zfs_znode_move() does not ensure valid file system pointer > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6843700 > http://mfsbsd.vx.sk/zfs/head-9788.patch I believe that for us this is no-op, because zfs_znode_move() is unused and commented out. > 2.) revision 9909:aa280f585a3e (locking bugfix - affects zfs rollback) > zfs related assertion failure: vp->v_filocks =3D=3D 0L, file: > ../../common/fs/vnode.c, line: 2344 > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6790232 > http://mfsbsd.vx.sk/zfs/head-9909.patch This is also no-op, as we define both cleanlocks() and cleanshares() as 'do { } while (0)'. > 3.) revision 9847:2f3ba86e857a (feature) > snapshots should be considered as descendants via zfs allow -d > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6809340 > http://mfsbsd.vx.sk/zfs/head-9847.patch This looks fine. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+JUInw4efm7IfTNU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxJoIcACgkQForvXbEpPzQASgCglUNPo4nP3S5LC0RkBvz9IjuV it8An2JbSGC2IHIN0oIvks2hC2UFVY16 =PIqT -----END PGP SIGNATURE----- --+JUInw4efm7IfTNU-- From owner-zfs-devel@FreeBSD.ORG Sat Jul 24 21:11:45 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56E8C1065670; Sat, 24 Jul 2010 21:11:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id AD2DC8FC16; Sat, 24 Jul 2010 21:11:44 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2FBD545E86; Sat, 24 Jul 2010 23:11:42 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0F28845685; Sat, 24 Jul 2010 23:11:36 +0200 (CEST) Date: Sat, 24 Jul 2010 23:11:05 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100724211105.GC2123@garage.freebsd.pl> References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> <4C497331.5040304@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kfjH4zxOES6UT95V" Content-Disposition: inline In-Reply-To: <4C497331.5040304@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jul 2010 21:11:45 -0000 --kfjH4zxOES6UT95V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 12:47:13PM +0200, Martin Matuska wrote: > To my experience, processes spend more time in zio (as of top). >=20 > When there is a lot of FS activity the system responsiveness goes down > (filesystem reacts slowly or gets stallish). > A good example is my tinderbox system - when tinderbox does a lot of > small ports (heavy ZFS activity, untarring of many files and compiling, > searching for changed files, etc.) the system shows a stallish behaviour > in disk reads and my shell reacts slowly. [...] This looks like priority problem. What kernel threads consume most CPU time when if feel slowdown? Do you have compression enabled on loaded datasets? I'd try lowering priority of the guilty thread and see what happen. > [...] With the stat and rrwlock > patch it responds slightly better as it did before v15. The question is, > why. >=20 > The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in > a ZFS mirror. > vm.kmem_size=3D"6G" > vfs.zfs.arc_max=3D"4G" >=20 > Anyway, I would vote for these patches, maybe including ABE (it doesn't > hurt). I'd prefer to address performance problem with v15 first and then merge stat(2) speed up changes, especially that with stat(2) patch it is harder to reproduce. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kfjH4zxOES6UT95V Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxLVugACgkQForvXbEpPzQ8wQCfdDTHW8WD+uqFexkvilWJfD8r WooAnRvHiojoMMycowrRI3xsNnfT8qJO =BK3M -----END PGP SIGNATURE----- --kfjH4zxOES6UT95V-- From owner-zfs-devel@FreeBSD.ORG Sun Jul 25 18:31:43 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBAEF106566B; Sun, 25 Jul 2010 18:31:43 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id D31B88FC0A; Sun, 25 Jul 2010 18:31:43 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AD33C31136; Sun, 25 Jul 2010 20:31:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 05gRRWIs616h; Sun, 25 Jul 2010 20:31:40 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 3AF083112D; Sun, 25 Jul 2010 20:31:40 +0200 (CEST) Message-ID: <4C4C830D.7040101@FreeBSD.org> Date: Sun, 25 Jul 2010 20:31:41 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> <4C497331.5040304@FreeBSD.org> <20100724211105.GC2123@garage.freebsd.pl> In-Reply-To: <20100724211105.GC2123@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jul 2010 18:31:44 -0000 Ok, after running for a longer time, I can confirm that the stat() speedup does not make the problem go away. It is priority-related. If I have the problematic tasks running with a nice >0, the system responds quickly. It looks like more that maybe rc.subr was changed in a way that my tinderbox daemon does not start with a lower priority anymore, so the behaviour will be probably the same on pre-v15 and this makes this issue probably not ZFS-v15-related or not even ZFS related at all. The system slowness is overall, not only ZFS - also reaction of the shell, too. Dòa 24. 7. 2010 23:11, Pawel Jakub Dawidek wrote / napísal(a): > On Fri, Jul 23, 2010 at 12:47:13PM +0200, Martin Matuska wrote: >> To my experience, processes spend more time in zio (as of top). >> >> When there is a lot of FS activity the system responsiveness goes down >> (filesystem reacts slowly or gets stallish). >> A good example is my tinderbox system - when tinderbox does a lot of >> small ports (heavy ZFS activity, untarring of many files and compiling, >> searching for changed files, etc.) the system shows a stallish behaviour >> in disk reads and my shell reacts slowly. [...] > This looks like priority problem. What kernel threads consume most CPU > time when if feel slowdown? Do you have compression enabled on loaded > datasets? I'd try lowering priority of the guilty thread and see what > happen. > >> [...] With the stat and rrwlock >> patch it responds slightly better as it did before v15. The question is, >> why. >> >> The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in >> a ZFS mirror. >> vm.kmem_size="6G" >> vfs.zfs.arc_max="4G" >> >> Anyway, I would vote for these patches, maybe including ABE (it doesn't >> hurt). > I'd prefer to address performance problem with v15 first and then merge > stat(2) speed up changes, especially that with stat(2) patch it is > harder to reproduce. > From owner-zfs-devel@FreeBSD.ORG Mon Jul 26 07:55:37 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3F23106566B for ; Mon, 26 Jul 2010 07:55:37 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6279E8FC1D for ; Mon, 26 Jul 2010 07:55:37 +0000 (UTC) Received: by vws7 with SMTP id 7so2635712vws.13 for ; Mon, 26 Jul 2010 00:55:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=OU4OgEFnqx7kidB5RGHN6ZyFPTY32XAoxtd7tTOIvKk=; b=lbBD35p1I47ye2RgjuQug/DYthZEW4z0N35UEmiGBAOyxPbF/lzByq/mAfTqQklpfk VJxqLfzKEttomsj1+ZNadlnbdoIg1C7MfaNfipzandXrCmqOToAcUXfVIsic/kF2dv/R VaUx4YvBvtmnR0hkr+MCrZvUQvSqAcT5tsBZs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=LNojSOayKn8tbqfqnbhdD4r3CPcyjCTRvZF8zEfi1PqtHoXSkeAM4fAudNaG6jVS07 Uojwrphdsohlx1CoGb5Zl7w5nLN7cVF11oRVNAzjdAvNw63J+lzYt8GV5Js1Kpc129iN eopsno2FILujCyRT0z0tJysuIToNfV3IRkmr0= MIME-Version: 1.0 Received: by 10.220.49.28 with SMTP id t28mr3902099vcf.93.1280129145272; Mon, 26 Jul 2010 00:25:45 -0700 (PDT) Received: by 10.220.185.201 with HTTP; Mon, 26 Jul 2010 00:25:45 -0700 (PDT) In-Reply-To: <20100722175145.GA2409@garage.freebsd.pl> References: <20100722175145.GA2409@garage.freebsd.pl> Date: Mon, 26 Jul 2010 12:55:45 +0530 Message-ID: From: Samir Desai To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jul 2010 07:55:37 -0000 hi Pawel Shouldn't the MAX ioctl size be defined to #define IOCPARM_MAX ((1 << IOCPARM_SHIFT) - 1 ) Strictly speaking size 8192 can not be encoded. regards Samir On Thu, Jul 22, 2010 at 11:21 PM, Pawel Jakub Dawidek wrote: > Samir, could you take a look at this change: > > http://svn.freebsd.org/changeset/base/206051 > > I think it should fix ioctl structure size limti for good. > I'll investigate if the change can be merged to stable/8. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > From owner-zfs-devel@FreeBSD.ORG Wed Jul 28 17:14:22 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61C861065686 for ; Wed, 28 Jul 2010 17:14:22 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B35758FC1A for ; Wed, 28 Jul 2010 17:14:21 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA08430 for ; Wed, 28 Jul 2010 20:02:32 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C5062A7.5030200@icyb.net.ua> Date: Wed, 28 Jul 2010 20:02:31 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: zfs-devel@freebsd.org X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: zfs threads X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 17:14:22 -0000 Not sure if this is something that happened recently or if it was always that way - I see a lot of zfs-something-named kernel threads running under 'kernel' process (pid 0). I thought the idea was to group them under that special kernel process 'zfskern'. Example: $ procstat -t 0 | fgrep zio 0 100079 kernel spa_zio_null_iss 0 68 sleep - 0 100080 kernel spa_zio_null_int 0 68 sleep - 0 100081 kernel spa_zio_read_iss 1 68 sleep - 0 100082 kernel spa_zio_read_iss 1 68 sleep - 0 100083 kernel spa_zio_read_iss 1 68 sleep - 0 100084 kernel spa_zio_read_iss 0 68 sleep - 0 100085 kernel spa_zio_read_iss 0 68 sleep - 0 100086 kernel spa_zio_read_iss 1 68 sleep - 0 100087 kernel spa_zio_read_iss 0 68 sleep - 0 100088 kernel spa_zio_read_iss 1 68 sleep - 0 100089 kernel spa_zio_read_int 0 68 sleep - 0 100090 kernel spa_zio_write_is 1 68 sleep - 0 100091 kernel spa_zio_write_in 0 68 sleep - 0 100092 kernel spa_zio_write_in 1 68 sleep - 0 100093 kernel spa_zio_write_in 1 68 sleep - 0 100094 kernel spa_zio_write_in 0 68 sleep - 0 100095 kernel spa_zio_write_in 1 68 sleep - 0 100096 kernel spa_zio_write_in 0 68 sleep - 0 100097 kernel spa_zio_write_in 0 68 sleep - 0 100098 kernel spa_zio_write_in 0 68 sleep - 0 100099 kernel spa_zio_free_iss 1 68 sleep - 0 100100 kernel spa_zio_free_int 1 68 sleep - 0 100101 kernel spa_zio_claim_is 1 68 sleep - 0 100102 kernel spa_zio_claim_in 1 68 sleep - 0 100103 kernel spa_zio_ioctl_is 1 68 sleep - 0 100104 kernel spa_zio_ioctl_in 1 68 sleep - -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Jul 28 21:00:39 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6762C1065673 for ; Wed, 28 Jul 2010 21:00:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 00C6E8FC14 for ; Wed, 28 Jul 2010 21:00:38 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id ACD2145C98; Wed, 28 Jul 2010 23:00:36 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B547345C89; Wed, 28 Jul 2010 23:00:31 +0200 (CEST) Date: Wed, 28 Jul 2010 23:00:29 +0200 From: Pawel Jakub Dawidek To: Samir Desai Message-ID: <20100728210029.GA2390@garage.freebsd.pl> References: <20100722175145.GA2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ReaqsoxgOBHFXBhH" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 21:00:39 -0000 --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 26, 2010 at 12:55:45PM +0530, Samir Desai wrote: > hi Pawel >=20 > Shouldn't the MAX ioctl size be defined to >=20 > #define IOCPARM_MAX ((1 << IOCPARM_SHIFT) - 1 ) >=20 > Strictly speaking size 8192 can not be encoded. You are right. Actually, now that we can use all the bits, this check in sys/kern/sys_generic.c:ioctl() is pointless: size =3D IOCPARM_LEN(com); if ((size > IOCPARM_MAX) || [...] I think I'll just remove IOCPARM_MAX entirely. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ReaqsoxgOBHFXBhH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxQmmwACgkQForvXbEpPzTrvACg1n2O22iXM8Nm26Cm7yEkTTFx jWAAnRb2zwDj1P+o85+a/ovEF4jTZ0Zk =34n4 -----END PGP SIGNATURE----- --ReaqsoxgOBHFXBhH-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 10 20:04:10 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE3EE1065670 for ; Tue, 10 Aug 2010 20:04:10 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id D3BA98FC08 for ; Tue, 10 Aug 2010 20:04:09 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 79A4545C98; Tue, 10 Aug 2010 22:04:07 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 64EAE4569A for ; Tue, 10 Aug 2010 22:04:00 +0200 (CEST) Date: Tue, 10 Aug 2010 22:03:47 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100810200347.GB1766@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OwLcNYc0lM97+oe1" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Updates on ZFS. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 20:04:10 -0000 --OwLcNYc0lM97+oe1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. For the past weeks I was working on getting recent ZFS to work. This is what I accomplished: 1. Fixed (implemented) onexit functionality that is now used by zfs send/recv. This is based on device cloning in OpenSolaris, but I manage to avoid using device cloning on FreeBSD and ended up using cdevpriv, which is much less problematic. This was the main reason ZFS in my tree didn't compile for quite a while - I needed to find a way and time to implement it properly. 2. I manage to get ZVOLs to work again. Commit log for this change should explain everything: OpenSolaris switched to lazy creation of /dev/ entires for ZVOLs. It creates /dev/ entries on VOP_LOOKUP() or VOP_READDIR(). This of course can't work this way on FreeBSD with GEOM, so we need to create ZVOL providers where appropriate. I found the following cases: 1. Pool first open (pool is loaded based on zpool/cache configuration and is then opened for a first time on eg. zfs mount). 2. Pool import. It's not the same as 1. 3. ZVOL creation: zfs create -V . 4. Creation of ZVOL snapshot, this includes recursive snapshot creation. To make it work I had to fix LOR between the zfsdev_state_lock, the GEOM topology lock and the spa_namespace_lock. They are now always obtained in the following order: 1. zfsdev_state_lock 2. g_topology_lock 3. spa_namespace_lock Also, we can't use taskqueue to scan for VDEVs as this introduces deadlock (because there is no way to honour the order above). This also allows to simplify vdev_geom.c quite a bit as it is no longer a problem to taste ZVOL or ZVOL-based provider. Update /etc/rc.d/zvol as there are no longer volinit and volfini subcommands to zfs(8). 3. I integrated all the recent changes (including zfs diff and readonly pools functionality). We are at v28 now and in sync with today's OpenSolaris. 4. I fixed a handful of bugs. All in all, it compiles and at least basic functionality should work. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --OwLcNYc0lM97+oe1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxhsKMACgkQForvXbEpPzQIdACg3AiixNILSMND+/XEdHosX/KS gK0AoN11LX1t7RE681OlmWgqYwRd5qLU =yCTx -----END PGP SIGNATURE----- --OwLcNYc0lM97+oe1-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 10 20:31:00 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B6B3106564A for ; Tue, 10 Aug 2010 20:31:00 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id E29FD8FC1F for ; Tue, 10 Aug 2010 20:30:59 +0000 (UTC) Received: by wwb13 with SMTP id 13so1807201wwb.31 for ; Tue, 10 Aug 2010 13:30:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=NJxjxbGrrvRprmQFDvHO6ldOg9tXJ8ew8KKvkyyxJ7w=; b=EG/oi0Uh9WSrbgblkgRKG5g3gTLzGCx2OdS6BVcagbNwY66Wm2nlPMV9dd9NH/sc1M SEKd1MwAZRFb0canO30Hm9GZbzph/ZT+di/17RQ/yzMKT7vLHbV4X8vdHz0hnnFyY8Pu l9QqGve2nELe9HVPXvQzwfMtdJTz1UHcYN5cc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=SONkdCOLmSkvOTP9p/IN5t22ctGtFNa7akOh30ks00LnC82YBGdxVvOtERxvBzv8bV gsaifB7La4SfE/KTci6scmn1hwxYBBZW6X751Hvir/lrGHBIcScw04OOtNlwcYzNuaov n5PjZay5bHEbD0Kfhi6ckoISklABQqRvBxJmU= MIME-Version: 1.0 Received: by 10.227.145.145 with SMTP id d17mr15773419wbv.148.1281470733163; Tue, 10 Aug 2010 13:05:33 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.227.136.21 with HTTP; Tue, 10 Aug 2010 13:05:33 -0700 (PDT) In-Reply-To: <20100810200347.GB1766@garage.freebsd.pl> References: <20100810200347.GB1766@garage.freebsd.pl> Date: Tue, 10 Aug 2010 13:05:33 -0700 X-Google-Sender-Auth: GwcQFxPFdRX0I6EHYUzr7qgTujE Message-ID: From: Matt Olander To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: zfs-devel@freebsd.org Subject: Re: Updates on ZFS. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 20:31:00 -0000 Wow, that's impressive, Pawel. Nice work! We'll get to work testing :D -matt On Tue, Aug 10, 2010 at 1:03 PM, Pawel Jakub Dawidek wrot= e: > Hi. > > For the past weeks I was working on getting recent ZFS to work. This is > what I accomplished: > > 1. Fixed (implemented) onexit functionality that is now used by zfs > =A0 send/recv. This is based on device cloning in OpenSolaris, but I > =A0 manage to avoid using device cloning on FreeBSD and ended up using > =A0 cdevpriv, which is much less problematic. > =A0 This was the main reason ZFS in my tree didn't compile for quite a > =A0 while - I needed to find a way and time to implement it properly. > > 2. I manage to get ZVOLs to work again. Commit log for this change > =A0 should explain everything: > > =A0 OpenSolaris switched to lazy creation of /dev/ entires for ZVOLs. > =A0 It creates /dev/ entries on VOP_LOOKUP() or VOP_READDIR(). > > =A0 This of course can't work this way on FreeBSD with GEOM, so we need > =A0 to create ZVOL providers where appropriate. I found the following > =A0 cases: > =A0 1. Pool first open (pool is loaded based on zpool/cache configuration > =A0 =A0 =A0and is then opened for a first time on eg. zfs mount). > =A0 2. Pool import. It's not the same as 1. > =A0 3. ZVOL creation: zfs create -V . > =A0 4. Creation of ZVOL snapshot, this includes recursive snapshot > =A0 =A0 =A0creation. > > =A0 To make it work I had to fix LOR between the zfsdev_state_lock, the > =A0 GEOM topology lock and the spa_namespace_lock. They are now always > =A0 obtained in the following order: > =A0 1. zfsdev_state_lock > =A0 2. g_topology_lock > =A0 3. spa_namespace_lock > =A0 Also, we can't use taskqueue to scan for VDEVs as this introduces > =A0 deadlock (because there is no way to honour the order above). > > =A0 This also allows to simplify vdev_geom.c quite a bit as it is no > =A0 longer a problem to taste ZVOL or ZVOL-based provider. > > =A0 Update /etc/rc.d/zvol as there are no longer volinit and volfini > =A0 subcommands to zfs(8). > > 3. I integrated all the recent changes (including zfs diff and readonly > =A0 pools functionality). We are at v28 now and in sync with today's > =A0 OpenSolaris. > > 4. I fixed a handful of bugs. > > All in all, it compiles and at least basic functionality should work. > > -- > Pawel Jakub Dawidek =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://ww= w.wheelsystems.com > pjd@FreeBSD.org =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http:= //www.FreeBSD.org > FreeBSD committer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Am I Ev= il? Yes, I Am! > From owner-zfs-devel@FreeBSD.ORG Sat Aug 21 14:11:01 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 226C3106567A; Sat, 21 Aug 2010 14:11:01 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [IPv6:2a01:4f8:100:1043::2]) by mx1.freebsd.org (Postfix) with ESMTP id B1B4C8FC0A; Sat, 21 Aug 2010 14:11:00 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5CB09D6E2; Sat, 21 Aug 2010 16:10:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 19QOMQcmJhi8; Sat, 21 Aug 2010 16:10:57 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id BBC5D9D6D7; Sat, 21 Aug 2010 16:10:57 +0200 (CEST) Message-ID: <4C6FDE77.5070607@FreeBSD.org> Date: Sat, 21 Aug 2010 16:11:03 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Patch proposal: Speeding up ZFS writes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Aug 2010 14:11:01 -0000 Hi Pawel, Xin and members of zfs-devel@ ! Many of our users today are complaining about slow ZFS writes. One of the causes for these writes is the allocation method for new blocks used [1]. Solaris 10 and OpenSolaris up to November 2009 used the following scenario: - pool has more than 30% free space: use first fit method [2] - pool has less than 70% free space: use best fit method [2] This causes a major slowdown and, let's say, unpleasant "fragmentation" of the writes if we go below 30% of free space. OpenSolaris has changed this and the Oracle Storage Appliances also included the new code in Q1/2010 [2]. The source [2] states, that with this change they archieved a speedup of: "50% Improved OLTP Performance, 70% Reduced Variability, 200% Improvement on MS Exchange" So what can we do to improve this situation? a) on the very short term, as a workaround soulution [3], we could just make metaslab_df_free_pct from metaslab.c tunable so users can test lower settings (I personally tend more to b)) b) on the mid-term, I suggest this patch for head with MFC to stable/8 after some reasonable time (1-2 months): http://people.freebsd.org/~mm/patches/zfs/zfs_metaslab.patch c) on the long term, updating to current ZFS code (v28) that integrates this patch The patch in b) includes the following OpenSolaris onnv revisions: 10921 (very small part, metaslab.c) 11146 (main patch, applies almost cleanly) 11728 (fix for zdb.c) 12047 (improvement to metaslab.c) OpenSolaris Bug IDs: 6826241 Sync write IOPS drops dramatically during TXG sync 6869229 zfs should switch to shiny new metaslabs more frequently 6917066 zfs block picking can be improved 6918420 zdb -m has issues printing metaslab statistics References: [1] http://blogs.sun.com/bonwick/entry/zfs_block_allocation [2] http://sun.systemnews.com/articles/147/2/OpenStorage/22963 [3] http://blogs.everycity.co.uk/alasdair/2010/07/zfs-runs-really-slowly-when-free-disk-usage-goes-above-80/ From owner-zfs-devel@FreeBSD.ORG Sun Aug 22 22:05:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49D001065670 for ; Sun, 22 Aug 2010 22:05:40 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A480D8FC13 for ; Sun, 22 Aug 2010 22:05:39 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA05762; Mon, 23 Aug 2010 00:46:35 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnIN9-000B06-02; Mon, 23 Aug 2010 00:46:35 +0300 Message-ID: <4C719AB9.9020006@freebsd.org> Date: Mon, 23 Aug 2010 00:46:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: zfs-devel@freebsd.org, freebsd-hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 22:05:40 -0000 I propose that the following code in arc_reclaim_needed (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) /* * If pages are needed or we're within 2048 pages * of needing to page need to reclaim */ if (vm_pages_needed || (vm_paging_target() > -2048)) be changed to if (vm_paging_needed()) Rationale. 1. Why not current checks. ARC sizing should cooperate with pagedaemon in freeing pages. If ARC starts shrinking "prematurely", before pagedaemon is waked up then no potentially eligible inactive pages will be recycled and no potentially eligible active pages will be inactive (subject to v_inactive_target). This would lead to ARC size going to its minimum value (which could hurt ZFS performance). Only after that there is a chance that pagedaemon would be waked up to do its cleaning. And conversely, if ARC doesn't shrink in time, then pagedaemon would have to recycle pages with data that could be needed again soon and that would lead to excessive swapping and disk I/O. vm_paging_target() is used only by pagedaemon internally, it effectively sets _upper_ limit on how many pages pagedaemon would free when it's activated. It is no indication of whether pagedaemon should be scanning/freeing pages. Thus check of vm_paging_target() leads to premature ARC shrinking. I believe that many people observe this behavior on sufficiently active systems (not dedicated file servers) with few GB of RAM (1-8). vm_pages_needed check is redundant, because this is a flag that is used to wake up pagedaemon. So when it is set vm_paging_needed() is true and vm_paging_target() is "way" above zero. And this flag is reset to zero when vm_page_count_min() becomes false, which corresponds to even fewer free pages than when vm_paging_needed() is true. 2. Why the new check. vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (see vm_page_alloc). pagedaemon would first of all run vm_lowmem event for which ARC already has a handler and which would cause ARC size to shrink. It would seems like having vm_paging_needed() check would be redundant then. Almost - if memory pressure is significant, then vm_paging_needed() may stay true for a while and that would cause additional ARC reduction by arc_reclaim_thread. Final notes. I think that vm_paging_target() > -2048 check was modeled after the check in the original OpenSolaris code: freemem < lotsfree + needfree + extra The issue is that in my understanding OpenSolaris pagedaemon works differently from FreeBSD pagedaemon. OpenSolaris pagedaemon is activated when freemem (equivalent of our free + cache) falls down to a certain higher mark (lotsfree). Initially it scans pages at a slow rate. If freemem falls further the rate linearly increases until it reaches its maximum when freemem goes to or below certain lower mark. Our pagedaemon is activated when free + cache falls down to a value when vm_paging_needed() is true (see definition of this function). When it is activated it makes a scan pass though inactive and active pages setting a certain target for free+cache, but that target is "soft" and actually is an upper limit of how many pages could be freed during the pass. pagedaemon would make the second (or subsequent) pass only if free+cache falls to value that is even lower than the threshold in vm_paging_needed(), which means significant (severe even) memory pressure/shortage. So on sufficiently active system free+cache would typically oscillate between v_free_reserved+v_cache_min at the bottom and some semi-random values "near" v_free_target+v_cache_min at the tops. That's when excluding ARC from the picture. And about pictures :-) Behavior of free+cache with current arc_reclaim_needed code: http://people.freebsd.org/~avg/avail-mem-before.png and its behavior after the patch: http://people.freebsd.org/~avg/avail-mem-after.png The legends on the pictures are incorrect, sorry, my mastery of drraw is not good yet. Correct legends: "aqua" color - v_free_target+v_cache_min (vm_paging_target() == 0) "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() threshold) "lime" color - v_free_count+v_cache_count indeed :) Y axis - % of total page count. I think the graphs speak for themselves. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 00:14:26 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02FDF106564A; Mon, 23 Aug 2010 00:14:26 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 82BE48FC08; Mon, 23 Aug 2010 00:14:25 +0000 (UTC) Received: by vws7 with SMTP id 7so5554391vws.13 for ; Sun, 22 Aug 2010 17:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=r9FCry+Eg1eohNrqtsr3DynwTWe0jWjx7Vt0CEW3rtM=; b=tahhLFARIOcsPuGUqF2+WVbxIvOKEvNSYnueVaMLVf9SHe2ZahUuH9Ng77/VW7HCVF 6GEBNj/D1FbQLaTj8c0SmdXw7Tzw9LBonpnB7P0zFiw1HeitrtaAjK5SI5RyS4uyrZxD N8tRMl0hAhOwAgOhRwR6AKi6ZuKxU1cTsM0eE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=jT/RK5jBCFGn6ftRc9YPLpbB2iGax8UlLmw8LxaQQ+DrY+SqvaM8KOOFM98lYmViqz ilQqKyNFVEH+V/4B3ywsW/HZp/lltHuGqGPRibGzhepiAOOIo0U++4A43V2vU9BZz94m /kXNdMv70WAsawhYAYQk09aG9rOKnXdeEAJ9A= MIME-Version: 1.0 Received: by 10.220.88.167 with SMTP id a39mr2792109vcm.73.1282521120809; Sun, 22 Aug 2010 16:52:00 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Sun, 22 Aug 2010 16:52:00 -0700 (PDT) In-Reply-To: <4C719AB9.9020006@freebsd.org> References: <4C719AB9.9020006@freebsd.org> Date: Sun, 22 Aug 2010 16:52:00 -0700 X-Google-Sender-Auth: 5-hwxrj3geu8D5AWbwn-u5kaWJ8 Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 00:14:26 -0000 Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size behavior in addition to the stuff included on your graphs now? All I can tell from your graphs is that v_free_count+v_cache_count shifted a bit lower relative to v_free_target+v_cache_min. It would be interesting to see what effect your patch has on ARC itself, especially when ARC will start giving up memory and when does it stop shrinking. --Artem On Sun, Aug 22, 2010 at 2:46 PM, Andriy Gapon wrote: > > I propose that the following code in arc_reclaim_needed > (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) > /* > =A0* If pages are needed or we're within 2048 pages > =A0* of needing to page need to reclaim > =A0*/ > if (vm_pages_needed || (vm_paging_target() > -2048)) > > be changed to > > if (vm_paging_needed()) > > Rationale. > > 1. Why not current checks. > > ARC sizing should cooperate with pagedaemon in freeing pages. > If ARC starts shrinking "prematurely", before pagedaemon is waked up then= no > potentially eligible inactive pages will be recycled and no potentially e= ligible > active pages will be inactive (subject to v_inactive_target). > This would lead to ARC size going to its minimum value (which could hurt = ZFS > performance). =A0Only after that there is a chance that pagedaemon would = be waked > up to do its cleaning. > And conversely, if ARC doesn't shrink in time, then pagedaemon would have= to > recycle pages with data that could be needed again soon and that would le= ad to > excessive swapping and disk I/O. > > vm_paging_target() is used only by pagedaemon internally, it effectively = sets > _upper_ limit on how many pages pagedaemon would free when it's activated= . > It is no indication of whether pagedaemon should be scanning/freeing page= s. > Thus check of vm_paging_target() leads to premature ARC shrinking. > I believe that many people observe this behavior on sufficiently active s= ystems > (not dedicated file servers) with few GB of RAM (1-8). > > vm_pages_needed check is redundant, because this is a flag that is used t= o wake > up pagedaemon. =A0So when it is set vm_paging_needed() is true and > vm_paging_target() is "way" above zero. =A0And this flag is reset to zero= when > vm_page_count_min() becomes false, which corresponds to even fewer free p= ages > than when vm_paging_needed() is true. > > > 2. Why the new check. > > vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (= see > vm_page_alloc). =A0pagedaemon would first of all run vm_lowmem event for = which ARC > already has a handler and which would cause ARC size to shrink. > It would seems like having vm_paging_needed() check would be redundant th= en. > Almost - if memory pressure is significant, then vm_paging_needed() may s= tay > true for a while and that would cause additional ARC reduction by > arc_reclaim_thread. > > > Final notes. > > I think that > vm_paging_target() > -2048 > check was modeled after the check in the original OpenSolaris code: > freemem < lotsfree + needfree + extra > > The issue is that in my understanding OpenSolaris pagedaemon works differ= ently > from FreeBSD pagedaemon. > > OpenSolaris pagedaemon is activated when freemem (equivalent of our free = + > cache) falls down to a certain higher mark (lotsfree). =A0Initially it sc= ans pages > at a slow rate. =A0If freemem falls further the rate linearly increases u= ntil it > reaches its maximum when freemem goes to or below certain lower mark. > > Our pagedaemon is activated when free + cache falls down to a value when > vm_paging_needed() is true (see definition of this function). =A0When it = is > activated it makes a scan pass though inactive and active pages setting a > certain target for free+cache, but that target is "soft" and actually is = an > upper limit of how many pages could be freed during the pass. pagedaemon = would > make the second (or subsequent) pass only if free+cache falls to value th= at is > even lower than the threshold in vm_paging_needed(), which means signific= ant > (severe even) memory pressure/shortage. > So on sufficiently active system free+cache would typically oscillate bet= ween > v_free_reserved+v_cache_min at the bottom and some semi-random values "ne= ar" > v_free_target+v_cache_min at the tops. =A0That's when excluding ARC from = the picture. > > And about pictures :-) > Behavior of free+cache with current arc_reclaim_needed code: > http://people.freebsd.org/~avg/avail-mem-before.png > and its behavior after the patch: > http://people.freebsd.org/~avg/avail-mem-after.png > > The legends on the pictures are incorrect, sorry, my mastery of drraw is = not > good yet. > Correct legends: > "aqua" color - v_free_target+v_cache_min (vm_paging_target() =3D=3D 0) > "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() thresho= ld) > "lime" color - v_free_count+v_cache_count indeed :) > Y axis - % of total page count. > > I think the graphs speak for themselves. > > -- > Andriy Gapon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 06:12:58 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F26E1065673; Mon, 23 Aug 2010 06:12:58 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CC7988FC17; Mon, 23 Aug 2010 06:12:54 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA11269; Mon, 23 Aug 2010 09:12:51 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnQH5-000Dm3-56; Mon, 23 Aug 2010 09:12:51 +0300 Message-ID: <4C721161.40403@freebsd.org> Date: Mon, 23 Aug 2010 09:12:49 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 06:12:58 -0000 on 23/08/2010 02:52 Artem Belevich said the following: > Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size > behavior in addition to the stuff included on your graphs now? Yes, I do and not by a chance :-) > All I > can tell from your graphs is that v_free_count+v_cache_count shifted a > bit lower relative to v_free_target+v_cache_min. Don't belittle those graphs :-) Remember that the "fuchsia" line is when pagedaemon is woken up. > It would be > interesting to see what effect your patch has on ARC itself, > especially when ARC will start giving up memory and when does it stop > shrinking. In an extreme case it stops at arc_c_min as expected. An extreme case is when userland application(s) demand a lot of memory fast. Now the graphs: http://people.freebsd.org/~avg/arc1.png http://people.freebsd.org/~avg/arc2.png http://people.freebsd.org/~avg/pages.png http://people.freebsd.org/~avg/arc3.png What do you see? What do you think? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 07:28:09 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ACE51065694; Mon, 23 Aug 2010 07:28:09 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 83C798FC18; Mon, 23 Aug 2010 07:28:08 +0000 (UTC) Received: by vws7 with SMTP id 7so5779518vws.13 for ; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Jgtv7TWfJwnpmhOm6b4jsBomOPI9yv3rowE3OKMkCZo=; b=pzN3epxiogxs+7ZvR6Wxm8GLOPWhd/jJ9CqqhTEL2UGvNErmfTms30kAKRJXCuX4y/ 1THI3jhqLTA1kB1Y9wD3/iBSFXPF2JmvVP3Ypz1tkQOWszqL1rEJl/P9Gc4oNkowneDn eIWevlPBz9syLluiCV8LL85jGVQs0O+eVQOjk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=II9i4RHi031WiwmDntjWMTNywCUCIWVJ+XW2EdC+kYlk/H6eEKQOW6e338ONYbEGWl ukwRvxBAhcQuajAXAVgqrmMoKmW4kfiNqjdpjGg2TEmb45PrVa4EkZ+q9h0ti9RBLPpQ GpKzR4n8lJiyaV/huaLyg127B/GgnOKDzTvwE= MIME-Version: 1.0 Received: by 10.220.124.74 with SMTP id t10mr2850507vcr.179.1282548487609; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) In-Reply-To: <4C721161.40403@freebsd.org> References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> Date: Mon, 23 Aug 2010 00:28:07 -0700 X-Google-Sender-Auth: 8X7QKvbEAmQydQw8TDx4sxgnh9c Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 07:28:09 -0000 Ah! After re-reading your first email and I think I've finally got what you're saying -- with your change ARC would only start giving up memory when pagedaemon is awake. Presumably once it's awake it will also run through inactive list pushing some of it to cache. On the other hand existing code voluntarily frees up memory even before pagedaemon wakes up and does such a good job that pagedaemon practically never wakes up and thus does not get to clean inactive list. Can anyone test the patch on a system with mix of UFS/ZFS filesystems and see if the change mitigates or solves the issue with inactive memory excessively backpressuring ARC. Overall I think that your proposed change seems to make sense to me. If we could also deal with zone fragmentation issue you've written in another thread, that should bring ZFS even closer to being usable without shaman-style (the one with lots of muttering, swearing and dancing around) tuning. Actually, it may be worth trying your test with re-enabled UMA allocator for ARC. Now that pagedaemon will be running, it would also invoke UMA's low memory handlers and those should be able to give some memory back to the system. --Artem On Sun, Aug 22, 2010 at 11:12 PM, Andriy Gapon wrote: > on 23/08/2010 02:52 Artem Belevich said the following: >> Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size >> behavior in addition to the stuff included on your graphs now? > > Yes, I do and not by a chance :-) > >> All I >> can tell from your graphs is that v_free_count+v_cache_count shifted a >> bit lower relative to v_free_target+v_cache_min. > > Don't belittle those graphs :-) > Remember that the "fuchsia" line is when pagedaemon is woken up. > >> It would be >> interesting to see what effect your patch has on ARC itself, >> especially when ARC will start giving up memory and when does it stop >> shrinking. > > In an extreme case it stops at arc_c_min as expected. =A0An extreme case = is when > userland application(s) demand a lot of memory fast. > > Now the graphs: > http://people.freebsd.org/~avg/arc1.png > http://people.freebsd.org/~avg/arc2.png > http://people.freebsd.org/~avg/pages.png > http://people.freebsd.org/~avg/arc3.png > > What do you see? =A0What do you think? > > -- > Andriy Gapon > From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 07:32:02 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3E1D10656AB; Mon, 23 Aug 2010 07:32:01 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 16F978FC24; Mon, 23 Aug 2010 07:32:00 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA12358; Mon, 23 Aug 2010 10:31:58 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnRVe-000DtM-7q; Mon, 23 Aug 2010 10:31:58 +0300 Message-ID: <4C7223E9.8070801@freebsd.org> Date: Mon, 23 Aug 2010 10:31:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 07:32:02 -0000 on 23/08/2010 10:28 Artem Belevich said the following: > If we could also deal with zone fragmentation issue you've written in > another thread, that should bring ZFS even closer to being usable > without shaman-style (the one with lots of muttering, swearing and > dancing around) tuning. > > Actually, it may be worth trying your test with re-enabled UMA > allocator for ARC. Now that pagedaemon will be running, it would also > invoke UMA's low memory handlers and those should be able to give some > memory back to the system. I tried, but still no go (for my taste). The fragmentation is too high and very significant portion of memory is effectively lost to it. E.g. ARC may think that it uses only 1GB but another 1GB is used by free items in ZFS zones (of 4GB total). -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 20:52:55 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7242D10656A9; Mon, 23 Aug 2010 20:52:55 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id CB4BE8FC0A; Mon, 23 Aug 2010 20:52:54 +0000 (UTC) Received: by wwf26 with SMTP id 26so1530016wwf.31 for ; Mon, 23 Aug 2010 13:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=FT7hMW6eazNziYd4vKtoM2YRJ8pwKkGhB7GK/+FgnpM=; b=MosJ5LqcETAT7bwEzn9CKjqe7pMqS4F8crts5rLP/XLM771PkDJOLOK0i6WK38TYBT Ci6UKJLhmNAqsxbvamRb0FB9hS9L8tpcjBf3v1f7NQghqjNOCnW00eYFK70dj/7vIbDA 8JLEYV0OLHEQEExHBNyO9vxf1piGPJoCZcGqA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=wGQKRGsmYpqyS4i+Q5wf+3qDobxP/pIgYMiNbNTL9m+FuUuLDj+gRsfqnqbwQoGAC2 DzzK5yAAheh6K4SFhzPyICxuDm2xtCQtZUnQ0+vkTFOZROWkxatLETRFn/7F7C7cNUCG gxJ/D5m8RirG9MddumVVHFh3vRsWWDnIloq+Y= Received: by 10.227.127.193 with SMTP id h1mr5000926wbs.139.1282596294803; Mon, 23 Aug 2010 13:44:54 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id i14sm3860128wbe.0.2010.08.23.13.44.53 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 13:44:54 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C72DDC3.1000006@DataIX.net> Date: Mon, 23 Aug 2010 16:44:51 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> In-Reply-To: <4C72DD1A.9070204@DataIX.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 23 Aug 2010 21:12:04 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 20:52:55 -0000 On 08/23/2010 16:42, jhell wrote: > On 08/23/2010 03:28, Artem Belevich wrote: >> Can anyone test the patch on a system with mix of UFS/ZFS filesystems >> and see if the change mitigates or solves the issue with inactive >> memory excessively backpressuring ARC. > > I have a system currently patched up to ZFSv15 and mm@'s metaslab patch > running that I can test this on. Throw me a patch and some specific > tests and I can have the results back to you in the next 2 days. > Forget the patch 1 line change I can hand type that in. As for specific tests, let me know... -- jhell,v From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 21:41:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57EA81065670; Mon, 23 Aug 2010 21:41:34 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 00C578FC19; Mon, 23 Aug 2010 21:41:34 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 15D29A6047C; Tue, 24 Aug 2010 05:41:33 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id UP1Bi1WzAqJZ; Tue, 24 Aug 2010 05:41:27 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id B33C3A6045F; Tue, 24 Aug 2010 05:41:24 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=dUoIu/evJhPKs5ojRXkSDwzOq+GGHKrQv1Vixe7Uc/mlUjgabV7LCtbEI3etvw+1M Oqm0pa5rMXkZpyPWtX7CQ== Message-ID: <4C72EAFF.6060800@delphij.net> Date: Mon, 23 Aug 2010 14:41:19 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.11) Gecko/20100721 Thunderbird/3.0.6 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: Martin Matuska References: <4C6FDE77.5070607@FreeBSD.org> In-Reply-To: <4C6FDE77.5070607@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.ORG, Pawel Jakub Dawidek Subject: Re: Patch proposal: Speeding up ZFS writes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 21:41:34 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Martin, On 2010/08/21 07:11, Martin Matuska wrote: > b) on the mid-term, I suggest this patch for head with MFC to stable/8 > after some reasonable time (1-2 months): > http://people.freebsd.org/~mm/patches/zfs/zfs_metaslab.patch [...] > The patch in b) includes the following OpenSolaris onnv revisions: > 10921 (very small part, metaslab.c) > 11146 (main patch, applies almost cleanly) > 11728 (fix for zdb.c) > 12047 (improvement to metaslab.c) > > OpenSolaris Bug IDs: > 6826241 Sync write IOPS drops dramatically during TXG sync > 6869229 zfs should switch to shiny new metaslabs more frequently > 6917066 zfs block picking can be improved > 6918420 zdb -m has issues printing metaslab statistics I think we should integrate this. The only thing I feel confusing about the change is that the part imported from OpenSolaris onnv revision 10921 on metaslab.c in metaslab_activate(), which adds a space_map_load_wait(sm), does not seem to be really necessary (the later 11146 change just added a blank space there) as the corresponding space_map_load() change that removed the wait() call was not merged. Or, did I missed something here? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMcur+AAoJEATO+BI/yjfBoQkH/18thCEm++1vNXrvWChRjLSF fDrd6yoU7JXjSHkM7t/rE+D0/TEbL2Na2Fu0K8Ex4+qU3OBy14AcAxU2a+c+BV0T mf3PK6ne8lv2NijQ6fhBQ6fnsHqCnGnbjNfSTmNTQq3PJzisBe4pajSqSxqXJDUU pxbXVzssoDQfTuaoGmUXVkqMY2Tkn0YmLJ9yMvZCq0xVSjHd8cLGy71gCh8RGFTs dLqItQiaLWIniO5iFAeJtx9b7SmqUU2pu/WpDDg9h2zpchn8b/hoTkpR6zonclke yrYAhp+pX29HxhUheIkqxyvZJxFQ/JUekaGMZisYGH3JUaU1Vb+4CGimH0u7dUc= =g+3i -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 21:05:15 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD63A10656A8; Mon, 23 Aug 2010 21:05:14 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 34A6C8FC22; Mon, 23 Aug 2010 21:05:13 +0000 (UTC) Received: by bwz20 with SMTP id 20so502957bwz.13 for ; Mon, 23 Aug 2010 14:05:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=jHbvOGVv5XG6OltEtbCGp0MzipA3KSYzOjAW7p8hngk=; b=UjTgf9uVkYYeyJN1LXC/EwdHff6K7RkJVa7OsGrqGpgFg6rpt0VIRGM2avsmjoMom8 o8HtYV6gEn05d9lWZb/jvg9Wdh3WI2OKn1b7x74qnQrhUYHajU2mtaTRfMCOdONc+tjd tJLEGsDaf8WapZ0nqXNFNCgLQEtnHC8ygPvLc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=n84hGOdbZ2ERFOfaBBYPDNwUnBvyFBccJenXvVsU4201jjKR/2UtVaut3NLm09ogQE W6sricUb6UxRIUs+VOjbWWJNkB6eTu9iYjtyagNHvfp/KY4l2wAAqHNNjo8CeTf9M+y8 R/B5rzQqKfEM+6dXWfLW4C4ssvUBbbSR72ObM= Received: by 10.213.17.203 with SMTP id t11mr4067082eba.66.1282596126483; Mon, 23 Aug 2010 13:42:06 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id v8sm11442799eeh.8.2010.08.23.13.42.04 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 13:42:05 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C72DD1A.9070204@DataIX.net> Date: Mon, 23 Aug 2010 16:42:02 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 23 Aug 2010 21:45:24 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 21:05:15 -0000 On 08/23/2010 03:28, Artem Belevich wrote: > Can anyone test the patch on a system with mix of UFS/ZFS filesystems > and see if the change mitigates or solves the issue with inactive > memory excessively backpressuring ARC. I have a system currently patched up to ZFSv15 and mm@'s metaslab patch running that I can test this on. Throw me a patch and some specific tests and I can have the results back to you in the next 2 days. Regards, -- jhell,v From owner-zfs-devel@FreeBSD.ORG Tue Aug 24 02:11:01 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BB4C106566B; Tue, 24 Aug 2010 02:11:01 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 067AE8FC0C; Tue, 24 Aug 2010 02:11:00 +0000 (UTC) Received: by vws7 with SMTP id 7so26409vws.13 for ; Mon, 23 Aug 2010 19:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=TH0jI61hCMLXRzy4CdFHziS2srG+iybtfFG5uSHpIlQ=; b=VtMn8GDDH6kpofmw5rw1EUyy4YrH+NWdkYQ42N4Hq5BnEOxxeyjWcD3+AgWNSRqZPv h8JQ41aNCOBd8WJs8ZfInNFv/lxJ+D3aLaj2UOpfCz0OnRDnzFbx19zeNvNVvqv/W4lH ix4zKVJPY5RRC3vPEFRNDDuP/xz9HDrwR4pzQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=HU+fiejTt5yHKodzNqWts6SwDcIY7ZWZGgmsPQMXlBShBebey5wwKt2YTxF7mWg/TO /l4aH4Wyuiw+q+y3e//A27NEGp+6EFfcFH2ni2ksoR+LuEOC4jMT5v8iGW/mqs46CA8Z BlCyN2AaTKMH5AQEs2+mPO1AUD62M48wrwrxU= MIME-Version: 1.0 Received: by 10.220.85.196 with SMTP id p4mr3667193vcl.271.1282615859967; Mon, 23 Aug 2010 19:10:59 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Mon, 23 Aug 2010 19:10:59 -0700 (PDT) In-Reply-To: <4C72DDC3.1000006@DataIX.net> References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> <4C72DDC3.1000006@DataIX.net> Date: Mon, 23 Aug 2010 19:10:59 -0700 X-Google-Sender-Auth: 185f49DgvbAQMAeXOWU8l1Vx1WQ Message-ID: From: Artem Belevich To: jhell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 02:11:01 -0000 Could you try following experiments before and after the patch while monitoring kstat.zfs.misc.arcstats.size and vm.stats.vm.v_inactive_count. First prepare the data. * You'll need some files totalling around the amount of physical memory on your box. Multiple copies of /usr/src should do the trick. * Place one copy on UFS filesystem and another on ZFS Experiment #1: * Prime ARC by tarring dataset on ZFS into /dev/null. * Now tar both datasets in parallel with output to /dev/null Previously you would end up with ARC size shrinking down to arc_min. What I hope to see after the patch is that inactive memory and ARC reach some sort of equilibrium with neither monopolizing all available memory. #Experiment #2: If equilibrium is reached, try running some application that would allocate and use about 1/2 of your physical memory. Something like that perl one-liner used to cause memory shortage, only a bit less drastic. perl -e '$x=3D"x"x1_000_000_000'; # this should allocate about 2GB. Tune the number to suit your system. Again, in the past ARC would be the one feeing up the memory. Let's see if inactive list gives up some, too. --Artem On Mon, Aug 23, 2010 at 1:44 PM, jhell wrote: > On 08/23/2010 16:42, jhell wrote: >> On 08/23/2010 03:28, Artem Belevich wrote: >>> Can anyone test the patch on a system with mix of UFS/ZFS filesystems >>> and see if the change mitigates or solves the issue with inactive >>> memory excessively backpressuring ARC. >> >> I have a system currently patched up to ZFSv15 and mm@'s metaslab patch >> running that I can test this on. Throw me a patch and some specific >> tests and I can have the results back to you in the next 2 days. >> > > Forget the patch 1 line change I can hand type that in. As for specific > tests, let me know... > > -- > > =A0jhell,v > From owner-zfs-devel@FreeBSD.ORG Tue Aug 24 02:28:24 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3DAC106566B; Tue, 24 Aug 2010 02:28:24 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2A2038FC0C; Tue, 24 Aug 2010 02:28:23 +0000 (UTC) Received: by qwg5 with SMTP id 5so6357727qwg.13 for ; Mon, 23 Aug 2010 19:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=4fqi2BKYIFEmvU/VaYL9GXnmLV0JV5sO/hxD8y23fXs=; b=M2Qj8kXme19LMVKALA+bjWIEeYNPKCijeayda53m1IusGhUGVRQBvU8s3v4PumuTX+ 72+f6C+h8lxI7s/yqsPaI3+NTn30gMn0YWW9fH48uTnSzOEnRAOnEDxb+waQ9pOFV3f4 1nxIonIY6FZMW6PTu5hzj325/yyWVW6sIYuAo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=CDhllPG/5p5AXC7g4Ag5NBQmenfeMYIZEVUIt66q69RkzsL2d3sMYWeHscJv9u6x68 5f9djOhJMH0rySgHs5bEBShxAdCjQ5U5tnU4sK+qdHySfJUgHyrg1uFrUCvJTb0grPy7 hPMPTIj3/pWmqq12DFOojVtP0EV3sualPRk4A= Received: by 10.229.222.6 with SMTP id ie6mr4311550qcb.28.1282616903208; Mon, 23 Aug 2010 19:28:23 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id t4sm7901199qcs.40.2010.08.23.19.28.21 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 19:28:22 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C732E44.50702@DataIX.net> Date: Mon, 23 Aug 2010 22:28:20 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> <4C72DDC3.1000006@DataIX.net> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 24 Aug 2010 02:39:54 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 02:28:24 -0000 On 08/23/2010 22:10, Artem Belevich wrote: > First prepare the data. > * You'll need some files totalling around the amount of physical > memory on your box. Multiple copies of /usr/src should do the trick. > * Place one copy on UFS filesystem and another on ZFS > > Experiment #1: > * Prime ARC by tarring dataset on ZFS into /dev/null. > * Now tar both datasets in parallel with output to /dev/null > > Previously you would end up with ARC size shrinking down to arc_min. > What I hope to see after the patch is that inactive memory and ARC > reach some sort of equilibrium with neither monopolizing all available > memory. > > #Experiment #2: > If equilibrium is reached, try running some application that would > allocate and use about 1/2 of your physical memory. > Something like that perl one-liner used to cause memory shortage, only > a bit less drastic. > perl -e '$x="x"x1_000_000_000'; # this should allocate about 2GB. > Tune the number to suit your system. > > Again, in the past ARC would be the one feeing up the memory. Let's > see if inactive list gives up some, too. Alright I should be able to have something together with the output of this by the 25th with my current workload being the ultimate determining factor. PBS, Regards, -- jhell,v From owner-zfs-devel@FreeBSD.ORG Tue Aug 31 22:19:01 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A7D510656A6 for ; Tue, 31 Aug 2010 22:19:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 728908FC1A for ; Tue, 31 Aug 2010 22:19:00 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4006745DD8; Tue, 31 Aug 2010 23:59:30 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 61EF845B36; Tue, 31 Aug 2010 23:59:24 +0200 (CEST) Date: Tue, 31 Aug 2010 23:59:15 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.org Message-ID: <20100831215915.GE1932@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RpqchZ26BWispMcB" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 X-Mailman-Approved-At: Tue, 31 Aug 2010 22:21:22 +0000 Cc: freebsd-current@FreeBSD.org Subject: ZFS v28 is ready for wider testing. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2010 22:19:01 -0000 --RpqchZ26BWispMcB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello. I'd like to give you ZFS v28 for testing. If you are neither brave nor mad, you can stop here. The patchset is very experimental. It can eat your cookie and hurt your teddy bear, so be warned. Don't try it for anything except testing. This patchset is also a message we, as the FreeBSD project, would like to send to our users: Eventhough OpenSolaris is dead, the ZFS file system is going to stay in FreeBSD. At this point we have quite a few developers involved in ZFS on FreeBSD as well as serveral companies. We are also looking forward to work with IllumOS. So, what this new ZFS brings? - Data deduplication. Read more here: http://blogs.sun.com/bonwick/entry/zfs_dedup - Triple parity RAIDZ (RAIDZ3). Read more here: http://dtrace.org/blogs/ahl/2009/07/21/triple-parity-raid-z/ - zfs diff. Read more here: http://arc.opensolaris.org/caselog/PSARC/2010/105/20100328_tim.haley - zpool split. Read more here: http://arc.opensolaris.org/caselog/PSARC/2009/511/20090924_mark.musante - Snapshot holds. Read more here: http://arc.opensolaris.org/caselog/PSARC/2009/297/20090511_chris.kirby - zpool import -F. Allows to rewind corrupted pool to earlier transaction group. - Possibility to import pool in read-only mode. And much, much more, including plenty of preformance improvements and bug fixes. So test whatever you can and report back. Look for regressions, strange behaviour, missing features, deadlocks, livelocks, preformance degradation, etc. The boot code is not updated at all, so booting off of ZFS doesn't currently work. The patch is against today's FreeBSD HEAD. The patch enables (in sys/modules/zfs/Makefile) ZFS internal debugging, please don't turn it off. Also, compile your kernel with the following options: options KDB options DDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options WITNESS_SKIPSPIN options DEBUG_LOCKS options DEBUG_VFS_LOCKS Ignore all the LOR (Lock Order Reversal) reports from WITNESS. There will be plenty of those, and you'll desperately want to report them, but please don't. The best way to report a problem is to answer to this e-mail with as short as possible procedure of how to reproduce it and debugging info. I'd prefer textdump if possible. Below you can find quick procedure how to setup textdumps: Choose spare/swap disk/partition in your system, let's say it is /dev/ad0s1b. Add the following line to /etc/fstab: /dev/ad0s1b none swap sw 0 0 Add the following line to /etc/rc.conf: ddb_enable=3D"YES" Run the following commands: # /etc/rc.d/swap1 start # /etc/rc.d/dumpon start # /etc/rc.d/ddb start This will setup swap, mark it as dump device and setup some DDB scripts. Or you can just reboot. Now when your system panic or deadlock, enter DDB and call the following command: ddb> run kdb.enter.panic It will execute all the commands I need, dump them in text format to your swap device and reboot machine. After the reboot, you should find textdump.tar.0 file in /var/crash/ directory. This is the debug info I need. End of textdumps procedure. Ok, now that I know you read everything carefully, here is the patch: http://people.freebsd.org/~pjd/patches/zfs_20100831.patch.bz2 Good luck! >:> --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --RpqchZ26BWispMcB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkx9ezMACgkQForvXbEpPzQ+ZwCg6EtfJjx6X1nJaj5uTkEM2fwx HkoAoJ5/L97SHbIHcyLrOqmH/t4oBmFi =ePEI -----END PGP SIGNATURE----- --RpqchZ26BWispMcB-- From owner-zfs-devel@FreeBSD.ORG Sun Sep 12 13:46:20 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60F331065672; Sun, 12 Sep 2010 13:46:20 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8653E8FC08; Sun, 12 Sep 2010 13:46:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AC12EEDAFA; Sun, 12 Sep 2010 15:46:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id YtO9QzL6oeBD; Sun, 12 Sep 2010 15:46:03 +0200 (CEST) Received: from [127.0.0.1] (ip-91.191.85.28.o2inet.sk [91.191.85.28]) by mail.vx.sk (Postfix) with ESMTPSA id 624EEEDAE0; Sun, 12 Sep 2010 15:46:00 +0200 (CEST) Message-ID: <4C8CD994.6060103@FreeBSD.org> Date: Sun, 12 Sep 2010 15:45:56 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4C719AB9.9020006@freebsd.org> In-Reply-To: <4C719AB9.9020006@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 13:46:20 -0000 I am running a really busy system with 48GB of RAM, equipped with 300GB of SSD L2 cache as well and the I have recently observed the problem. The system runs static fileserving from ZFS via sendfile() and NFS export. My vfs.zfs.arc_max is set to 40GB, vfs.zfs.arc_min is auto-set to 5GB (= metalimit / 2, metalimit = arc_max / 4) NFS generates inactive memory that is not in use and leads to a reduction of ARC up to vfs.zfs.arc_min (not below). But I actually don't care about the cache of NFS and I really need the ARC. So if I understand this correctly, this might help me - but it would lead to some kind of balance of incative memory vs ARC. Anyway, best help for me would probably be setting vfs.zfs.arc_min to a high value (20GB or more) so that the system tries to stay at this level. mm Dňa 22. 8. 2010 23:46, Andriy Gapon wrote / napísal(a): > I propose that the following code in arc_reclaim_needed > (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) > /* > * If pages are needed or we're within 2048 pages > * of needing to page need to reclaim > */ > if (vm_pages_needed || (vm_paging_target() > -2048)) > > be changed to > > if (vm_paging_needed()) > > Rationale. > > 1. Why not current checks. > > ARC sizing should cooperate with pagedaemon in freeing pages. > If ARC starts shrinking "prematurely", before pagedaemon is waked up then no > potentially eligible inactive pages will be recycled and no potentially eligible > active pages will be inactive (subject to v_inactive_target). > This would lead to ARC size going to its minimum value (which could hurt ZFS > performance). Only after that there is a chance that pagedaemon would be waked > up to do its cleaning. > And conversely, if ARC doesn't shrink in time, then pagedaemon would have to > recycle pages with data that could be needed again soon and that would lead to > excessive swapping and disk I/O. > > vm_paging_target() is used only by pagedaemon internally, it effectively sets > _upper_ limit on how many pages pagedaemon would free when it's activated. > It is no indication of whether pagedaemon should be scanning/freeing pages. > Thus check of vm_paging_target() leads to premature ARC shrinking. > I believe that many people observe this behavior on sufficiently active systems > (not dedicated file servers) with few GB of RAM (1-8). > > vm_pages_needed check is redundant, because this is a flag that is used to wake > up pagedaemon. So when it is set vm_paging_needed() is true and > vm_paging_target() is "way" above zero. And this flag is reset to zero when > vm_page_count_min() becomes false, which corresponds to even fewer free pages > than when vm_paging_needed() is true. > > > 2. Why the new check. > > vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (see > vm_page_alloc). pagedaemon would first of all run vm_lowmem event for which ARC > already has a handler and which would cause ARC size to shrink. > It would seems like having vm_paging_needed() check would be redundant then. > Almost - if memory pressure is significant, then vm_paging_needed() may stay > true for a while and that would cause additional ARC reduction by > arc_reclaim_thread. > > > Final notes. > > I think that > vm_paging_target() > -2048 > check was modeled after the check in the original OpenSolaris code: > freemem < lotsfree + needfree + extra > > The issue is that in my understanding OpenSolaris pagedaemon works differently > from FreeBSD pagedaemon. > > OpenSolaris pagedaemon is activated when freemem (equivalent of our free + > cache) falls down to a certain higher mark (lotsfree). Initially it scans pages > at a slow rate. If freemem falls further the rate linearly increases until it > reaches its maximum when freemem goes to or below certain lower mark. > > Our pagedaemon is activated when free + cache falls down to a value when > vm_paging_needed() is true (see definition of this function). When it is > activated it makes a scan pass though inactive and active pages setting a > certain target for free+cache, but that target is "soft" and actually is an > upper limit of how many pages could be freed during the pass. pagedaemon would > make the second (or subsequent) pass only if free+cache falls to value that is > even lower than the threshold in vm_paging_needed(), which means significant > (severe even) memory pressure/shortage. > So on sufficiently active system free+cache would typically oscillate between > v_free_reserved+v_cache_min at the bottom and some semi-random values "near" > v_free_target+v_cache_min at the tops. That's when excluding ARC from the picture. > > And about pictures :-) > Behavior of free+cache with current arc_reclaim_needed code: > http://people.freebsd.org/~avg/avail-mem-before.png > and its behavior after the patch: > http://people.freebsd.org/~avg/avail-mem-after.png > > The legends on the pictures are incorrect, sorry, my mastery of drraw is not > good yet. > Correct legends: > "aqua" color - v_free_target+v_cache_min (vm_paging_target() == 0) > "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() threshold) > "lime" color - v_free_count+v_cache_count indeed :) > Y axis - % of total page count. > > I think the graphs speak for themselves. > From owner-zfs-devel@FreeBSD.ORG Sun Sep 12 15:25:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 858321065673 for ; Sun, 12 Sep 2010 15:25:36 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9477D8FC08 for ; Sun, 12 Sep 2010 15:25:35 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA28288; Sun, 12 Sep 2010 18:25:34 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OuoQv-000FyL-TR; Sun, 12 Sep 2010 18:25:33 +0300 Message-ID: <4C8CF0ED.9030004@freebsd.org> Date: Sun, 12 Sep 2010 18:25:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Martin Matuska References: <4C719AB9.9020006@freebsd.org> <4C8CD994.6060103@FreeBSD.org> In-Reply-To: <4C8CD994.6060103@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 15:25:36 -0000 on 12/09/2010 16:45 Martin Matuska said the following: > I am running a really busy system with 48GB of RAM, equipped with 300GB > of SSD L2 cache as well and the I have recently observed the problem. > The system runs static fileserving from ZFS via sendfile() and NFS export. > > My vfs.zfs.arc_max is set to 40GB, vfs.zfs.arc_min is auto-set to 5GB (= > metalimit / 2, metalimit = arc_max / 4) > > NFS generates inactive memory that is not in use and leads to a > reduction of ARC up to vfs.zfs.arc_min (not below). > But I actually don't care about the cache of NFS and I really need the ARC. > > So if I understand this correctly, this might help me - but it would > lead to some kind of balance of incative memory vs ARC. > Anyway, best help for me would probably be setting vfs.zfs.arc_min to a Well, I am not sure - until you define a criterion "best" sounds a bit subjective :-) Anyway, I will greatly appreciate if you could test the patch before changing arc_min. > high value (20GB or more) so that the system tries to stay at this level. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 09:38:25 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84C12106564A for ; Thu, 16 Sep 2010 09:38:25 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 1D0848FC15 for ; Thu, 16 Sep 2010 09:38:25 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 4B86110DCE0 for ; Thu, 16 Sep 2010 11:38:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id gbq8ixxlsMxc for ; Thu, 16 Sep 2010 11:38:11 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 9279B10DCD5 for ; Thu, 16 Sep 2010 11:38:11 +0200 (CEST) Message-ID: <4C91E582.3080509@FreeBSD.org> Date: Thu, 16 Sep 2010 11:38:10 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 09:38:25 -0000 Hi everyone, Oracle has released Solaris 10 update 9 on September, 8th. The new update comes with ZFS v22 without deduplication (legal or technical issues?), but includes the new metaslab code update. > zpool upgrade -v This system is currently running ZFS pool version 22. The following versions are supported: VER DESCRIPTION --- -------------------------------------------------------- 1 Initial ZFS version 2 Ditto blocks (replicated metadata) 3 Hot spares and double parity RAID-Z 4 zpool history 5 Compression using the gzip algorithm 6 bootfs pool property 7 Separate intent log devices 8 Delegated administration 9 refquota and refreservation properties 10 Cache devices 11 Improved scrub performance 12 Snapshot properties 13 snapused property 14 passthrough-x aclinherit 15 user/group space accounting 16 stmf property support 17 Triple-parity RAID-Z 18 Snapshot user holds 19 Log device removal 20 Compression using zle (zero-length encoding) 21 Reserved 22 Received properties From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 11:48:17 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 585E2106564A; Thu, 16 Sep 2010 11:48:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 1BDB68FC08; Thu, 16 Sep 2010 11:48:17 +0000 (UTC) Received: from [10.0.0.3] (warner-iphone.bsdimp.com [10.0.0.3]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o8GBf6WG002084; Thu, 16 Sep 2010 05:41:07 -0600 (MDT) (envelope-from imp@bsdimp.com) From: Warner Losh To: Martin Matuska In-Reply-To: <4C91E582.3080509@FreeBSD.org> X-Mailer: iPhone Mail (7D11) References: <4C91E582.3080509@FreeBSD.org> Message-Id: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (iPhone Mail 7D11) Date: Thu, 16 Sep 2010 05:40:33 -0600 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 11:48:17 -0000 With or without sources??? Warner On Sep 16, 2010, at 3:38 AM, Martin Matuska wrote: > Hi everyone, > > Oracle has released Solaris 10 update 9 on September, 8th. > > The new update comes with ZFS v22 without deduplication (legal or > technical issues?), but includes the new metaslab code update. > >> zpool upgrade -v > This system is currently running ZFS pool version 22. > > The following versions are supported: > > VER DESCRIPTION > --- -------------------------------------------------------- > 1 Initial ZFS version > 2 Ditto blocks (replicated metadata) > 3 Hot spares and double parity RAID-Z > 4 zpool history > 5 Compression using the gzip algorithm > 6 bootfs pool property > 7 Separate intent log devices > 8 Delegated administration > 9 refquota and refreservation properties > 10 Cache devices > 11 Improved scrub performance > 12 Snapshot properties > 13 snapused property > 14 passthrough-x aclinherit > 15 user/group space accounting > 16 stmf property support > 17 Triple-parity RAID-Z > 18 Snapshot user holds > 19 Log device removal > 20 Compression using zle (zero-length encoding) > 21 Reserved > 22 Received properties > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 12:18:09 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DA6E106566C for ; Thu, 16 Sep 2010 12:18:09 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DC2DB8FC18 for ; Thu, 16 Sep 2010 12:18:08 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA00676; Thu, 16 Sep 2010 14:59:17 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C920695.5000400@icyb.net.ua> Date: Thu, 16 Sep 2010 14:59:17 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100909 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: zfs-devel@freebsd.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: changes in arc_reclaim_needed X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 12:18:09 -0000 Pawel, I would like to go ahead and commit a (much tested now) change to arc_reclaim_needed: Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 212636) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -2164,7 +2164,7 @@ * If pages are needed or we're within 2048 pages * of needing to page need to reclaim */ - if (vm_pages_needed || (vm_paging_target() > -2048)) + if (vm_paging_needed()) return (1); #if 0 Additionally I also want to commit the following change: Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 212636) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -2155,10 +2155,6 @@ #ifdef _KERNEL if (needfree) return (1); - if (arc_size > arc_c_max) - return (1); - if (arc_size <= arc_c_min) - return (0); /* * If pages are needed or we're within 2048 pages Rationale. OpenSolaris upstream code doesn't have these checks. I've been using code changed as above for more than two months and all this time ARC size always stayed within its bounds. So the range is enforced in some other, more appropriate places. Reducing diff with upstream is also not bad. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 19:55:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D9A11065675 for ; Thu, 16 Sep 2010 19:55:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id A6A4B8FC1E for ; Thu, 16 Sep 2010 19:55:03 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 8679A110354; Thu, 16 Sep 2010 21:55:02 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id TDJBXhjv31oi; Thu, 16 Sep 2010 21:54:48 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 85514110324; Thu, 16 Sep 2010 21:54:48 +0200 (CEST) Message-ID: <4C927608.8080609@FreeBSD.org> Date: Thu, 16 Sep 2010 21:54:48 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Warner Losh References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> In-Reply-To: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 19:55:04 -0000 Of course without sources. And more news, Netapp and Oracle have agreed to dismiss lawsuits. http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html The question is, why is Solaris 10 U9 without deduplication? If this has something to do with the agreement, one possibility is that they agreed deduplication may only be in Oracle Storage Appliances and Oracle will honor their patent. That would mean for us: no deduplication, or anybody using or selling it without paying a royalty to netapp can go to court But that is just a speculation as there is no more information available. Anyway, it is better to be cautious than to end like Coraid. Dňa 16. 9. 2010 13:40, Warner Losh wrote / napísal(a): > With or without sources??? > > Warner > > > On Sep 16, 2010, at 3:38 AM, Martin Matuska wrote: > >> Hi everyone, >> >> Oracle has released Solaris 10 update 9 on September, 8th. >> >> The new update comes with ZFS v22 without deduplication (legal or >> technical issues?), but includes the new metaslab code update. >> >>> zpool upgrade -v >> This system is currently running ZFS pool version 22. >> >> The following versions are supported: >> >> VER DESCRIPTION >> --- -------------------------------------------------------- >> 1 Initial ZFS version >> 2 Ditto blocks (replicated metadata) >> 3 Hot spares and double parity RAID-Z >> 4 zpool history >> 5 Compression using the gzip algorithm >> 6 bootfs pool property >> 7 Separate intent log devices >> 8 Delegated administration >> 9 refquota and refreservation properties >> 10 Cache devices >> 11 Improved scrub performance >> 12 Snapshot properties >> 13 snapused property >> 14 passthrough-x aclinherit >> 15 user/group space accounting >> 16 stmf property support >> 17 Triple-parity RAID-Z >> 18 Snapshot user holds >> 19 Log device removal >> 20 Compression using zle (zero-length encoding) >> 21 Reserved >> 22 Received properties >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> >> From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 20:30:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44B25106567A; Thu, 16 Sep 2010 20:30:40 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id D77BB8FC16; Thu, 16 Sep 2010 20:30:39 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9D11345D8D; Thu, 16 Sep 2010 22:30:37 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 96B6945C98; Thu, 16 Sep 2010 22:30:31 +0200 (CEST) Date: Thu, 16 Sep 2010 22:30:15 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100916203015.GA1967@garage.freebsd.pl> References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jI8keyz6grp/JLjh" Content-Disposition: inline In-Reply-To: <4C927608.8080609@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 20:30:40 -0000 --jI8keyz6grp/JLjh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: > Of course without sources. >=20 > And more news, Netapp and Oracle have agreed to dismiss lawsuits. >=20 > http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement= .html >=20 > The question is, why is Solaris 10 U9 without deduplication? If this has > something to do with the agreement, one possibility is that they agreed > deduplication may only be in Oracle Storage Appliances and Oracle will > honor their patent. >=20 > That would mean for us: no deduplication, or anybody using or selling it > without paying a royalty to netapp can go to court >=20 > But that is just a speculation as there is no more information available. >=20 > Anyway, it is better to be cautious than to end like Coraid. My understanding is very different. I was under impression that NetApp cannot go after people using OpenSolaris until they win in court. Before that every OpenSolaris code consumer is protected by CDDL. NetApp didn't win, right? Also: "both parties have agreed to dismiss their pending patent litigation" My English might not be enough here, but my reading is that because patent litigation was dismissed, NetApp no longer claims (in legal sense) that Oracle violates NetApp's patents via OpenSolaris. Is my reading correct or not? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --jI8keyz6grp/JLjh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkySflYACgkQForvXbEpPzQCEACgjC3q6U1mCGaI5SrWf2nK6+Xo MQYAniKJAC49OYfal0THimuxto07iqsT =i9Po -----END PGP SIGNATURE----- --jI8keyz6grp/JLjh-- From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 21:08:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35FA71065672; Thu, 16 Sep 2010 21:08:04 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8F84D8FC18; Thu, 16 Sep 2010 21:08:03 +0000 (UTC) Received: by wyb33 with SMTP id 33so2384128wyb.13 for ; Thu, 16 Sep 2010 14:08:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=JuYvCJvJmqEXl1RHpPBLEAj71pH2b9VLqeFd/G2qluk=; b=FQas36J82wLE8GxUyLEK+gAh28gInWsqAZdSTOb7e6IRXTq1+tEJuiG2u4tWoSBgwU DHg023Tc0AWQzrx3UbSyJYUZUJ/L1sn2Bw9/a1vbu2PDCtHvoW7WTHhvXpCf3rT+7Kdn P/Hrt1HhlKM1tvcUnFrR5ETdrtkMa4PfP8kuI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=ujMFeB78uee28rRNCpbHZopSTqGOq9MTRNGE+bTSQEBLpO7ttYSPqFscF/2FAe066F T2+rnDQSXQiZ3ZSzS/K4hISB+kfaCmrv+Z9Z/R8TtKxa+JSf+hTCuKXY9qMxZOyzWiFR Ez3h6U3JAt6a3pFE6rp9NortmR0lil/JRXkZI= MIME-Version: 1.0 Received: by 10.227.142.136 with SMTP id q8mr3346262wbu.95.1284669466507; Thu, 16 Sep 2010 13:37:46 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.227.154.147 with HTTP; Thu, 16 Sep 2010 13:37:46 -0700 (PDT) In-Reply-To: <20100916203015.GA1967@garage.freebsd.pl> References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> Date: Thu, 16 Sep 2010 13:37:46 -0700 X-Google-Sender-Auth: rtd5N9ri6027KF0nxeEulYvGLLI Message-ID: From: Matt Olander To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 21:08:04 -0000 On Thu, Sep 16, 2010 at 1:30 PM, Pawel Jakub Dawidek wrote: > On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: >> Of course without sources. >> >> And more news, Netapp and Oracle have agreed to dismiss lawsuits. >> >> http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html >> >> The question is, why is Solaris 10 U9 without deduplication? If this has >> something to do with the agreement, one possibility is that they agreed >> deduplication may only be in Oracle Storage Appliances and Oracle will >> honor their patent. >> >> That would mean for us: no deduplication, or anybody using or selling it >> without paying a royalty to netapp can go to court >> >> But that is just a speculation as there is no more information available. >> >> Anyway, it is better to be cautious than to end like Coraid. > > My understanding is very different. I was under impression that NetApp > cannot go after people using OpenSolaris until they win in court. Before > that every OpenSolaris code consumer is protected by CDDL. NetApp didn't > win, right? > > Also: > > "both parties have agreed to dismiss their pending patent litigation" > > My English might not be enough here, but my reading is that because patent > litigation was dismissed, NetApp no longer claims (in legal sense) that > Oracle violates NetApp's patents via OpenSolaris. Is my reading correct > or not? Yes, this is also how I interpreted it. Coraid backed down selling an appliance that was not theirs (Nexenta on Supermicro) based on a simple cease & desist letter. They are a very small company and would rather back down than face any kind of possible legal action. In fact, Nexenta (the more appropriate target) never received the letter. Either way, both parties dismissed their claims. -matt From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 21:22:53 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FD1D10656A3; Thu, 16 Sep 2010 21:22:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C4EC58FC16; Thu, 16 Sep 2010 21:22:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o8GLGVCY008143; Thu, 16 Sep 2010 15:16:31 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 16 Sep 2010 15:16:40 -0600 (MDT) Message-Id: <20100916.151640.585585711625015566.imp@bsdimp.com> To: pjd@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <20100916203015.GA1967@garage.freebsd.pl> References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, mm@FreeBSD.org Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 21:22:53 -0000 In message: <20100916203015.GA1967@garage.freebsd.pl> Pawel Jakub Dawidek writes: : On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: : > Of course without sources. : > : > And more news, Netapp and Oracle have agreed to dismiss lawsuits. : > : > http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html : > : > The question is, why is Solaris 10 U9 without deduplication? If this has : > something to do with the agreement, one possibility is that they agreed : > deduplication may only be in Oracle Storage Appliances and Oracle will : > honor their patent. : > : > That would mean for us: no deduplication, or anybody using or selling it : > without paying a royalty to netapp can go to court : > : > But that is just a speculation as there is no more information available. : > : > Anyway, it is better to be cautious than to end like Coraid. : : My understanding is very different. I was under impression that NetApp : cannot go after people using OpenSolaris until they win in court. Before : that every OpenSolaris code consumer is protected by CDDL. NetApp didn't : win, right? : : Also: : : "both parties have agreed to dismiss their pending patent litigation" : : My English might not be enough here, but my reading is that because patent : litigation was dismissed, NetApp no longer claims (in legal sense) that : Oracle violates NetApp's patents via OpenSolaris. Is my reading correct : or not? Not necessarily. My reading of it was that they had some cross licensing agreement, and agreed to dismiss their claims without prejudice (meaning the patents weren't held invalid, among other things). Warner From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 07:10:26 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 356821065673; Fri, 17 Sep 2010 07:10:26 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id D285F8FC23; Fri, 17 Sep 2010 07:10:25 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2DB1145C9C; Fri, 17 Sep 2010 09:10:24 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 2E7B245C89; Fri, 17 Sep 2010 09:10:19 +0200 (CEST) Date: Fri, 17 Sep 2010 09:10:02 +0200 From: Pawel Jakub Dawidek To: "M. Warner Losh" Message-ID: <20100917071002.GB1967@garage.freebsd.pl> References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> <20100916.151640.585585711625015566.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: <20100916.151640.585585711625015566.imp@bsdimp.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, mm@FreeBSD.org Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 07:10:26 -0000 --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 16, 2010 at 03:16:40PM -0600, M. Warner Losh wrote: > : My understanding is very different. I was under impression that NetApp > : cannot go after people using OpenSolaris until they win in court. Before > : that every OpenSolaris code consumer is protected by CDDL. NetApp didn't > : win, right? > :=20 > : Also: > :=20 > : "both parties have agreed to dismiss their pending patent litigation" > :=20 > : My English might not be enough here, but my reading is that because pat= ent > : litigation was dismissed, NetApp no longer claims (in legal sense) that > : Oracle violates NetApp's patents via OpenSolaris. Is my reading correct > : or not? >=20 > Not necessarily. My reading of it was that they had some cross > licensing agreement, and agreed to dismiss their claims without > prejudice (meaning the patents weren't held invalid, among other > things). The patents are still valid of course, but NetApp didn't prove in court that Oracle violates them with OpenSolaris, so CDDL still protects code consumers, no? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkyTFEkACgkQForvXbEpPzTCXQCfRsNOU2ejiiD0lRoPsQdwSfxD XdgAnR1J/hfelzrx7Zp9NheIiOhNRnCI =7oUV -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6-- From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 10:02:16 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBDC6106566C; Fri, 17 Sep 2010 10:02:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 510058FC0C; Fri, 17 Sep 2010 10:02:16 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 7DA1A11911B; Fri, 17 Sep 2010 12:02:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SFqoSxjktPoc; Fri, 17 Sep 2010 12:02:00 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id B410F119105; Fri, 17 Sep 2010 12:01:59 +0200 (CEST) Message-ID: <4C933C99.9030802@FreeBSD.org> Date: Fri, 17 Sep 2010 12:02:01 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: ZFS patch proposal: offlining log devices X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 10:02:16 -0000 The handling of log devices prior to pool version 19 is quite problematic. They cannot be removed nor offlined. I want to propose a patch against head and later MFC that imports the following OpenSolaris revision, that enables offlining of log devices: 9701:cc5b64682e64 6803605 should be able to offline log devices 6726045 vdev_deflate_ratio is not set when offlining a log device 6599442 zpool import has faults in the display http://mfsbsd.vx.sk/zfs/head-9701.patch I did successfully test the following: - offlining a log device (ZIL is written to disk and device offlined) - importing the pool without offlined log devices (UNAVAIL with saved zpool.cache) Next I would also like to propose this additional patch: http://mfsbsd.vx.sk/zfs/head-9725.patch 9725:0bf7402e8022 6843014 ZFS B_FAILFAST handling is broken Both patches reduce diff against v28. Thank you, mm From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 17:57:23 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 104C41065695 for ; Fri, 17 Sep 2010 17:57:23 +0000 (UTC) (envelope-from 2010@joshcarter.com) Received: from joshcarter.com (67-207-137-80.slicehost.net [67.207.137.80]) by mx1.freebsd.org (Postfix) with ESMTP id E329C8FC08 for ; Fri, 17 Sep 2010 17:57:22 +0000 (UTC) Received: from [192.168.4.126] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) by joshcarter.com (Postfix) with ESMTPSA id 315F4128040 for ; Fri, 17 Sep 2010 17:38:02 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1081) From: Josh Carter <2010@joshcarter.com> In-Reply-To: <20100917071002.GB1967@garage.freebsd.pl> Date: Fri, 17 Sep 2010 11:37:56 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> <20100916.151640.585585711625015566.imp@bsdimp.com> <20100917071002.GB1967@garage.freebsd.pl> To: zfs-devel@FreeBSD.org X-Mailer: Apple Mail (2.1081) Cc: Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 17:57:23 -0000 Hi all, We've been watching the NetApp/Oracle case at Spectra Logic with great = interest, and our conclusions: - The settlement is a "good thing" simply because things are moving = along. - The terms of the settlement are confidential so far. For us, this = could mean anything. The deal could be exclusively between Oracle and = NetApp, therefore other companies are left to fend for themselves. The = deal could extend to OpenSolaris, which would be far better. We need to = wait and see. - CDDL protects us only from Oracle. It gives us license to *their* = relevant patents. It does not indemnify us from prosecution by other = companies who claim infringement. Summary: wait and see. We should hear something more about it soon. In = the meantime, Coraid and other companies are still just as exposed to = NetApp as before. Best regards, Josh From owner-zfs-devel@FreeBSD.ORG Tue Sep 21 18:41:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DEF1106564A for ; Tue, 21 Sep 2010 18:41:40 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 93D4E8FC12 for ; Tue, 21 Sep 2010 18:41:39 +0000 (UTC) Received: by eyx24 with SMTP id 24so2580477eyx.13 for ; Tue, 21 Sep 2010 11:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=5jCo6JrgzBAd651U2BYOl1cg4gUiWPDHnOH/DZhxs7E=; b=NUxJiqdkhvAPKApMQTDpINuOsGsoGZeD1tLr0gKtThVapm3izTqRWMWysGgwFO+Gbz kWEbvqO/B/VTi5hcHg9C2mgaRLvFWlRmCtgaLSJbYH2noGc554lz7PjiW5uJZAGZYCbd x87CfrxFyPWcX1j16Mr74qPp0G2200lzf/o2E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=AdCyTq+YiBexOFqHPNk4mdRCBjBGzUY8a1u+N7T/94uEFy157L9tLiraxSiDbU6QU4 yQfU1LYPGFJ8tJzzPkRuBjKG+fA4y68/FieWvro5CvV3Kq/zEg/zq42lCwuTnrZZy2sW s/xnz3sXDnWNvBzUNFKgGeiSnvcfUpOzez4HY= MIME-Version: 1.0 Received: by 10.213.36.3 with SMTP id r3mr4462997ebd.88.1285093125064; Tue, 21 Sep 2010 11:18:45 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.14.119.78 with HTTP; Tue, 21 Sep 2010 11:18:44 -0700 (PDT) Date: Tue, 21 Sep 2010 11:18:44 -0700 X-Google-Sender-Auth: uRjrg5a_mtw1eAOSxGVInc6SnZY Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: notes from today's call X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2010 18:41:40 -0000 participants : samir, ganesh, xin, steve, tushar date : 9/21/10 brief action items from today's call :- ganesh / samir will send out details of test suite location & instructions for running the same. xin will try the new test suite on iXSystems h/w ganesh / samir will send out performance report on iozone based synchronous write performance testing that they did recently. xin will share his results on zfs performance testing so far and also run the iozone tests to confirm results. -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 10:43:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D558A106564A for ; Thu, 23 Sep 2010 10:43:36 +0000 (UTC) (envelope-from ganesh.varadarajan@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 735468FC17 for ; Thu, 23 Sep 2010 10:43:35 +0000 (UTC) Received: by qyk4 with SMTP id 4so2328637qyk.13 for ; Thu, 23 Sep 2010 03:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=357MCLVcl6PZeZE5Xvan++41/5y+f8nlgfkHF5ae4MM=; b=tFAF/CPxUbNq1xpe5XbrvrFFgFnmRI/R7R6+ARDDQK9XAOWG9Lvm80Iz37FTw1v1q6 eDSDVJLbOSLVoDRbBmXr8VMOSTrFn6F1WKMdfG5ABnoXmWZwy6nn96AGmJSQNx+578o0 blpoe8iGW0Yl5IwakXrNa/NrFRtiTtRK1nm/I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=FW/uKZtcv9eIlPWlikVbAxaYxJyQSxW92TPkIIT0At4wF6B97VaFAjdQ16cwojUO02 Cj0U3cnjWx6Y3cfGz75AEOoNxIm8cLwj66ft57UZofO7zWDV67wrVZ2NcSysgA5a4I3V Q3xkX0i/t1B+pckISQ/M6Z3i1YV5iz6r8Do7c= MIME-Version: 1.0 Received: by 10.224.72.2 with SMTP id k2mr1111456qaj.242.1285237103298; Thu, 23 Sep 2010 03:18:23 -0700 (PDT) Received: by 10.229.221.12 with HTTP; Thu, 23 Sep 2010 03:18:23 -0700 (PDT) Date: Thu, 23 Sep 2010 15:48:23 +0530 Message-ID: From: Ganesh Varadarajan To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS performance notes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 10:43:36 -0000 [This is extracted from a series of mails Samir and I wrote while investigating ZFS performance on FreeBSD. Version doesn't matter; we used both v14 and v25 with largely similar results.] We've finally got some understanding of ZFS performance in the sync-write case (important for NFS and file server loads) and why it's as low as it is. Bottom line: It's not ZFS's fault. We need a fast NVRAM-type log device if we want performance. This is the iostat output for an 8k sync write load with iozone on a ZFS filesystem (FreeBSD 8.1, local, no NFS, pool consisting of just one 7200rpm SATA disk) iozone -s 100m -r 8 -i 0 -o -f /tp/fs/f1 (gives 935 KB/sec throughput) tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 12.00 118 1.39 0.00 0 0.00 0 0 0 0 100 0 39 0.00 0 0.00 12.00 117 1.38 0.00 0 0.00 0 0 0 0 100 0 39 0.00 0 0.00 69.32 220 14.88 0.00 0 0.00 0 0 0 0 99 0 39 0.00 0 0.00 12.00 118 1.38 0.00 0 0.00 0 0 1 0 99 0 39 0.00 0 0.00 12.00 118 1.39 0.00 0 0.00 0 0 0 0 100 Several interesting things here. Note that each transaction is 12K. This is as we expect, each 8k write is padded with 4k of ZIL stuff. We're able to to 118 transactions per sec (aka IOPS). The steady state throughput on disk is 1.39 MB/sec, correlating well to the 935 KB/sec observed by the application after discounting ZIL metadata overhead. The spike of 14MB/sec corresponds to a flush of log data to primary fs. Removing the sync option, we try iozone -s 1000m -r 8 -i 0 -f /tp/fs/f1 and see that ZFS can generate quite a storm of IO. tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 8.38 2 0.02 128.00 751 93.82 0.00 0 0.00 0 0 13 1 86 0 39 0.00 0 0.00 128.00 1020 127.56 0.00 0 0.00 0 0 1 1 98 0 40 0.00 0 0.00 128.00 1015 126.92 0.00 0 0.00 0 0 2 1 97 0 40 0.00 0 0.00 124.94 702 85.66 0.00 0 0.00 0 0 14 0 85 Now, the question is: if ZFS can generate 120MB/sec sustained throughput to the disk, why is the ZIL throttled so low? And the answer is... On-disk write cache. By default, FreeBSD enables on-disk cache. That's why iozone to a raw disk shows very large numbers: iozone -s 1000m -r 8 -i 0 -o -f /dev/ad10 (gives 80 MB/sec) tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 0.00 0 0.00 8.00 10031 78.36 0 0 9 6 86 0 40 0.00 0 0.00 0.00 0 0.00 8.00 10019 78.28 0 0 9 6 84 0 40 0.00 0 0.00 0.00 0 0.00 8.00 10040 78.43 0 0 9 6 85 Note that we're doing 8k IOs as expected, but we're completing 10,000 of them in a second (10,000 IOPS!) Clearly, this is because the disk is caching them. They're not going to the platter, and might very well disappear if there is a power failure. Default UFS on such a disk therefore exhibits good performance. At higher IOsizes (64-128K), we max out at 120MB/sec - not bad for a single 7200 RPM 500Gig SATA disk, what? Random reads of 8k on raw disk give ~1.4MB/sec (since nothing can be cached) and this is a real reflection on platter performance. Notice that this correlates well with the ZFS sync-write numbers we got first. Random writes are higher (~3MB/sec), but nowhere near the max rate, reflecting that the disk cache can't absorb these as well as writes coming in a sequence. Now if we turn FreeBSD write cache off, (add a line hw.ata.wc = 0 to /boot/loader.conf and reboot), then we see, for sync IO to a raw device, iozone -s 10m -r 12 -i 0 -o -f /dev/ad10, a throughput of 1.4MB/sec, exactly the same as the first ZFS case. tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 0.00 0 0.00 12.00 119 1.39 0 0 0 0 100 0 39 0.00 0 0.00 0.00 0 0.00 12.00 118 1.39 0 0 0 0 100 0 39 0.00 0 0.00 0.00 0 0.00 12.00 119 1.40 0 0 0 0 100 I chose 12k to correspond to the 8k data +4k ZIL overhead workload. We see that IOPS drops to ~120 from ~10,000 when we turn off the disk cache. UFS does *worse* than ZFS if we turn off the write-cache. iozone -s 10m -r 8 -i 0 -o 0 39 0.00 0 0.00 0.00 0 0.00 16.00 145 2.27 0 0 0 0 100 It manages to generate 2.2 MB/sec throughput, but notice that it turned every 8k app write to a 16k write. The actual throughput as seen by iozone is 495K/sec, a fourth of what's going to disk. This is because of the major inefficiencies in UFS sync writes (8k->16k, metadata updates) as discussed earlier. When we do async writes (also with write-cache off), it goes upto a throughput of ~14MB/sec. iozone -s 100m -r 8 -i 0 0 40 0.00 0 0.00 0.00 0 0.00 128.00 107 13.43 0 0 0 0 100 0 40 0.00 0 0.00 0.00 0 0.00 128.00 108 13.56 0 0 0 0 100 Note that UFS gathers writes into 128k chunks, but we're band-limited by IOPs here. The platters can't do more than 110-120 IOPS (when the cache is useless - either turned off or due to random load) With write-cache turned off, ZFS async writes give exactly the same performance as UFS - ~14MB/sec, 108 IOPS, 128k IO sizes. Sync writes are the same old 1.3MB/sec. So, ZFS is actually being pretty smart. In the normal case, when disk write-cache is turned on, it uses the cache for all regular writes, but when it comes to ZIL writes, it makes sure that the bytes hit the platter. So the ZIL, although in theory sequential, actually degenerates to close to random write performance, because we cannot gather the IO at any stage. 8k hits the platter, we go back to the application, which spits out 8k more. Meanwhile, the disk has spun so the second IO has to wait for the planets to align again. There is a ZFS tunable which says "don't force a platter write for ZIL, treat it as normal IO". Set vfs.zfs.cache_flush_disable="1" in /boot/loader.conf and reboot Now you will see the ZIL unfettered by the ~100 IOPS limit iozone -s 1000m -r 8 -i 0 -o -f /tp/fs/f1 (gives ~40MB/sec throughput, up from 1.3!) 0 39 0.00 0 0.00 12.00 5530 64.80 0.00 0 0.00 0 0 13 3 84 0 39 0.00 0 0.00 12.00 5506 64.53 0.00 0 0.00 1 0 11 2 86 Note that we go to 5k IOPS and 64MB/sec ZIL throughput. To summarize so far: 1. If we turn off the disk write-cache, we are IOPS-limited. 128k * 100-120 IOPS is the best we can hope for under any circumstances per disk of this type. 2. ZFS does the sane thing, using disk cache when possible and avoiding it when semantics demand. In our case that boils down to ZIL and platter writes every time, since our NFS-client issues sync writes. 3. If we want to honour NFS sync write semantics (and we must), without getting bad performance, 1. some kind of NVRAM based log accelerator is needed. 2. large number of spindles in mirror-stripe config (untested) All this has been for single-threaded loads. Now let's look into logbias and scaling with multi-threaded loads. Multi-threading does break through the 100-120 IOPS sound barrier for Real Sync Writes (TM) if and only if disk write cache is enabled. We can go upto 300 IOPS, delivering an aggregate of 27MB/sec for 8k, 64 threads. Per-thread throughput remains low, at around 500K/sec. 1. How does this happen? As we saw earlier, ZFS performs best when the write cache is enabled. In such cases, 1. ZIL writes are done to the log device, going to the disks write-cache 2. then a cache flush is issued 3. On successful flush (bits have hit platter), return success to the caller. 2. If another ZIL write happens between 1.1 and 1.2 above (multi-threading case) then one cache flush does for both. This is what enables us to break through the IOPS barrier without compromising correctness. 3. If we turn off the write-cache, then we simply cannot bust the 100 IOPS barrier, no matter how many threads we use. This serves to confirm the above. All tests have been with logbias=latency. If we change logbias=throughput, an FS-level option, ZFS avoids using the separate log device for this filesystem and dumps it directly to primary storage. This is used when there are multiple filesystems in a pool and a log device configured on an SSD. Instead of flooding the limited SSD with sync-writes from all filesystems, you may want to selectively do it for a few (logbias=latency) and not for others (logbias=throughput). Not much use for us. Multiple disks, no log device. Adding multiple disks to the pool does help; ZFS nicely distributes data, including sync-writes to all disks. Ramdisk (as a substitute for NVRAM) as log disk improved performance significantly, as expected, to about 40 MB/sec for a single-threaded 8k sync write. Ramdisks sometimes cause odd oscillations - the log gets full very fast, ZFS can't or doesn't drain the pent-up IO to primary fast enough, then ZFS switches to the primary disks for writing the log for several seconds. This results in sharp oscillations in throughput. Ramdisks also help a lot with RAID-Z performance. Everything gets staged through the log, thus helping to gather large-striped writes in memory. Rewrites do worse than writes when the IO size is less than the record size. Dedup has negligible effect on throughput when measured for single-threaded loads. Apparently performance falls off a cliff when the dedup table runs out of memory, but we didn't push it that far. Tentative recommendations: 1. Use a dedicated NVRAM device for the log. Size should be at least 1 GB, ideally 2-4 GB. Random write latencies for sizes from 8k-64k to NVRAM should be good. 2. Since all writes are sync, increasing RAM size beyond 2x of NVRAM size won't help much. So 8 GB RAM should be enough for a typical 2-4 GB NVRAM case. We expect reads to be cached at the client. 3. For dedup, many people recommend keeping a flash device which can do fast random-access reads to keep the dedup table. This avoids the performance cliff for typical storage and record sizes without requiring extra RAM. 4. Mirroring is preferred to RAID-Z for raw performance. RAID-Z ain't bad though, if we have an NVRAM log. 5. Set recordsize to 32k - NFS write ops don't exceed this size (this needs more study) 6. Leave logbias to its default value, 'latency'. More accurate measurements and sizing recommendations can be made if we can get our hands on a "typical" customer system, with 6 or 12 spindles and an NVRAM device. From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 12:41:35 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92CBF1065674 for ; Thu, 23 Sep 2010 12:41:35 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 511378FC0A for ; Thu, 23 Sep 2010 12:41:35 +0000 (UTC) Received: by mail-gx0-f182.google.com with SMTP id 8so650615gxk.13 for ; Thu, 23 Sep 2010 05:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=leZyiLURAusLBruyhjNOKWU7QnDlWkJqkFjPwDQDCmY=; b=dmwMjYp4hzmcQxKybCe1DG7wXD4RVbL3cd3wh2ofxfgfg8cFLNWQhTQO0cDsQYh0Xn O5J25+9cZrHDlUfmttUsoTdpVkN7pcvNR+9smwHjOZsg1owPFIT2A4ltxz/oDQxJJRNr Ynw5vClFbBR8BXXCd03rNYBHu19ooCxyG8TA8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=MuAPEj0NFIVQJhKMlwVqaYU+iDeGYA6SCoqO2Xiv6fuufvyDovDbU7QsfIdlFYw0N1 TTqpiRtYqugcyJFlGjlmza3eNU8e6ZxvZlPSoi837EPAFxWwN3TeWlu/U2u2iLTbi8wD lYlFH1vSBcAA110ipo09abpVifJSvzU/jYpOA= MIME-Version: 1.0 Received: by 10.151.6.5 with SMTP id j5mr2726049ybi.37.1285245206255; Thu, 23 Sep 2010 05:33:26 -0700 (PDT) Received: by 10.220.195.68 with HTTP; Thu, 23 Sep 2010 05:33:23 -0700 (PDT) Date: Thu, 23 Sep 2010 18:03:23 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs test suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 12:41:35 -0000 hi We have checked in all the code and test suite in to perforce the branch is //depot/user/samir/zfs, this is a branch of //depot/projects/zfs/stable/8 Once you checkout, go to the zfs directory # ls zfs_stable_8/zfs README build-test.sh build.sh patch-kernel.sh please read README here, it has all the information necessary to setup the freebsd system, building and installing zfs kernel/command and building/running zfs test suite regards Samir From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 12:58:19 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22413106564A for ; Thu, 23 Sep 2010 12:58:19 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id D1ECA8FC12 for ; Thu, 23 Sep 2010 12:58:18 +0000 (UTC) Received: by gyg4 with SMTP id 4so666953gyg.13 for ; Thu, 23 Sep 2010 05:58:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=Ol6SQ0yxMCyUaUfDJNW88PiN/ZtDNzIBcvy2ssy0X/s=; b=pTjRngsDqox1RKbw4720mJnqsqh2LDT0SsZF4zV+lJIye76tH+XU7R/ajEOzabi6T9 rx7ACEz39PFrtQXb2vvpZoxfSkJ5oZmCY6EPJ0NcUmLkTBertJHO0ZcUCnCFXWknrIak EklngIx8mlX+sQpJLuatF/Ga72eWQlf0at1L8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=B/Hodl+fIwchwiffYV/3+Wb0eKoln1v0hkmtr2xmvHcpcUA5AcEBJodeOcqPiZs5Ug 0gGBz/B+40olkjdktdE2vhY6mIgFboilXvt3iH+i5IeTye7VhRqqqz3CkE15kAhx6o9y lsEGpWdQ1xomfOPH4kRi9WDYUdB03J7UPcvow= MIME-Version: 1.0 Received: by 10.220.58.129 with SMTP id g1mr705706vch.218.1285244945073; Thu, 23 Sep 2010 05:29:05 -0700 (PDT) Received: by 10.220.195.68 with HTTP; Thu, 23 Sep 2010 05:28:53 -0700 (PDT) Date: Thu, 23 Sep 2010 17:58:53 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs test suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 12:58:19 -0000 hi We have checked in all the code and test suite in to perforce the branch is //depot/user/samir/zfs, this is a branch of //depot/projects/zfs/stable/8 Once you checkout, go to the zfs directory # ls zfs_stable_8/zfs README build-test.sh build.sh patch-kernel.sh please read README here, it has all the information necessary to setup the freebsd system, building and installing zfs kernel/command and building/running zfs test suite regards Samir From owner-zfs-devel@FreeBSD.ORG Wed Sep 29 09:56:37 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA6E8106566C; Wed, 29 Sep 2010 09:56:37 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D03798FC12; Wed, 29 Sep 2010 09:56:36 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA23405; Wed, 29 Sep 2010 12:56:35 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P0tOt-0003L1-Bz; Wed, 29 Sep 2010 12:56:35 +0300 Message-ID: <4CA30D53.5080706@freebsd.org> Date: Wed, 29 Sep 2010 12:56:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: adjust kmem_size for large vm_kmem_size X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 09:56:38 -0000 We use kmem_size() function in ZFS ARC sizing code to determine values like arc_c_min and arc_c_mac. kmem_size() currently returns vm_kmem_size. OpenSolaris uses 'physmem' value for the same purpose. I guess that we use vm_kmem_size because typically it is (much) smaller than amount of physical memory. On the other hand, it is possible to set vm_kmem_size to value larger than physical memory (up to 2x is allowed). Sometimes it is even advisable to do so to work around kernel KVA space fragmentation. I think that we should use smaller of the two parameters when deciding ARC size. The patch below tries to implement that logic. What do you think? --- a/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c @@ -115,8 +115,14 @@ zfs_kmem_free(void *buf, size_t size __unused) uint64_t kmem_size(void) { + static uint64_t kmem_size; - return ((uint64_t)vm_kmem_size); + if (kmem_size == 0) { + kmem_size = (uint64_t)cnt.v_page_count * PAGE_SIZE; + if (kmem_size > vm_kmem_size) + kmem_size = vm_kmem_size; + } + return (kmem_size); } uint64_t Perhaps, this should be split into two functions: one that sets kmem_size at appropriate moment and one that simply returns its value. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 17:30:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA3BB106567A; Thu, 7 Oct 2010 17:30:36 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C587C8FC16; Thu, 7 Oct 2010 17:30:35 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18206; Thu, 07 Oct 2010 20:30:34 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE03B9.1060504@freebsd.org> Date: Thu, 07 Oct 2010 20:30:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org References: <4CA30D53.5080706@freebsd.org> In-Reply-To: <4CA30D53.5080706@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit Cc: Subject: Re: adjust kmem_size for large vm_kmem_size X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:30:36 -0000 on 29/09/2010 12:56 Andriy Gapon said the following: > > We use kmem_size() function in ZFS ARC sizing code to determine values like > arc_c_min and arc_c_mac. kmem_size() currently returns vm_kmem_size. > OpenSolaris uses 'physmem' value for the same purpose. > > I guess that we use vm_kmem_size because typically it is (much) smaller than > amount of physical memory. > On the other hand, it is possible to set vm_kmem_size to value larger than > physical memory (up to 2x is allowed). Sometimes it is even advisable to do so > to work around kernel KVA space fragmentation. > > I think that we should use smaller of the two parameters when deciding ARC size. > The patch below tries to implement that logic. [snip] > Perhaps, this should be split into two functions: one that sets kmem_size at > appropriate moment and one that simply returns its value. Updated patch: http://people.freebsd.org/~avg/osol-kmem_size.diff I am going to commit it if no one objects. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 17:33:14 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D0A9106564A for ; Thu, 7 Oct 2010 17:33:14 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AA65A8FC08 for ; Thu, 7 Oct 2010 17:33:12 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18456; Thu, 07 Oct 2010 20:33:11 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE0457.8040900@freebsd.org> Date: Thu, 07 Oct 2010 20:33:11 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org, hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:33:14 -0000 A simple, probably incomplete and perhaps incorrect patch for ru_inblock/ru_oublock accounting in zfs: http://people.freebsd.org/~avg/zfs-ru.diff Still quite better than nothing. P.S. This is about top -m io. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 18:05:36 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2D1D106566B for ; Thu, 7 Oct 2010 18:05:36 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 6021E8FC15 for ; Thu, 7 Oct 2010 18:05:32 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8866045CAC; Thu, 7 Oct 2010 20:05:31 +0200 (CEST) Received: from localhost (chello089073192049.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 04F5945C98 for ; Thu, 7 Oct 2010 20:05:25 +0200 (CEST) Date: Thu, 7 Oct 2010 20:05:01 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20101007180501.GA1733@garage.freebsd.pl> References: <4CAE0457.8040900@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <4CAE0457.8040900@freebsd.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=4.5 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: Subject: Re: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 18:05:36 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 07, 2010 at 08:33:11PM +0300, Andriy Gapon wrote: [...] Picking up random e-mail, I hope Andriy doesn't mind. Please do not cross-post to zfs-devel@ and public mailing lists, so I don't have to moderate incoming responses, as zfs-devel@ is invitation-only;) BCCing zfs-devel@ is probably the best choice. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkyuC80ACgkQForvXbEpPzQo5gCgl6Ptzvq06xRlK6PNTsUrm2N/ pH8AoLIzSSx8vcCo/GBhrJT5gHM41a8Q =JtWu -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-zfs-devel@FreeBSD.ORG Fri Oct 8 08:23:31 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0F811065672; Fri, 8 Oct 2010 08:23:31 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BA0FA8FC2B; Fri, 8 Oct 2010 08:23:30 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA01056; Fri, 08 Oct 2010 11:23:29 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P48Ej-000AFi-8D; Fri, 08 Oct 2010 11:23:29 +0300 Message-ID: <4CAED500.3090001@freebsd.org> Date: Fri, 08 Oct 2010 11:23:28 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: hackers@freebsd.org References: <4CAE0457.8040900@freebsd.org> In-Reply-To: <4CAE0457.8040900@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 08 Oct 2010 11:19:28 +0000 Cc: Subject: Re: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 08:23:31 -0000 on 07/10/2010 20:33 Andriy Gapon said the following: > > A simple, probably incomplete and perhaps incorrect patch for > ru_inblock/ru_oublock accounting in zfs: > http://people.freebsd.org/~avg/zfs-ru.diff I've updated the patch at the same location. Thanks to "swell.k" for pointing out that the changes are applicable in kernel only. > Still quite better than nothing. > > P.S. > This is about top -m io. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Oct 10 08:38:18 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F31C106564A for ; Sun, 10 Oct 2010 08:38:18 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D0C898FC08 for ; Sun, 10 Oct 2010 08:38:17 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA05547 for ; Sun, 10 Oct 2010 11:38:16 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P4rQ8-000HTZ-5M for zfs-devel@freebsd.org; Sun, 10 Oct 2010 11:38:16 +0300 Message-ID: <4CB17B77.7070509@freebsd.org> Date: Sun, 10 Oct 2010 11:38:15 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Subject: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Oct 2010 08:38:18 -0000 Guys, could you please review and or test the following addition to FreeBSD ZFS? http://people.freebsd.org/~avg/zfs-getpages.diff Does it improve anything for you? Is it something worth committing? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 10:34:54 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 149641065670; Mon, 11 Oct 2010 10:34:54 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C89FB8FC0C; Mon, 11 Oct 2010 10:34:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 3233512A553; Mon, 11 Oct 2010 12:34:53 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 9WqyumyIgZY7; Mon, 11 Oct 2010 12:34:51 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 67E1012A547; Mon, 11 Oct 2010 12:34:51 +0200 (CEST) Message-ID: <4CB2E84A.2000008@FreeBSD.org> Date: Mon, 11 Oct 2010 12:34:50 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> In-Reply-To: <4CB17B77.7070509@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 10:34:54 -0000 As of this code: + for (i = 0; i < pcount; i++) { + if (i != reqpage) { + vm_page_lock(m[i]); + vm_page_free(m[i]); + vm_page_unlock(m[i]); + } + } Do you think locking a page is sufficient to protect against a race? As to our reference implementation it should be more like this: + vm_page_lock_queues(); + for (i = 0; i < pcount; i++) { + if (i != reqpage) { + vm_page_free(m[i]); + } + } + vm_page_unlock_queues(); Cheers, mm Dňa 10. 10. 2010 10:38, Andriy Gapon wrote / napísal(a): > > Guys, > > could you please review and or test the following addition to FreeBSD ZFS? > http://people.freebsd.org/~avg/zfs-getpages.diff > > Does it improve anything for you? > Is it something worth committing? > From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:03:47 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B140A10656A3; Mon, 11 Oct 2010 11:03:47 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C8B378FC25; Mon, 11 Oct 2010 11:03:46 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA21822; Mon, 11 Oct 2010 14:03:45 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P5GAT-000KaJ-6G; Mon, 11 Oct 2010 14:03:45 +0300 Message-ID: <4CB2EF10.50601@freebsd.org> Date: Mon, 11 Oct 2010 14:03:44 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Martin Matuska References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> In-Reply-To: <4CB2E84A.2000008@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:03:47 -0000 on 11/10/2010 13:34 Martin Matuska said the following: > As of this code: > + for (i = 0; i < pcount; i++) { > + if (i != reqpage) { > + vm_page_lock(m[i]); > + vm_page_free(m[i]); > + vm_page_unlock(m[i]); > + } > + } > > Do you think locking a page is sufficient to protect against a race? What race? > As to our reference implementation it should be more like this: Which reference implementation would that be? > + vm_page_lock_queues(); > + for (i = 0; i < pcount; i++) { > + if (i != reqpage) { > + vm_page_free(m[i]); > + } > + } > + vm_page_unlock_queues(); I think that in head only page lock is required for vm_page_free(). Locking queues around the loop should not be required, it is done internally only for a short duration when it's needed. > Dňa 10. 10. 2010 10:38, Andriy Gapon wrote / napísal(a): >> >> Guys, >> >> could you please review and or test the following addition to FreeBSD ZFS? >> http://people.freebsd.org/~avg/zfs-getpages.diff >> >> Does it improve anything for you? >> Is it something worth committing? >> > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:25:49 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BB3C1065674; Mon, 11 Oct 2010 11:25:49 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 4B14C8FC1C; Mon, 11 Oct 2010 11:25:49 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A8AAF12B95A; Mon, 11 Oct 2010 13:25:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QHN29E1X5moo; Mon, 11 Oct 2010 13:25:47 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id D8A6012B94E; Mon, 11 Oct 2010 13:25:46 +0200 (CEST) Message-ID: <4CB2F43A.7050503@FreeBSD.org> Date: Mon, 11 Oct 2010 13:25:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> In-Reply-To: <4CB2EF10.50601@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:25:49 -0000 > Which reference implementation would that be? vop_stdgetpages() kern/vfs_default.c utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c We have four filesystems that implement their own vop_getpages and all use the queue locking: sys/fs/nfsclient/nfs_clbio.c sys/fs/nwfs/nwfs_io.c sys/fs/tmpfs/tmpfs_vnops.c sys/fs/smbfs/smbfs_io.c From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:33:02 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0881410656A8; Mon, 11 Oct 2010 11:33:02 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 240BD8FC13; Mon, 11 Oct 2010 11:33:00 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA22806; Mon, 11 Oct 2010 14:32:59 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P5Gcl-000Kc3-EU; Mon, 11 Oct 2010 14:32:59 +0300 Message-ID: <4CB2F5EA.2020908@freebsd.org> Date: Mon, 11 Oct 2010 14:32:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Martin Matuska References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> <4CB2F43A.7050503@FreeBSD.org> In-Reply-To: <4CB2F43A.7050503@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:33:02 -0000 on 11/10/2010 14:25 Martin Matuska said the following: >> Which reference implementation would that be? > vop_stdgetpages() kern/vfs_default.c > utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c > > We have four filesystems that implement their own vop_getpages and all > use the queue locking: I am sorry, but I don't see that in my copy of FreeBSD source tree, head branch. > sys/fs/nfsclient/nfs_clbio.c > sys/fs/nwfs/nwfs_io.c > sys/fs/tmpfs/tmpfs_vnops.c > sys/fs/smbfs/smbfs_io.c -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:53:18 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40669106567A; Mon, 11 Oct 2010 11:53:18 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id F26BD8FC2A; Mon, 11 Oct 2010 11:53:17 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 51B9B1194E5; Mon, 11 Oct 2010 13:53:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id iN4PfNpbZci9; Mon, 11 Oct 2010 13:53:15 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 77CA41194D7; Mon, 11 Oct 2010 13:53:15 +0200 (CEST) Message-ID: <4CB2FAAA.6060000@FreeBSD.org> Date: Mon, 11 Oct 2010 13:53:14 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> <4CB2F43A.7050503@FreeBSD.org> <4CB2F5EA.2020908@freebsd.org> In-Reply-To: <4CB2F5EA.2020908@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:53:18 -0000 I apologize, need probably more sleep after EuroBSDcon. I had a checkout of 8-STABLE into a wrong directory. Thx. Dňa 11. 10. 2010 13:32, Andriy Gapon wrote / napísal(a): > on 11/10/2010 14:25 Martin Matuska said the following: >>> Which reference implementation would that be? >> vop_stdgetpages() kern/vfs_default.c >> utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c >> >> We have four filesystems that implement their own vop_getpages and all >> use the queue locking: > > I am sorry, but I don't see that in my copy of FreeBSD source tree, head branch. > >> sys/fs/nfsclient/nfs_clbio.c >> sys/fs/nwfs/nwfs_io.c >> sys/fs/tmpfs/tmpfs_vnops.c >> sys/fs/smbfs/smbfs_io.c > > From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 20:29:00 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24C87106566C for ; Mon, 11 Oct 2010 20:29:00 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 3DBA98FC08 for ; Mon, 11 Oct 2010 20:28:58 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 3FAD1A6A59D; Tue, 12 Oct 2010 04:28:52 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id aSgbba03TEHZ; Tue, 12 Oct 2010 04:28:45 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id A14A2A6A297; Tue, 12 Oct 2010 04:28:42 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type; b=AeNHZdwhGghaooYrOfaeq8TyHRFjnOE4eUjghCtAjTMwFOEChtb4s3JoB12Zv36CW sYpbjmVPKYmlwTO+RNztA== Message-ID: <4CB37377.9060202@delphij.net> Date: Mon, 11 Oct 2010 13:28:39 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.12) Gecko/20100920 Thunderbird/3.0.8 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------080904090108030003080008" Cc: Subject: ZFS deadlock on 9-CURRENT with NFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 20:29:00 -0000 This is a multi-part message in MIME format. --------------080904090108030003080008 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, This is a user's report, a first glance shows that 'nfsd' stuck in 'zfs', which happens from time to time. He was uploading a big file to the server, which might be related. We are still trying to find out a way to provoke the issue more reliably. The ZFS pool have some data errors on it. I have grabbed the output of procstat -k output from his system when it happens, just FYI. The user is willing to help so please let me know if you need more data. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMs3N2AAoJEATO+BI/yjfBOxQH/jLaRVADGIZpt3Qcp3hvgMIH 7yAbNAsIfUmVktyw4P5/o7EHnXk+o0avLgQMWYub/YoJDVeBL3h/QpjKEX6Z3fsb Zk2MvIrQhOT5ZZWBU9lJh/OSNtZRjCji8K130rriQcJLtAv00qem654i9w7tDHW5 sFVHnzcXu1e4g3cbuy0b8IFh4jBSpeOmsWRmXdM/cpDrGQqTtG/1tuqefBsAZj0g M4r3kZNflgpBIZtpFPepzVARvH9gc+k7V+ihjjcR0bXNEzP5X0eaSezbiQA6lydT BTi/coie8ODDOEkW2xO1ue//4k20xIkHADy0o0MoOZKP1JfydLUXL2sqzurxrps= =iDwb -----END PGP SIGNATURE----- --------------080904090108030003080008 Content-Type: text/plain; name="procstat.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="procstat.txt" IyBwcm9jc3RhdCAtayAyNjc1IDI0MzYgMTA4MSAxOCA1IDAKICBQSUQgICAgVElEIENPTU0g ICAgICAgICAgICAgVEROQU1FICAgICAgICAgICBLU1RBQ0sgICAgICAgICAgICAgICAgICAg ICAgIAogMjY3NSAxMDAyOTIgc2Z0cC1zZXJ2ZXIgICAgICAtICAgICAgICAgICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2FpdCB0eGdfd2FpdF9vcGVuIHpmc19mcmVlYnNk X3dyaXRlIFZPUF9XUklURV9BUFYgdm5fd3JpdGUgZG9maWxld3JpdGUga2Vybl93cml0ZXYg d3JpdGUgc3lzY2FsbGVudGVyIHN5c2NhbGwgWGZhc3Rfc3lzY2FsbCAKIDI0MzYgMTAwMzM3 IGN2c3VwICAgICAgICAgICAgLSAgICAgICAgICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX2N2X3dhaXQgemlvX3dhaXQgZGJ1Zl9yZWFkIGRtdV9idWZfaG9sZCB6YXBfbG9ja2Rp ciB6YXBfbG9va3VwX25vcm0gemFwX2xvb2t1cCB6ZnNfZGlyZW50X2xvY2sgemZzX2Rpcmxv b2sgemZzX2xvb2t1cCB6ZnNfZnJlZWJzZF9sb29rdXAgdmZzX2NhY2hlX2xvb2t1cCBWT1Bf TE9PS1VQX0FQViBsb29rdXAgbmFtZWkga2Vybl9zdGF0YXRfdm5ob29rIAogMTA4MSAxMDAy OTkgbmZzZCAgICAgICAgICAgICBuZnNkOiBtYXN0ZXIgICAgIG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfY3Zfd2FpdCB6aW9fd2FpdCBkYnVmX3JlYWQgZGJ1Zl9maW5kYnAgZGJ1Zl9ob2xk X2ltcGwgZGJ1Zl9ob2xkIGRtdV9idWZfaG9sZCB6YXBfbG9ja2RpciB6YXBfbG9va3VwX25v cm0gemFwX2xvb2t1cCB6ZnNfZGlyZW50X2xvY2sgemZzX2Rpcmxvb2sgemZzX2xvb2t1cCB6 ZnNfZnJlZWJzZF9sb29rdXAgdmZzX2NhY2hlX2xvb2t1cCBWT1BfTE9PS1VQX0FQViAKIDEw ODEgMTAwMzE4IG5mc2QgICAgICAgICAgICAgbmZzZDogc2VydmljZSAgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX19sb2NrbWdyX2FyZ3Mgdm9wX3N0ZGxvY2sgVk9QX0xPQ0sxX0FQViBf dm5fbG9jayB6ZnNfZmh0b3ZwIG5mc3Zub19maHRvdnAgbmZzZF9maHRvdnAgbmZzcnZkX2Rv cnBjIG5mc3N2Y19wcm9ncmFtIHN2Y19ydW5faW50ZXJuYWwgc3ZjX3RocmVhZF9zdGFydCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogMTA4MSAxMDAzMTkgbmZzZCAgICAgICAgICAg ICBuZnNkOiBzZXJ2aWNlICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfX2xvY2ttZ3JfYXJn cyB2b3Bfc3RkbG9jayBWT1BfTE9DSzFfQVBWIF92bl9sb2NrIHpmc19maHRvdnAgbmZzdm5v X2ZodG92cCBuZnNkX2ZodG92cCBuZnNydmRfZG9ycGMgbmZzc3ZjX3Byb2dyYW0gc3ZjX3J1 bl9pbnRlcm5hbCBzdmNfdGhyZWFkX3N0YXJ0IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAxMDgxIDEwMDMyMCBuZnNkICAgICAgICAgICAgIG5mc2Q6IHNlcnZpY2UgICAgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9fbG9ja21ncl9hcmdzIHZvcF9zdGRsb2NrIFZPUF9MT0NLMV9B UFYgX3ZuX2xvY2sgemZzX2ZodG92cCBuZnN2bm9fZmh0b3ZwIG5mc2RfZmh0b3ZwIG5mc3J2 ZF9kb3JwYyBuZnNzdmNfcHJvZ3JhbSBzdmNfcnVuX2ludGVybmFsIHN2Y190aHJlYWRfc3Rh cnQgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgMTggMTAwMDgxIHN5bmNlciAgICAg ICAgICAgLSAgICAgICAgICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX2N2X3dhaXQg emlvX3dhaXQgemlsX2NvbW1pdCB6ZnNfc3luYyBzeW5jX2ZzeW5jIHN5bmNfdm5vZGUgc2No ZWRfc3luYyBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgNSAxMDAwNzEgemZza2Vy biAgICAgICAgICBhcmNfcmVjbGFpbV90aHJlIG1pX3N3aXRjaCBzbGVlcHFfdGltZWR3YWl0 IF9jdl90aW1lZHdhaXQgYXJjX3JlY2xhaW1fdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICA1IDEwMDA3MiB6ZnNrZXJuICAgICAgICAgIGwyYXJjX2ZlZWRfdGhyZWEg bWlfc3dpdGNoIHNsZWVwcV90aW1lZHdhaXQgX2N2X3RpbWVkd2FpdCBsMmFyY19mZWVkX3Ro cmVhZCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgNSAxMDAxMjUgemZza2VybiAg ICAgICAgICB0eGdfdGhyZWFkX2VudGVyIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2Fp dCB0eGdfdGhyZWFkX3dhaXQgdHhnX3F1aWVzY2VfdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICA1IDEwMDEyNiB6ZnNrZXJuICAgICAgICAgIHR4Z190aHJlYWRfZW50 ZXIgbWlfc3dpdGNoIHNsZWVwcV90aW1lZHdhaXQgX2N2X3RpbWVkd2FpdCB0eGdfdGhyZWFk X3dhaXQgdHhnX3N5bmNfdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICA1 IDEwMDE4MiB6ZnNrZXJuICAgICAgICAgIHR4Z190aHJlYWRfZW50ZXIgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9jdl93YWl0IHR4Z19xdWllc2NlX3RocmVhZCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgNSAxMDAxODMgemZza2VybiAgICAgICAgICB0eGdfdGhyZWFkX2Vu dGVyIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2FpdCB6aW9fd2FpdCBkc2xfcG9vbF9z eW5jIHNwYV9zeW5jIHR4Z19zeW5jX3RocmVhZCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAwMDAga2VybmVsICAgICAgICAgICBzd2FwcGVyICAgICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfdGltZWR3YWl0IF9zbGVlcCBzY2hlZHVsZXIgbWlfc3RhcnR1cCBidGV4 dCAKICAgIDAgMTAwMDE2IGtlcm5lbCAgICAgICAgICAgZmlybXdhcmUgdGFza3EgICBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwMjEga2VybmVsICAgICAgICAgICBhY3Bp X3Rhc2tfMCAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBtc2xlZXBfc3BpbiB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDIy IGtlcm5lbCAgICAgICAgICAgYWNwaV90YXNrXzEgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgbXNsZWVwX3NwaW4gdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDAyMyBrZXJuZWwgICAgICAgICAgIGFjcGlfdGFza18yICAg ICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IG1zbGVlcF9zcGluIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwMjQga2VybmVsICAg ICAgICAgICBrcXVldWUgdGFza3EgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDAyNiBrZXJuZWwgICAgICAgICAgIHRocmVhZCB0YXNrcSAgICAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMDU5IGtlcm5lbCAgICAgICAgICAgZW0wIHRhc2txICAg ICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgbXNsZWVwX3NwaW4gdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA2NyBrZXJuZWwg ICAgICAgICAgIHN5c3RlbV90YXNrcV8wICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMDY4IGtlcm5lbCAgICAgICAgICAgc3lzdGVtX3Rhc2txXzEgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAwNjkga2VybmVsICAgICAgICAgICBzeXN0ZW1fdGFz a3FfMiAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA3MCBrZXJuZWwgICAg ICAgICAgIHN5c3RlbV90YXNrcV8zICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMDczIGtlcm5lbCAgICAgICAgICAgZGVhZGxrcmVzICAgICAgICBtaV9zd2l0Y2ggc2xl ZXBxX3RpbWVkd2FpdCBfc2xlZXAgZGVhZGxrcmVzIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDA4NCBrZXJuZWwgICAgICAgICAgIHppb19udWxsX2lzc3VlICAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDg1IGtlcm5lbCAgICAgICAgICAgemlv X251bGxfaW50ciAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwODYga2Vy bmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8wIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDA4NyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzEgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDg4IGtlcm5lbCAgICAgICAgICAgemlvX3Jl YWRfaXNzdWVfMiBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwODkga2VybmVs ICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8zIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDA5MCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzQgbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDkxIGtlcm5lbCAgICAgICAgICAgemlvX3JlYWRf aXNzdWVfNSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTIga2VybmVsICAg ICAgICAgICB6aW9fcmVhZF9pc3N1ZV82IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDA5MyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzcgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMDk0IGtlcm5lbCAgICAgICAgICAgemlvX3JlYWRfaW50 cl8wICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTUga2VybmVsICAgICAg ICAgICB6aW9fcmVhZF9pbnRyXzEgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDA5NiBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2ludHJfMiAgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMDk3IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3Vl XyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTgga2VybmVsICAgICAgICAg ICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA5 OSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMTAwIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3VlXyBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDEga2VybmVsICAgICAgICAgICB6 aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwMiBr ZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMTAzIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3VlXyBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDQga2VybmVsICAgICAgICAgICB6aW9f d3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwNSBrZXJu ZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMTA2IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfMSBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDcga2VybmVsICAgICAgICAgICB6aW9fd3Jp dGVfaW50cl8yIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwOCBrZXJuZWwg ICAgICAgICAgIHppb193cml0ZV9pbnRyXzMgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMTA5IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNCBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAxMTAga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVf aW50cl81IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExMSBrZXJuZWwgICAg ICAgICAgIHppb193cml0ZV9pbnRyXzYgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMTEyIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNyBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAxMTMga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVfaW50 cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExNCBrZXJuZWwgICAgICAg ICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MTE1IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfaCBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAxMTYga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVfaW50cl9o IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExNyBrZXJuZWwgICAgICAgICAg IHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTE4 IGtlcm5lbCAgICAgICAgICAgemlvX2ZyZWVfaXNzdWUgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAxMTkga2VybmVsICAgICAgICAgICB6aW9fZnJlZV9pbnRyICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEyMCBrZXJuZWwgICAgICAgICAgIHpp b19jbGFpbV9pc3N1ZSAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTIxIGtl cm5lbCAgICAgICAgICAgemlvX2NsYWltX2ludHIgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAxMjIga2VybmVsICAgICAgICAgICB6aW9faW9jdGxfaXNzdWUgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEyMyBrZXJuZWwgICAgICAgICAgIHppb19p b2N0bF9pbnRyICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTI0IGtlcm5l bCAgICAgICAgICAgemZzX3ZuX3JlbGVfdGFzayBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAxMjcga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE0MSBrZXJuZWwgICAgICAgICAgIHppb19udWxs X2lzc3VlICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQyIGtlcm5lbCAg ICAgICAgICAgemlvX251bGxfaW50ciAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVw IHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAg MCAxMDAxNDMga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8wIG1pX3N3aXRjaCBz bGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3Jr X3RyYW1wb2xpbmUgCiAgICAwIDEwMDE0NCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lz c3VlXzEgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xv b3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQ1IGtlcm5lbCAgICAg ICAgICAgemlvX3JlYWRfaXNzdWVfMiBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRh c2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAx MDAxNDYga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8zIG1pX3N3aXRjaCBzbGVl cHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDE0NyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3Vl XzQgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3Ag Zm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQ4IGtlcm5lbCAgICAgICAg ICAgemlvX3JlYWRfaXNzdWVfNSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2tx dWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAx NDkga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV82IG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICAwIDEwMDE1MCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzcg bWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9y a19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTUxIGtlcm5lbCAgICAgICAgICAg emlvX3JlYWRfaW50cl8wICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1 ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTIg a2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pbnRyXzEgIG1pX3N3aXRjaCBzbGVlcHFfd2Fp dCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDE1MyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2ludHJfMiAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTU0IGtlcm5lbCAgICAgICAgICAgemlv X3dyaXRlX2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTUga2Vy bmVsICAgICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDE1NiBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTU3IGtlcm5lbCAgICAgICAgICAgemlvX3dy aXRlX2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTgga2VybmVs ICAgICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDE1OSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTYwIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRl X2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjEga2VybmVsICAg ICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDE2MiBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMTYzIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lu dHJfMSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjQga2VybmVsICAgICAg ICAgICB6aW9fd3JpdGVfaW50cl8yIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDE2NSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzMgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMTY2IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJf NCBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjcga2VybmVsICAgICAgICAg ICB6aW9fd3JpdGVfaW50cl81IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE2 OCBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzYgbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMTY5IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNyBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzAga2VybmVsICAgICAgICAgICB6 aW9fd3JpdGVfaW50cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3MSBr ZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMTcyIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfaCBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzMga2VybmVsICAgICAgICAgICB6aW9f d3JpdGVfaW50cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3NCBrZXJu ZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMTc1IGtlcm5lbCAgICAgICAgICAgemlvX2ZyZWVfaXNzdWUgICBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzYga2VybmVsICAgICAgICAgICB6aW9fZnJl ZV9pbnRyICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3NyBrZXJuZWwg ICAgICAgICAgIHppb19jbGFpbV9pc3N1ZSAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMTc4IGtlcm5lbCAgICAgICAgICAgemlvX2NsYWltX2ludHIgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAxNzkga2VybmVsICAgICAgICAgICB6aW9faW9jdGxf aXNzdWUgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE4MCBrZXJuZWwgICAg ICAgICAgIHppb19pb2N0bF9pbnRyICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMTgxIGtlcm5lbCAgICAgICAgICAgemZzX3ZuX3JlbGVfdGFzayBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAxOTUga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAg ICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE5NiBrZXJuZWwgICAgICAg ICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MTk3IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAxOTgga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAg IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE5OSBrZXJuZWwgICAgICAgICAg IHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjAw IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAyMDEga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwMiBrZXJuZWwgICAgICAgICAgIHpp bF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjAzIGtl cm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAyMDQga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwNSBrZXJuZWwgICAgICAgICAgIHppbF9j bGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjA2IGtlcm5l bCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAyMDcga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwOCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVh biAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjA5IGtlcm5lbCAg ICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVw IHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAg MCAxMDAyMTAga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBz bGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3Jr X3RyYW1wb2xpbmUgCiAgICAwIDEwMDIxMSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAg ICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xv b3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjEyIGtlcm5lbCAgICAg ICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRh c2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAx MDAyMTMga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVl cHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDIxNCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAg ICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3Ag Zm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjE1IGtlcm5lbCAgICAgICAg ICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2tx dWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAy MTYga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICAwIDEwMDIxNyBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAg bWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9y a19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjE4IGtlcm5lbCAgICAgICAgICAg emlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1 ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMTkg a2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2Fp dCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDIyMCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjIxIGtlcm5lbCAgICAgICAgICAgemls X2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjIga2Vy bmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDIyMyBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjI0IGtlcm5lbCAgICAgICAgICAgemlsX2Ns ZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjUga2VybmVs ICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDIyNiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjI3IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFu ICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjgga2VybmVsICAg ICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDIyOSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMjMwIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAg ICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzEga2VybmVsICAgICAg ICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDIzMiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMjMzIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAg ICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzQga2VybmVsICAgICAgICAg ICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIz NSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMjM2IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzcga2VybmVsICAgICAgICAgICB6 aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIzOCBr ZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMjM5IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyNDAga2VybmVsICAgICAgICAgICB6aWxf Y2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0MSBrZXJu ZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMjQyIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyNDMga2VybmVsICAgICAgICAgICB6aWxfY2xl YW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0NCBrZXJuZWwg ICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMjQ1IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAyNDYga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4g ICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0NyBrZXJuZWwgICAg ICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMjQ4IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAyNDkga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAg ICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1MCBrZXJuZWwgICAgICAg ICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MjUxIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAyNTIga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAg IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1MyBrZXJuZWwgICAgICAgICAg IHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjU0 IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAyNTUga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1NiBrZXJuZWwgICAgICAgICAgIHpp bF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjU3IGtl cm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAyNTgga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1OSBrZXJuZWwgICAgICAgICAgIHppbF9j bGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjYwIGtlcm5l bCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAyNjEga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI2MiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVh biAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKCg== --------------080904090108030003080008-- From owner-zfs-devel@FreeBSD.ORG Mon Oct 25 10:46:23 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 397DE106566C; Mon, 25 Oct 2010 10:46:23 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C39BF8FC16; Mon, 25 Oct 2010 10:46:22 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id ED7E7119B71; Mon, 25 Oct 2010 12:46:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id eYgPoc+zv+HL; Mon, 25 Oct 2010 12:46:20 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 1BE6B119B60; Mon, 25 Oct 2010 12:46:20 +0200 (CEST) Message-ID: <4CC55FFB.1010303@FreeBSD.org> Date: Mon, 25 Oct 2010 12:46:19 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: ZFS patch for review: osol rev. 10204, 10209 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2010 10:46:23 -0000 Hi, I would like to propose the following two upstream changesets for import: OpenSolaris revision 10204: (fixes for zfs receive) Bug ID 6760420: zfs unmount -f causes recv to fail Bug ID 6759975: zplprops unavailable to examiner of in-progress receive http://people.freebsd.org/~mm/patches/zfs/head-10204.patch OpenSolaris revision 10209: Bug ID 6830541: zfs_get_data_trips on a verify Bug ID 6696242: multiple zfs_fillpage() zfs: accessing past end of object panics Bug ID 6785914: zfs fails to drop dn_struct_rwlock in recovery code path http://people.freebsd.org/~mm/patches/zfs/head-10209.patch Next, I am currently also working to import the corresponding fix for Bug ID 6860996 (%temporary clones are not automatically destroyed on error) that fixes in revision 10298 a problem that got introduced in 9396. From owner-zfs-devel@FreeBSD.ORG Tue Oct 26 08:25:04 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 402C8106564A; Tue, 26 Oct 2010 08:25:04 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 81D888FC12; Tue, 26 Oct 2010 08:25:03 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 6F827A68C2D; Tue, 26 Oct 2010 16:25:01 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id k+QsEb73O1cq; Tue, 26 Oct 2010 16:24:52 +0800 (CST) Received: from delta.delphij.net (c-76-102-48-203.hsd1.ca.comcast.net [76.102.48.203]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 1C0A6A68C1E; Tue, 26 Oct 2010 16:24:49 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=IWHHuVDDpccqmQZBz5OGZoQBR4KC3n2MyFj1/4nsddjUjO5EfGse5W54uvhSqjqE4 D928dsKsoOkztINXqCymw== Message-ID: <4CC6904B.4090905@delphij.net> Date: Tue, 26 Oct 2010 01:24:43 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.14) Gecko/20101020 Thunderbird/3.0.9 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: Martin Matuska References: <4CC55FFB.1010303@FreeBSD.org> In-Reply-To: <4CC55FFB.1010303@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.ORG, Pawel Jakub Dawidek Subject: Re: ZFS patch for review: osol rev. 10204, 10209 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Oct 2010 08:25:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, On 10/25/10 03:46, Martin Matuska wrote: > Hi, I would like to propose the following two upstream changesets for > import: > > OpenSolaris revision 10204: (fixes for zfs receive) > Bug ID 6760420: zfs unmount -f causes recv to fail > Bug ID 6759975: zplprops unavailable to examiner of in-progress receive > > http://people.freebsd.org/~mm/patches/zfs/head-10204.patch > > OpenSolaris revision 10209: > Bug ID 6830541: zfs_get_data_trips on a verify > Bug ID 6696242: multiple zfs_fillpage() zfs: accessing past end of > object panics > Bug ID 6785914: zfs fails to drop dn_struct_rwlock in recovery code path > > http://people.freebsd.org/~mm/patches/zfs/head-10209.patch I have ran these patches on my laptop and checked with OpenSolaris code, I believe both are good for merge. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMxpBKAAoJEATO+BI/yjfBF6QIAIGzbmx14mN2E2qYRaLuVF3r OCfThQpI5bUxALMRyYwTs2s5ybWsfyc0KguYffmoHZrv4f0KPJAwuxUzik4rFDDw FtJiblJTFfFXYkaSZk04nI/fS694xcnWkp2Jrn9h6Y1x1rn/d/liU2MTGWeIgOAo KvR3hF9cqvb5wWuJijy+BwiNp3x2Feb2HRQ7+391++9Yj7tOx1LaInmWlq8NdAaQ YhQ6LtusYHl1/CsvbitNWMMMQdx5kjnMqnTIwl6leVq/P5CmwWSG71TP68xQJP3k WarnguSDP0/m3SYrz0cga5iVb7YEDGYwi/IqQijsHe9y1LpKLPa9tHxV6GgfgwA= =P0dR -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Oct 28 08:20:46 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 454B5106564A; Thu, 28 Oct 2010 08:20:46 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id CCE538FC08; Thu, 28 Oct 2010 08:20:45 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 1DF8D126AC5; Thu, 28 Oct 2010 10:20:44 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id B3+vfEmPCQYZ; Thu, 28 Oct 2010 10:20:42 +0200 (CEST) Received: from [127.0.0.1] (unknown [217.67.16.66]) by mail.vx.sk (Postfix) with ESMTPSA id 2CF5A126AB6; Thu, 28 Oct 2010 10:20:42 +0200 (CEST) Message-ID: <4CC9325F.2030006@FreeBSD.org> Date: Thu, 28 Oct 2010 10:20:47 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Patch propisal: device opening and creation speedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Oct 2010 08:20:46 -0000 Hi, I would like to propose the following two upstream changesets for import: OpenSolaris revision 9846:6527c7b4a92e Bug ID 6566744: vdev_open() should be done in parallel OpenSolaris revision 10588:dc03f981ea18 (partial) - vdev.c change for zpools on top of zvol devices http://people.freebsd.org/~mm/patches/zfs/head-9846-10588p.patch The code is identical to state of V28 and produces a noticeable speedup of vdev_open() and vdev_create() on systems with many devices (e.g. storage servers - FreeNAS etc.). It would be good to have this in 8.2-RELEASE as well. From owner-zfs-devel@FreeBSD.ORG Thu Nov 4 16:56:17 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7B5C106566B; Thu, 4 Nov 2010 16:56:17 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 70B638FC0A; Thu, 4 Nov 2010 16:56:15 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA29317; Thu, 04 Nov 2010 18:56:11 +0200 (EET) (envelope-from avg@freebsd.org) Message-ID: <4CD2E5AA.1090208@freebsd.org> Date: Thu, 04 Nov 2010 18:56:10 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.11) Gecko/20101021 Lightning/1.0b2 Thunderbird/3.1.5 MIME-Version: 1.0 To: Kostik Belousov , Pawel Jakub Dawidek , Alan Cox References: <4C91F031.1010801@freebsd.org> <20101010134717.GA59922@deviant.kiev.zoral.com.ua> In-Reply-To: <20101010134717.GA59922@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 04 Nov 2010 17:25:47 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: vop_getpages for zfs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Nov 2010 16:56:18 -0000 on 10/10/2010 16:47 Kostik Belousov said the following: > I think that readahead requests should be satisfied, if possible. Esp. > in the case the readahead does not cause additional i/o. So, per Kostik's suggestion, here is patch for your review and testing: http://people.freebsd.org/~avg/zfs_getpages.2.diff The patch tries to fill as many of the passed in pages as possible as long as that can be done from a block (or even two blocks) that has to be read to fill the requested page. This means that the optimization kicks in only if a vnode has a block size greater than the page size. I mentioned "two blocks" above because I tried to optimize the case when, for example, block size is 6KB and page size is 4KB and the requested page covers parts of two adjacent blocks (e.g. range from 4KB to 8KB). Not sure if this was worth the trouble. In other words, the code tries to avoid reading more blocks than is strictly required, because additional blocks may be far apart on a disk and thus read latency could get increased. You will also notice that I added vop_bmap method to ZFS vops. This is needed to satisfy logic in vnode_pager_haspage(); otherwise, for example, vm_fault() would never request any additional pages besides the required one. zfs_bmap() implementation is really simple - it returns zeros for a_runp and a_runb (and some reasonable-ish but unused values in other fields), and vnode_pager_haspage() figures out number of eligible pages from the block size (f_iosize actually[*]) and the page size. Now, that ZFS has vop_getpages method, it is safe to provide this dummy-ish implementation of vop_bmap, because this operation would not be called by any other place but vnode_pager_haspage(). Code from vfs_bio.c and vfs_cluster.c is never called for ZFS, because it doesn't use the buffer cache layer. [*] There is a few interesting questions with respect to f_iosize usage in vnode_pager_haspage() and ZFS. For one, ZFS has a variable block size and f_iosize for ZFS has a currently configured maximum block ('record' in ZFS terms) size. This should not result in non-optimal behavior for files with smaller block sizes, because those files would have smaller sizes too (within the block size). Unless, of course, recordsize property is modified after some files are created on a ZFS filesystem. In that case there could be large files with some smaller (previously maximum) block size; or files with a block size larger than the current recordsize == f_iosize. So, such situations may lead to sub-optimal performance. Another issue related to the above is that f_iosize for ZFS can change on the fly when an administrators runs zfs set recordsize=NNN, and that may happen concurrently with vnode_pager_haspage() using f_iosize for its calculations. Not sure how to handle this correctly. I will appreciate your comment comments, test reports, reviews and suggestions. Thanks a lot! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Nov 13 09:15:14 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 150B3106564A for ; Sat, 13 Nov 2010 09:15:14 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 48E3C8FC17 for ; Sat, 13 Nov 2010 09:15:13 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 9AC2FA68247; Sat, 13 Nov 2010 17:15:10 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id xZGweRuNh-WE; Sat, 13 Nov 2010 17:15:03 +0800 (CST) Received: from delta.delphij.net (c-76-102-26-215.hsd1.ca.comcast.net [76.102.26.215]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 823A3A68219; Sat, 13 Nov 2010 17:15:02 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=Q3exD4DsTH9p+QNMOv1+M4dnNnq/y6fl7E1eT2++mycK+RhVO9EaT97Ud/DH14a4x pzBuLG9StxSElHr40uFaA== Message-ID: <4CDE5712.2040306@delphij.net> Date: Sat, 13 Nov 2010 01:14:58 -0800 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.15) Gecko/20101028 Thunderbird/3.0.10 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Dead/Live lock near VOP_LOCK1_APV? X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Nov 2010 09:15:14 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I have observed a live/dead lock on a system serving both NFS and Samba CIFS, running 8.1-STABLE. Is this a known issue or should I investigate further? The system is not running with WITNESS but if further debugging is needed I can add it. procstat -kk shows something like: 7 100068 zfskern l2arc_feed_threa mi_switch sleepq_timedwait _cv_timedwait l2arc_feed_thread fork_exit fork_trampoline *7 100122 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline 7 100123 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit fork_trampoline *7 100184 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline 7 100185 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit fork_trampoline 1502 100284 nfsd nfsd: service mi_switch sleepq_wait __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock zfs_fhtovp nfsrv_fhtovp nfsrv_statfs nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 27663 100779 smbd - mi_switch sleepq_wait __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock cache_lookup vfs_cache_lookup VOP_LOOK UP_APV lookup namei vn_open_cred kern_openat syscallenter syscall Xfast_syscall Full procstat -kk output can be found at: http://neptune.delphij.net/procstat.txt Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJM3lcSAAoJEATO+BI/yjfB9NsIAKWvZ2NDaK1kNm2fQsmh+fmX jwMZAeceeySdCF3/VD5y9AQefGF4xUVRjJkN9iPnqFW2ggEhJbmftd9soVKXQlgK zO8Bky4roINImjwyz4bjfrkU/0yEXitgGpvusLTKZyDMNqqnMHZCkR8mthGGfpFU iCU3dpEI4I7Vm1sOf9tsGOsXS88cf/mbJupUI0IwUIjtfBIbZnoDQYq6oeFd3+L9 Vw2XjGCw5cqONTG6UKoixQbQ8UoD+chXUDbrf211ECe9fE8PDD5GAduQ4teLeH3G Tdsl3zLrw7plMXRR18lwx2oMLngJVnJrMJ7N5/LMnfVvwvK+h0ZzPMCyfErrNvs= =cclF -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sat Nov 13 10:05:27 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 380BE106566B for ; Sat, 13 Nov 2010 10:05:27 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7E17F8FC23 for ; Sat, 13 Nov 2010 10:05:26 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA11724; Sat, 13 Nov 2010 11:47:33 +0200 (EET) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1PHChp-000MrO-9x; Sat, 13 Nov 2010 11:47:33 +0200 Message-ID: <4CDE5E64.3090708@icyb.net.ua> Date: Sat, 13 Nov 2010 11:46:12 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101029 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: d@delphij.net References: <4CDE5712.2040306@delphij.net> In-Reply-To: <4CDE5712.2040306@delphij.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: Dead/Live lock near VOP_LOCK1_APV? X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Nov 2010 10:05:27 -0000 on 13/11/2010 11:14 Xin LI said the following: > Hi, > > I have observed a live/dead lock on a system serving both NFS and Samba > CIFS, running 8.1-STABLE. Is this a known issue or should I investigate > further? Yes, it looks like a known issue. A fix has been recently committed and MFC-ed. It was a lock leak in NFS. > The system is not running with WITNESS but if further debugging is > needed I can add it. > > procstat -kk shows something like: > > 7 100068 zfskern l2arc_feed_threa mi_switch > sleepq_timedwait _cv_timedwait l2arc_feed_thread fork_exit fork_trampoline > *7 100122 zfskern txg_thread_enter mi_switch sleepq_wait > _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline > 7 100123 zfskern txg_thread_enter mi_switch > sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit > fork_trampoline > *7 100184 zfskern txg_thread_enter mi_switch sleepq_wait > _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline > 7 100185 zfskern txg_thread_enter mi_switch > sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit > fork_trampoline > > 1502 100284 nfsd nfsd: service mi_switch sleepq_wait > __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock zfs_fhtovp > nfsrv_fhtovp nfsrv_statfs nfssvc_program svc_run_internal > svc_thread_start fork_exit fork_trampoline > 27663 100779 smbd - mi_switch sleepq_wait > __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock cache_lookup > vfs_cache_lookup VOP_LOOK > UP_APV lookup namei vn_open_cred kern_openat syscallenter syscall > Xfast_syscall > > Full procstat -kk output can be found at: > > http://neptune.delphij.net/procstat.txt > > Cheers, _______________________________________________ zfs-devel@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/zfs-devel To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Dec 13 22:33:48 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCD1F1065670 for ; Mon, 13 Dec 2010 22:33:48 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 879568FC15 for ; Mon, 13 Dec 2010 22:33:47 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A10A745CAC; Mon, 13 Dec 2010 23:01:05 +0100 (CET) Received: from localhost (89-73-192-49.dynamic.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 713E145C99 for ; Mon, 13 Dec 2010 23:01:00 +0100 (CET) Date: Mon, 13 Dec 2010 23:00:56 +0100 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20101213220056.GG2038@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kjpMrWxdCilgNbo1" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Fwd: Next ZFSv28 patchset ready for testing. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Dec 2010 22:33:48 -0000 --kjpMrWxdCilgNbo1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FYI: The new patchset is ready for testing: http://people.freebsd.org/~pjd/patches/zfs_20101212.patch.bz2 When applying the patch be sure to use correct options for patch(1)!: # cd /usr/src # fetch http://people.freebsd.org/~pjd/patches/zfs_20101212.patch.bz2 # bzip2 -d zfs_20101212.patch.bz2 # patch -E -p0 < zfs_20101212.patch The patch is against FreeBSD HEAD as of 2010-12-12. Some of the changes since the last patchset (zfs_20100831.patch): - Boot support for ZFS v28 (only RAIDZ3 is not yet supported). - Various fixes for the existing ZFS boot code. - Support for sendfile(2) (by avg@). - Userland<->kernel compatibility with v13-v15 (by mm@). - ACL fixes (by trasz@). - Various bug fixes. Please test, test, test. Chances are this is the last patchset before v28 going to HEAD (finally). Especially test new changes, like boot support and sendfile(2) support. Also be sure to verify if you can import for existing ZFS pools (v13-v15) when running v28 or boot from your existing pools. Enjoy! PS. Martin (mm@) will be providing patch against 8-STABLE soon. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kjpMrWxdCilgNbo1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEUEARECAAYFAk0Gl5cACgkQForvXbEpPzR/zwCXc1bmawHui1+BqpzvD/FdUTfw gQCdFolhu/bZXcXBJ2lYXs7l3DyrfpU= =K1Xb -----END PGP SIGNATURE----- --kjpMrWxdCilgNbo1-- From owner-zfs-devel@FreeBSD.ORG Mon Jan 31 21:58:00 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86106106566B for ; Mon, 31 Jan 2011 21:58:00 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 3AAE08FC29 for ; Mon, 31 Jan 2011 21:57:59 +0000 (UTC) Received: from [192.168.4.138] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p0VLdCJv074828 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 31 Jan 2011 14:39:13 -0700 (MST) (envelope-from gibbs@scsiguy.com) Message-ID: <4D472BFC.5070404@scsiguy.com> Date: Mon, 31 Jan 2011 14:39:08 -0700 From: "Justin T. Gibbs" Organization: SCSIGuy.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110123 Thunderbird/3.1.8 MIME-Version: 1.0 To: zfs-devel@freebsd.org Content-Type: multipart/mixed; boundary="------------090502030703040508070704" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Mon, 31 Jan 2011 14:39:13 -0700 (MST) Subject: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@scsiguy.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jan 2011 21:58:00 -0000 This is a multi-part message in MIME format. --------------090502030703040508070704 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On both -current and -stable (v16), I've been able to reliably panic the system by pulling a drive (leaf vdev in a RAIDZ2) while I/O is in progress. The source of the panic is either from GEOM complaining that vdev_geom's consumer isn't idle, or "use after free" bugs caused by in-flight I/O (often due to a re-probe). The following patch resolves the issue for me. >From a brief review of the v28 code, it seems appropriate there too. The code in vdev_geom_orphan is identical save for an added call to zfs_post_remove(). In our port, this call appears to do nothing more than to post the devctl notice event a little earlier than via the scheduled removal of the device by the spa code. Is there some benefit I'm missing that makes the zfs_post_remove() necessary? -- Justin --------------090502030703040508070704 Content-Type: text/plain; name="vdev_geom.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="vdev_geom.c.diff" Change 475132 by justing@justing-ns1 on 2011/01/14 13:08:34 Correct ZFS panic during device removal. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c: When the ZFS geom device consumer is orphaned, simply signal the SPA to begin device removal proceedings instead of attempting to destroy the consumer directly. The orphan handler is called with the geom topology lock held from the geom event thread. In order to perform a complete consumer shutdown from this context, the SPA ZIO configuration lock would need to be held as a writer to insure that no ZIOs are active on this consumer. This lock acquisition order is the reverse of the normal SPA config lock -> geom topology lock order. We could probably drop locks and acquire in the correct order, but it is much simpler to just defer all of this until the SPA's eventual close of the vdev_geom instance. Affected files ... ... //depot/SpectraBSD/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c#7 edit Differences ... ==== //depot/SpectraBSD/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c#7 (text) ==== @@ -50,28 +50,26 @@ static void vdev_geom_orphan(struct g_consumer *cp) { - struct g_geom *gp; vdev_t *vd; - int error; g_topology_assert(); vd = cp->private; - gp = cp->geom; - error = cp->provider->error; - ZFS_LOG(1, "Closing access to %s.", cp->provider->name); - if (cp->acr + cp->acw + cp->ace > 0) - g_access(cp, -cp->acr, -cp->acw, -cp->ace); - ZFS_LOG(1, "Destroyed consumer to %s.", cp->provider->name); - g_detach(cp); - g_destroy_consumer(cp); - /* Destroy geom if there are no consumers left. */ - if (LIST_EMPTY(&gp->consumer)) { - ZFS_LOG(1, "Destroyed geom %s.", gp->name); - g_wither_geom(gp, error); - } - vd->vdev_tsd = NULL; + /* + * Orphan callbacks occur from the GEOM event thread. + * Concurrent with this call, new I/O requests may be + * working their way through GEOM about to find out + * (only once executed by the g_down thread) that we've + * been orphaned from our disk provider. These I/Os + * must be retired before we can detach our consumer. + * This is most easily achieved by acquiring the + * SPA ZIO configuration lock as a writer, but doing + * so with the GEOM topology lock held would cause + * a lock order reversal. Instead, rely on the SPA's + * async removal support to invoke a close on this + * vdev once it is safe to do so. + */ vd->vdev_remove_wanted = B_TRUE; spa_async_request(vd->vdev_spa, SPA_ASYNC_REMOVE); } --------------090502030703040508070704-- From owner-zfs-devel@FreeBSD.ORG Fri Feb 4 23:11:18 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B7071065672 for ; Fri, 4 Feb 2011 23:11:18 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 1719A8FC08 for ; Fri, 4 Feb 2011 23:11:18 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E529645E93; Fri, 4 Feb 2011 23:52:55 +0100 (CET) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id CA6AD4569A; Fri, 4 Feb 2011 23:52:50 +0100 (CET) Date: Fri, 4 Feb 2011 23:52:34 +0100 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20110204225234.GP2035@garage.freebsd.pl> References: <4D472BFC.5070404@scsiguy.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/d7X7C0hV/blnKmH" Content-Disposition: inline In-Reply-To: <4D472BFC.5070404@scsiguy.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Feb 2011 23:11:18 -0000 --/d7X7C0hV/blnKmH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > On both -current and -stable (v16), I've been able to reliably panic the > system by pulling a drive (leaf vdev in a RAIDZ2) while I/O is > in progress. The source of the panic is either from GEOM complaining > that vdev_geom's consumer isn't idle, or "use after free" bugs caused > by in-flight I/O (often due to a re-probe). The following patch resolves > the issue for me. I was able to reproduce the panic on v28 and I integrated your patch there. > >From a brief review of the v28 code, it seems appropriate there too. > The code in vdev_geom_orphan is identical save for an added call to > zfs_post_remove(). In our port, this call appears to do nothing more > than to post the devctl notice event a little earlier than via the > scheduled removal of the device by the spa code. Is there some benefit > I'm missing that makes the zfs_post_remove() necessary? Without it I see no info about vdev removal in devd. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/d7X7C0hV/blnKmH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk1MgzEACgkQForvXbEpPzT5tQCgwy7D63QdPQBkVPdSxIXJ3NiP CKIAoIgAo6Ce/X0l5YmwKJHo7x3lKzM1 =d7XB -----END PGP SIGNATURE----- --/d7X7C0hV/blnKmH-- From owner-zfs-devel@FreeBSD.ORG Sun Feb 6 02:56:07 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AE16106564A; Sun, 6 Feb 2011 02:56:07 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id EDAA98FC08; Sun, 6 Feb 2011 02:56:06 +0000 (UTC) Received: from [192.168.0.21] ([192.168.0.21]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p162u5oJ031680 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 5 Feb 2011 19:56:05 -0700 (MST) (envelope-from gibbs@scsiguy.com) Message-ID: <4D4E0DC1.4030002@scsiguy.com> Date: Sat, 05 Feb 2011 19:56:01 -0700 From: "Justin T. Gibbs" Organization: SCSIGuy.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110123 Thunderbird/3.1.8 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4D472BFC.5070404@scsiguy.com> <20110204225234.GP2035@garage.freebsd.pl> In-Reply-To: <20110204225234.GP2035@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Sat, 05 Feb 2011 19:56:05 -0700 (MST) Cc: zfs-devel@FreeBSD.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@scsiguy.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 02:56:07 -0000 On 2/4/2011 3:52 PM, Pawel Jakub Dawidek wrote: > On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > >> >From a brief review of the v28 code, it seems appropriate there too. >> The code in vdev_geom_orphan is identical save for an added call to >> zfs_post_remove(). In our port, this call appears to do nothing more >> than to post the devctl notice event a little earlier than via the >> scheduled removal of the device by the spa code. Is there some benefit >> I'm missing that makes the zfs_post_remove() necessary? > > Without it I see no info about vdev removal in devd. Do you know why the call to zfs_post_remove() was removed from vdev_set_state()? This is where the notification is made in -current (v15). It seems wrong to put this policy into each vdev driver instead of just doing it in the common vdev code. -- Justin From owner-zfs-devel@FreeBSD.ORG Sun Feb 6 14:30:25 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F98106564A for ; Sun, 6 Feb 2011 14:30:25 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id A82608FC18 for ; Sun, 6 Feb 2011 14:30:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 1348B45E9D; Sun, 6 Feb 2011 15:30:23 +0100 (CET) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 55C024569A; Sun, 6 Feb 2011 15:30:17 +0100 (CET) Date: Sun, 6 Feb 2011 15:29:56 +0100 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20110206142956.GG2035@garage.freebsd.pl> References: <4D472BFC.5070404@scsiguy.com> <20110204225234.GP2035@garage.freebsd.pl> <4D4E0DC1.4030002@scsiguy.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+bs7B30DeWCM5QK8" Content-Disposition: inline In-Reply-To: <4D4E0DC1.4030002@scsiguy.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 14:30:25 -0000 --+bs7B30DeWCM5QK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 05, 2011 at 07:56:01PM -0700, Justin T. Gibbs wrote: > On 2/4/2011 3:52 PM, Pawel Jakub Dawidek wrote: > > On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > >=20 > >> >From a brief review of the v28 code, it seems appropriate there too. > >> The code in vdev_geom_orphan is identical save for an added call to > >> zfs_post_remove(). In our port, this call appears to do nothing more > >> than to post the devctl notice event a little earlier than via the > >> scheduled removal of the device by the spa code. Is there some benefit > >> I'm missing that makes the zfs_post_remove() necessary? > >=20 > > Without it I see no info about vdev removal in devd. >=20 > Do you know why the call to zfs_post_remove() was removed from > vdev_set_state()? This is where the notification is made in -current (v1= 5). > It seems wrong to put this policy into each vdev driver instead of > just doing it in the common vdev code. This was changed in 2a8816c5173b. And the added comment states: /* * We post the resource as soon as possible, instead of * when the async removal actually happens, because the * DE is using this information to discard previous I/O * errors. */ zfs_post_remove(zio->io_spa, vd); This is from vdev_disk.c. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+bs7B30DeWCM5QK8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk1OsGQACgkQForvXbEpPzS4rACg99/PHRApFriPZbN1kFhovYTC pJYAnA4wiPAePum5Di8SJK7/3ocJpTuo =c49u -----END PGP SIGNATURE----- --+bs7B30DeWCM5QK8-- From owner-zfs-devel@FreeBSD.ORG Fri May 6 23:43:27 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id BDFBF1065675; Fri, 6 May 2011 23:43:27 +0000 (UTC) Date: Fri, 6 May 2011 23:43:27 +0000 From: Alexander Best To: zfs-devel@freebsd.org Message-ID: <20110506234327.GA71369@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="x+6KMIRAuhnl3hBn" Content-Disposition: inline Subject: lots of missing declarations X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2011 23:43:27 -0000 --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline hi everybody, i'm trying to successfully run 'make MAKE_JUST_KERNELS=yes tinderbox' with the -Wmissing-declarations flag. apart from one issue in the xfs code, all warnings issued by this flag come from inside the zfs sources. i've attached a log file, which documents all of them. is there any possibility these can be fixed and we can then permanently add -Wmissing-declarations to CWARNFLAGS in sys/conf/kern.mk? cheers. alex -- a13x --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=zfs-missing-decl /usr/git-freebsd-head/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_kobj.c:169: warning: no previous declaration for 'kobj_read_file_vnode' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_kobj.c:200: warning: no previous declaration for 'kobj_read_file_loader' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/callb.c:99: warning: no previous declaration for 'callb_init' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/callb.c:107: warning: no previous declaration for 'callb_fini' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/fm.c:1167: warning: no previous declaration for 'fm_ena_generate_cpu' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:43: warning: no previous declaration for 'zcalloc' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:59: warning: no previous declaration for 'zcfree' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:70: warning: no previous declaration for 'zmemcpy' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:76: warning: no previous declaration for 'zmemcmp' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:82: warning: no previous declaration for 'zmemzero' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1651: warning: no previous declaration for 'arc_buf_free' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2183: warning: no previous declaration for 'arc_shrink' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:612: warning: no previous declaration for 'ddt_select_by_checksum' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:806: warning: no previous declaration for 'dsl_pool_user_hold_create_obj' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c:550: warning: no previous declaration for 'dsl_prop_set_sync' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/gzip.c:41: warning: no previous declaration for 'gzip_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/gzip.c:60: warning: no previous declaration for 'gzip_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:51: warning: no previous declaration for 'lzjb_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:99: warning: no previous declaration for 'lzjb_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:445: warning: no previous declaration for 'metaslab_pp_maxsize' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:473: warning: no previous declaration for 'metaslab_ff_fragmented' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:276: warning: no previous declaration for 'sa_layout_equal' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:327: warning: no previous declaration for 'sa_attr_op' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1131: warning: no previous declaration for 'sa_build_idx_tab' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1195: warning: no previous declaration for 'sa_byteswap_cb' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1204: warning: no previous declaration for 'sa_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1279: warning: no previous declaration for 'sa_evict' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1409: warning: no previous declaration for 'sa_lookup_impl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c:35: warning: no previous declaration for 'zio_checksum_SHA256' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:4924: warning: no previous declaration for 'spa_vdev_set_common' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:646: warning: no previous declaration for 'spa_aux_add' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:664: warning: no previous declaration for 'spa_aux_remove' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:684: warning: no previous declaration for 'spa_aux_exists' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:709: warning: no previous declaration for 'spa_aux_activate' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1778: warning: no previous declaration for 'vdev_dtl_sync' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1987: warning: no previous declaration for 'vdev_remove' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:963: warning: no previous declaration for 'vdev_uberblock_sync_list' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:1072: warning: no previous declaration for 'vdev_label_sync_list' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91: warning: no previous declaration for 'vdev_queue_deadline_compare' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:115: warning: no previous declaration for 'vdev_queue_offset_compare' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:203: warning: no previous declaration for 'zap_name_alloc_uint64' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1998: warning: no previous declaration for 'zfs_release_sa_handle' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1447: warning: no previous declaration for 'zio_rewrite_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1485: warning: no previous declaration for 'zio_free_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1493: warning: no previous declaration for 'zio_claim_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zle.c:38: warning: no previous declaration for 'zle_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zle.c:68: warning: no previous declaration for 'zle_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c:165: warning: no previous declaration for 'zrl_refcount' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:136: warning: no previous declaration for 'fletcher_2_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:153: warning: no previous declaration for 'fletcher_2_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:170: warning: no previous declaration for 'fletcher_4_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:187: warning: no previous declaration for 'fletcher_4_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:205: warning: no previous declaration for 'fletcher_4_incremental_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:228: warning: no previous declaration for 'fletcher_4_incremental_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c:658: warning: no previous declaration for 'zfs_copy_ace_2_fuid' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:583: warning: no previous declaration for 'zfsctl_freebsd_root_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:919: warning: no previous declaration for 'zfsctl_snapdir_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1082: warning: no previous declaration for 'zfsctl_shares_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:350: warning: no previous declaration for 'zfs_secpolicy_write_perms' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:365: warning: no previous declaration for 'zfs_secpolicy_write_perms_ds' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:534: warning: no previous declaration for 'zfs_secpolicy_fsacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:550: warning: no previous declaration for 'zfs_secpolicy_rollback' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:557: warning: no previous declaration for 'zfs_secpolicy_send' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:618: warning: no previous declaration for 'zfs_secpolicy_share' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:631: warning: no previous declaration for 'zfs_secpolicy_smb_acl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:1904: warning: no previous declaration for 'dataset_name_hidden' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:4786: warning: no previous declaration for 'pool_status_check' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1175: warning: no previous declaration for 'zfs_unregister_callbacks' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:2259: warning: no previous declaration for 'zfs_init' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:2285: warning: no previous declaration for 'zfs_fini' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1078: warning: no previous declaration for 'zfs_get_done' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4526: warning: no previous declaration for 'zfs_inactive' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6354: warning: no previous declaration for 'zfs_deleteextattr' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6597: warning: no previous declaration for 'zfs_freebsd_getacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6624: warning: no previous declaration for 'zfs_freebsd_setacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6671: warning: no previous declaration for 'zfs_freebsd_aclcheck' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:637: warning: no previous declaration for 'zvol_first_open' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:677: warning: no previous declaration for 'zvol_last_close' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:729: warning: no previous declaration for 'zvol_update_volsize' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:1194: warning: no previous declaration for 'zvol_strategy' --x+6KMIRAuhnl3hBn-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:15:24 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2731106566B; Mon, 20 Jun 2011 09:15:24 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 6C1048FC1B; Mon, 20 Jun 2011 09:15:24 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 9C51917912B; Mon, 20 Jun 2011 11:15:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id FE5ft0ggqakW; Mon, 20 Jun 2011 11:15:21 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id DAAD5179117; Mon, 20 Jun 2011 11:15:20 +0200 (CEST) Message-ID: <4DFF0FA8.5070805@FreeBSD.org> Date: Mon, 20 Jun 2011 11:15:20 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------080801060903000605080804" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13295 (zfs get /path) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:15:24 -0000 This is a multi-part message in MIME format. --------------080801060903000605080804 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit 'zfs get' command should be able to deal with mountpoint as an argument. It already works with 'zfs list' command. https://www.illumos.org/issues/510 https://www.illumos.org/projects/illumos-gate/repository/revisions/13295 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------080801060903000605080804 Content-Type: text/x-patch; name="13295.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13295.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs_main.c (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs_main.c (working copy) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -1292,7 +1292,7 @@ zfs_do_get(int argc, char **argv) { zprop_get_cbdata_t cb = { 0 }; - int i, c, flags = 0; + int i, c, flags = ZFS_ITER_ARGS_CAN_BE_PATHS; char *value, *fields; int ret; int limit = 0; --------------080801060903000605080804-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:16:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BA50106566B; Mon, 20 Jun 2011 09:16:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id E0C6A8FC0A; Mon, 20 Jun 2011 09:16:14 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 43C121791A4; Mon, 20 Jun 2011 11:16:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id uWS8aXhrBjaR; Mon, 20 Jun 2011 11:16:09 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id B3492179185; Mon, 20 Jun 2011 11:16:08 +0200 (CEST) Message-ID: <4DFF0FD8.6040708@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:08 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------060909070202060002080902" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13346 (set vdev_cache to zero) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:15 -0000 This is a multi-part message in MIME format. --------------060909070202060002080902 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit zfs vdev cache (readahead) consumes excessive memory and is very underutilized. Set zfs_vdev_cache_size to zero. This change is present in Solaris 11. https://www.illumos.org/issues/175 https://www.illumos.org/projects/illumos-gate/repository/revisions/13346 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------060909070202060002080902 Content-Type: text/x-patch; name="13346.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13346.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c (working copy) @@ -71,9 +71,16 @@ * 1< Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A77BB106564A; Mon, 20 Jun 2011 09:16:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 06A2D8FC13; Mon, 20 Jun 2011 09:16:40 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 669FA1791F8; Mon, 20 Jun 2011 11:16:39 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CPgBW3+n-63O; Mon, 20 Jun 2011 11:16:36 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 1A87C1791E7; Mon, 20 Jun 2011 11:16:36 +0200 (CEST) Message-ID: <4DFF0FF3.1030308@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------010907030303010503000607" Cc: Pawel Jakub Dawidek , Edward Tomasz Napierala Subject: [CFR][ZFS] merge Illumos rev. 13370 (resurrect "aclmode" property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:40 -0000 This is a multi-part message in MIME format. --------------010907030303010503000607 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Resurrect "aclmode" property for ZFS. The Illumos changeset includes Issues submitted by trasz (279, 664). https://www.illumos.org/issues/742 https://www.illumos.org/issues/664 https://www.illumos.org/issues/279 https://www.illumos.org/projects/illumos-gate/repository/revisions/13370 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------010907030303010503000607 Content-Type: text/x-patch; name="13370.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13370.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) @@ -6,6 +6,7 @@ .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License. You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. .\" See the License for the specific language governing permissions and limitations under the License. When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with .\" the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner] +.\" Copyright 2011 Nexenta Systems, Inc. All rights reserved. .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" .SH NAME zfs \- configures ZFS file systems @@ -630,7 +631,7 @@ .ad .sp .6 .RS 4n -Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file system with an \fBaclmode\fR property of \fBdiscard\fR deletes all \fBACL\fR entries that do not represent the mode of the file. An \fBaclmode\fR property of \fBgroupmask\fR (the default) reduces user or group permissions. The permissions are reduced, such that they are no greater than the group permission bits, unless it is a user entry that has the same \fBUID\fR as the owner of the file or directory. In this case, the \fBACL\fR permissions are reduced so that they are no greater than owner permission bits. A file system with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no changes are made to the \fBACL\fR other than generating the necessary \fBACL\fR entries to represent the new mode of the file or directory. +Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file system with an \fBaclmode\fR property of \fBdiscard\fR (the default) deletes all \fBACL\fR entries that do not represent the mode of the file. An \fBaclmode\fR property of \fBgroupmask\fR reduces permissions granted in all \fBALLOW\fR entries found in the \fBACL\fR such that they are no greater than the group permissions specified by \fBchmod\fR. A file system with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no changes are made to the \fBACL\fR other than creating or updating the necessary \fBACL\fR entries to represent the new mode of the file or directory. .RE .sp @@ -2685,7 +2686,7 @@ pool/home/bob readonly off default pool/home/bob zoned off default pool/home/bob snapdir hidden default -pool/home/bob aclmode groupmask default +pool/home/bob aclmode discard default pool/home/bob aclinherit restricted default pool/home/bob canmount on default pool/home/bob shareiscsi off default Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.c =================================================================== --- sys/cddl/contrib/opensolaris/common/acl/acl_common.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -376,7 +377,7 @@ * by nfsace, assuming aclent_t -> nfsace semantics. */ static uint32_t -mode_to_ace_access(mode_t mode, int isdir, int isowner, int isallow) +mode_to_ace_access(mode_t mode, boolean_t isdir, int isowner, int isallow) { uint32_t access = 0; int haswriteperm = 0; @@ -419,7 +420,7 @@ access |= ACE_DELETE_CHILD; } /* exec */ - if (mode & 01) { + if (mode & S_IXOTH) { access |= ACE_EXECUTE; } @@ -670,7 +671,7 @@ } static int -convert_aent_to_ace(aclent_t *aclentp, int aclcnt, int isdir, +convert_aent_to_ace(aclent_t *aclentp, int aclcnt, boolean_t isdir, ace_t **retacep, int *retacecnt) { ace_t *acep; @@ -696,7 +697,7 @@ dfaclcnt = aclcnt - i; } - if (dfaclcnt && isdir == 0) { + if (dfaclcnt && !isdir) { return (EINVAL); } @@ -734,7 +735,7 @@ } static int -ace_mask_to_mode(uint32_t mask, o_mode_t *modep, int isdir) +ace_mask_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) { int error = 0; o_mode_t mode = 0; @@ -1031,7 +1032,7 @@ } static int -ace_allow_to_mode(uint32_t mask, o_mode_t *modep, int isdir) +ace_allow_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) { /* ACE_READ_ACL and ACE_READ_ATTRIBUTES must both be set */ if ((mask & (ACE_READ_ACL | ACE_READ_ATTRIBUTES)) != @@ -1044,7 +1045,7 @@ static int acevals_to_aent(acevals_t *vals, aclent_t *dest, ace_list_t *list, - uid_t owner, gid_t group, int isdir) + uid_t owner, gid_t group, boolean_t isdir) { int error; uint32_t flips = ACE_POSIX_SUPPORTED_BITS; @@ -1084,7 +1085,7 @@ static int ace_list_to_aent(ace_list_t *list, aclent_t **aclentp, int *aclcnt, - uid_t owner, gid_t group, int isdir) + uid_t owner, gid_t group, boolean_t isdir) { int error = 0; aclent_t *aent, *result = NULL; @@ -1264,7 +1265,7 @@ static int ln_ace_to_aent(ace_t *ace, int n, uid_t owner, gid_t group, aclent_t **aclentp, int *aclcnt, aclent_t **dfaclentp, int *dfaclcnt, - int isdir) + boolean_t isdir) { int error = 0; ace_t *acep; @@ -1459,7 +1460,7 @@ } static int -convert_ace_to_aent(ace_t *acebufp, int acecnt, int isdir, +convert_ace_to_aent(ace_t *acebufp, int acecnt, boolean_t isdir, uid_t owner, gid_t group, aclent_t **retaclentp, int *retaclcnt) { int error = 0; @@ -1501,7 +1502,7 @@ int -acl_translate(acl_t *aclp, int target_flavor, int isdir, uid_t owner, +acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, uid_t owner, gid_t group) { int aclcnt; @@ -1573,101 +1574,105 @@ } void -acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyone) +acl_trivial_access_masks(mode_t mode, boolean_t isdir, trivial_acl_t *masks) { - *deny1 = *deny2 = *allow0 = *group = 0; + uint32_t read_mask = ACE_READ_DATA; + uint32_t write_mask = ACE_WRITE_DATA|ACE_APPEND_DATA; + uint32_t execute_mask = ACE_EXECUTE; + (void) isdir; /* will need this later */ + + masks->deny1 = 0; if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) - *deny1 |= ACE_READ_DATA; + masks->deny1 |= read_mask; if (!(mode & S_IWUSR) && (mode & (S_IWGRP|S_IWOTH))) - *deny1 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->deny1 |= write_mask; if (!(mode & S_IXUSR) && (mode & (S_IXGRP|S_IXOTH))) - *deny1 |= ACE_EXECUTE; + masks->deny1 |= execute_mask; + masks->deny2 = 0; if (!(mode & S_IRGRP) && (mode & S_IROTH)) - *deny2 = ACE_READ_DATA; + masks->deny2 |= read_mask; if (!(mode & S_IWGRP) && (mode & S_IWOTH)) - *deny2 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->deny2 |= write_mask; if (!(mode & S_IXGRP) && (mode & S_IXOTH)) - *deny2 |= ACE_EXECUTE; + masks->deny2 |= execute_mask; + masks->allow0 = 0; if ((mode & S_IRUSR) && (!(mode & S_IRGRP) && (mode & S_IROTH))) - *allow0 |= ACE_READ_DATA; + masks->allow0 |= read_mask; if ((mode & S_IWUSR) && (!(mode & S_IWGRP) && (mode & S_IWOTH))) - *allow0 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->allow0 |= write_mask; if ((mode & S_IXUSR) && (!(mode & S_IXGRP) && (mode & S_IXOTH))) - *allow0 |= ACE_EXECUTE; + masks->allow0 |= execute_mask; - *owner = ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| + masks->owner = ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| ACE_WRITE_NAMED_ATTRS|ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE; if (mode & S_IRUSR) - *owner |= ACE_READ_DATA; + masks->owner |= read_mask; if (mode & S_IWUSR) - *owner |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->owner |= write_mask; if (mode & S_IXUSR) - *owner |= ACE_EXECUTE; + masks->owner |= execute_mask; - *group = ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| + masks->group = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| ACE_SYNCHRONIZE; if (mode & S_IRGRP) - *group |= ACE_READ_DATA; + masks->group |= read_mask; if (mode & S_IWGRP) - *group |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->group |= write_mask; if (mode & S_IXGRP) - *group |= ACE_EXECUTE; + masks->group |= execute_mask; - *everyone = ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| + masks->everyone = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| ACE_SYNCHRONIZE; if (mode & S_IROTH) - *everyone |= ACE_READ_DATA; + masks->everyone |= read_mask; if (mode & S_IWOTH) - *everyone |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->everyone |= write_mask; if (mode & S_IXOTH) - *everyone |= ACE_EXECUTE; + masks->everyone |= execute_mask; } int -acl_trivial_create(mode_t mode, ace_t **acl, int *count) +acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count) { - uint32_t deny1, deny2; - uint32_t allow0; - uint32_t owner, group, everyone; - int index = 0; + int index = 0; int error; + trivial_acl_t masks; *count = 3; - acl_trivial_access_masks(mode, &allow0, &deny1, &deny2, &owner, &group, - &everyone); + acl_trivial_access_masks(mode, isdir, &masks); - if (allow0) + if (masks.allow0) (*count)++; - if (deny1) + if (masks.deny1) (*count)++; - if (deny2) + if (masks.deny2) (*count)++; if ((error = cacl_malloc((void **)acl, *count * sizeof (ace_t))) != 0) return (error); - if (allow0) { - SET_ACE(acl, index, -1, allow0, ACE_ACCESS_ALLOWED_ACE_TYPE, - ACE_OWNER); + if (masks.allow0) { + SET_ACE(acl, index, -1, masks.allow0, + ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); } - if (deny1) { - SET_ACE(acl, index, -1, deny1, ACE_ACCESS_DENIED_ACE_TYPE, - ACE_OWNER); + if (masks.deny1) { + SET_ACE(acl, index, -1, masks.deny1, + ACE_ACCESS_DENIED_ACE_TYPE, ACE_OWNER); } - if (deny2) { - SET_ACE(acl, index, -1, deny2, ACE_ACCESS_DENIED_ACE_TYPE, - ACE_GROUP|ACE_IDENTIFIER_GROUP); + if (masks.deny2) { + SET_ACE(acl, index, -1, masks.deny2, + ACE_ACCESS_DENIED_ACE_TYPE, ACE_GROUP|ACE_IDENTIFIER_GROUP); } - SET_ACE(acl, index, -1, owner, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); - SET_ACE(acl, index, -1, group, ACE_ACCESS_ALLOWED_ACE_TYPE, + SET_ACE(acl, index, -1, masks.owner, ACE_ACCESS_ALLOWED_ACE_TYPE, + ACE_OWNER); + SET_ACE(acl, index, -1, masks.group, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_IDENTIFIER_GROUP|ACE_GROUP); - SET_ACE(acl, index, -1, everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, + SET_ACE(acl, index, -1, masks.everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_EVERYONE); return (0); Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.h =================================================================== --- sys/cddl/contrib/opensolaris/common/acl/acl_common.h (revision 223325) +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #ifndef _ACL_COMMON_H @@ -33,7 +34,14 @@ extern "C" { #endif -extern ace_t trivial_acl[6]; +typedef struct trivial_acl { + uint32_t allow0; /* allow mask for bits only in owner */ + uint32_t deny1; /* deny mask for bits not in owner */ + uint32_t deny2; /* deny mask for bits not in group */ + uint32_t owner; /* allow mask matching mode */ + uint32_t group; /* allow mask matching mode */ + uint32_t everyone; /* allow mask matching mode */ +} trivial_acl_t; extern int acltrivial(const char *); extern void adjust_ace_pair(ace_t *pair, mode_t mode); @@ -45,14 +53,14 @@ #if !defined(_KERNEL) extern acl_t *acl_alloc(acl_type_t); extern void acl_free(acl_t *aclp); -extern int acl_translate(acl_t *aclp, int target_flavor, - int isdir, uid_t owner, gid_t group); +extern int acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, + uid_t owner, gid_t group); #endif /* !_KERNEL */ void ksort(caddr_t v, int n, int s, int (*f)()); int cmp2acls(void *a, void *b); -int acl_trivial_create(mode_t mode, ace_t **acl, int *count); -void acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyone); +int acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count); +void acl_trivial_access_masks(mode_t mode, boolean_t isdir, + trivial_acl_t *masks); #ifdef __cplusplus } Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c =================================================================== --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) @@ -104,6 +104,13 @@ { NULL } }; + static zprop_index_t acl_mode_table[] = { + { "discard", ZFS_ACL_DISCARD }, + { "groupmask", ZFS_ACL_GROUPMASK }, + { "passthrough", ZFS_ACL_PASSTHROUGH }, + { NULL } + }; + static zprop_index_t acl_inherit_table[] = { { "discard", ZFS_ACL_DISCARD }, { "noallow", ZFS_ACL_NOALLOW }, @@ -207,6 +214,9 @@ zprop_register_index(ZFS_PROP_SNAPDIR, "snapdir", ZFS_SNAPDIR_HIDDEN, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, "hidden | visible", "SNAPDIR", snapdir_table); + zprop_register_index(ZFS_PROP_ACLMODE, "aclmode", ZFS_ACL_DISCARD, + PROP_INHERIT, ZFS_TYPE_FILESYSTEM, + "discard | groupmask | passthrough", "ACLMODE", acl_mode_table); zprop_register_index(ZFS_PROP_ACLINHERIT, "aclinherit", ZFS_ACL_RESTRICTED, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, "discard | noallow | restricted | passthrough | passthrough-x", @@ -370,13 +380,6 @@ zprop_register_hidden(ZFS_PROP_OBJSETID, "objsetid", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "OBJSETID"); - /* - * Property to be removed once libbe is integrated - */ - zprop_register_hidden(ZFS_PROP_PRIVATE, "priv_prop", - PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_FILESYSTEM, - "PRIV_PROP"); - /* oddball properties */ zprop_register_impl(ZFS_PROP_CREATION, "creation", PROP_TYPE_NUMBER, 0, NULL, PROP_READONLY, ZFS_TYPE_DATASET, Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -3223,7 +3223,8 @@ uint64_t acl_obj; new_mode = (pmode & S_IFMT) | (vap->va_mode & ~S_IFMT); - zfs_acl_chmod_setattr(zp, &aclp, new_mode); + if (err = zfs_acl_chmod_setattr(zp, &aclp, new_mode)) + goto out; mutex_enter(&zp->z_lock); if (!zp->z_is_sa && ((acl_obj = zfs_external_acl(zp)) != 0)) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (working copy) @@ -364,6 +364,14 @@ } static void +acl_mode_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + + zfsvfs->z_acl_mode = newval; +} + +static void acl_inherit_changed_cb(void *arg, uint64_t newval) { zfsvfs_t *zfsvfs = arg; @@ -488,6 +496,8 @@ error = error ? error : dsl_prop_register(ds, "snapdir", snapdir_changed_cb, zfsvfs); error = error ? error : dsl_prop_register(ds, + "aclmode", acl_mode_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs); error = error ? error : dsl_prop_register(ds, "vscan", vscan_changed_cb, zfsvfs); @@ -525,6 +535,7 @@ (void) dsl_prop_unregister(ds, "setuid", setuid_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "exec", exec_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs); + (void) dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "vscan", vscan_changed_cb, zfsvfs); @@ -1202,6 +1213,9 @@ VERIFY(dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs) == 0); + VERIFY(dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, + zfsvfs) == 0); + VERIFY(dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs) == 0); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -1327,76 +1328,9 @@ return (sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); } -/* - * Update access mask for prepended ACE - * - * This applies the "groupmask" value for aclmode property. - */ static void -zfs_acl_prepend_fixup(zfs_acl_t *aclp, void *acep, void *origacep, - mode_t mode, uint64_t owner) +zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t trim, zfs_acl_t *aclp) { - int rmask, wmask, xmask; - int user_ace; - uint16_t aceflags; - uint32_t origmask, acepmask; - uint64_t fuid; - - aceflags = aclp->z_ops.ace_flags_get(acep); - fuid = aclp->z_ops.ace_who_get(acep); - origmask = aclp->z_ops.ace_mask_get(origacep); - acepmask = aclp->z_ops.ace_mask_get(acep); - - user_ace = (!(aceflags & - (ACE_OWNER|ACE_GROUP|ACE_IDENTIFIER_GROUP))); - - if (user_ace && (fuid == owner)) { - rmask = S_IRUSR; - wmask = S_IWUSR; - xmask = S_IXUSR; - } else { - rmask = S_IRGRP; - wmask = S_IWGRP; - xmask = S_IXGRP; - } - - if (origmask & ACE_READ_DATA) { - if (mode & rmask) { - acepmask &= ~ACE_READ_DATA; - } else { - acepmask |= ACE_READ_DATA; - } - } - - if (origmask & ACE_WRITE_DATA) { - if (mode & wmask) { - acepmask &= ~ACE_WRITE_DATA; - } else { - acepmask |= ACE_WRITE_DATA; - } - } - - if (origmask & ACE_APPEND_DATA) { - if (mode & wmask) { - acepmask &= ~ACE_APPEND_DATA; - } else { - acepmask |= ACE_APPEND_DATA; - } - } - - if (origmask & ACE_EXECUTE) { - if (mode & xmask) { - acepmask &= ~ACE_EXECUTE; - } else { - acepmask |= ACE_EXECUTE; - } - } - aclp->z_ops.ace_mask_set(acep, acepmask); -} - -static void -zfs_acl_chmod(zfsvfs_t *zfsvfs, uint64_t mode, zfs_acl_t *aclp) -{ void *acep = NULL; uint64_t who; int new_count, new_bytes; @@ -1407,30 +1341,31 @@ zfs_acl_node_t *newnode; size_t abstract_size = aclp->z_ops.ace_abstract_size(); void *zacep; - uint32_t owner, group, everyone; - uint32_t deny1, deny2, allow0; + boolean_t isdir; + trivial_acl_t masks; new_count = new_bytes = 0; - acl_trivial_access_masks((mode_t)mode, &allow0, &deny1, &deny2, - &owner, &group, &everyone); + isdir = (vtype == VDIR); + acl_trivial_access_masks((mode_t)mode, isdir, &masks); + newnode = zfs_acl_node_alloc((abstract_size * 6) + aclp->z_acl_bytes); zacep = newnode->z_acldata; - if (allow0) { - zfs_set_ace(aclp, zacep, allow0, ALLOW, -1, ACE_OWNER); + if (masks.allow0) { + zfs_set_ace(aclp, zacep, masks.allow0, ALLOW, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; - } if (deny1) { - zfs_set_ace(aclp, zacep, deny1, DENY, -1, ACE_OWNER); + } if (masks.deny1) { + zfs_set_ace(aclp, zacep, masks.deny1, DENY, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; } - if (deny2) { - zfs_set_ace(aclp, zacep, deny2, DENY, -1, OWNING_GROUP); + if (masks.deny2) { + zfs_set_ace(aclp, zacep, masks.deny2, DENY, -1, OWNING_GROUP); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; @@ -1449,10 +1384,17 @@ continue; } + /* + * If this ACL has any inheritable ACEs, mark that in + * the hints (which are later masked into the pflags) + * so create knows to do inheritance. + */ + if (isdir && (inherit_flags & + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + aclp->z_hints |= ZFS_INHERIT_ACE; + if ((type != ALLOW && type != DENY) || (inherit_flags & ACE_INHERIT_ONLY_ACE)) { - if (inherit_flags) - aclp->z_hints |= ZFS_INHERIT_ACE; switch (type) { case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: @@ -1465,20 +1407,13 @@ /* * Limit permissions to be no greater than - * group permissions + * group permissions. + * The "aclinherit" and "aclmode" properties + * affect policy for create and chmod(2), + * respectively. */ - if (type == ALLOW && zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED) { - if (!(mode & S_IRGRP)) - access_mask &= ~ACE_READ_DATA; - if (!(mode & S_IWGRP)) - access_mask &= - ~(ACE_WRITE_DATA|ACE_APPEND_DATA); - if (!(mode & S_IXGRP)) - access_mask &= ~ACE_EXECUTE; - access_mask &= - ~(ACE_WRITE_OWNER|ACE_WRITE_ACL| - ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS); - } + if ((type == ALLOW) && trim) + access_mask &= masks.group; } zfs_set_ace(aclp, zacep, access_mask, type, who, iflags); ace_size = aclp->z_ops.ace_size(acep); @@ -1486,11 +1421,11 @@ new_count++; new_bytes += ace_size; } - zfs_set_ace(aclp, zacep, owner, 0, -1, ACE_OWNER); + zfs_set_ace(aclp, zacep, masks.owner, 0, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, group, 0, -1, OWNING_GROUP); + zfs_set_ace(aclp, zacep, masks.group, 0, -1, OWNING_GROUP); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, everyone, 0, -1, ACE_EVERYONE); + zfs_set_ace(aclp, zacep, masks.everyone, 0, -1, ACE_EVERYONE); new_count += 3; new_bytes += abstract_size * 3; @@ -1502,17 +1437,27 @@ list_insert_tail(&aclp->z_acl, newnode); } -void +int zfs_acl_chmod_setattr(znode_t *zp, zfs_acl_t **aclp, uint64_t mode) { + int error = 0; + mutex_enter(&zp->z_acl_lock); mutex_enter(&zp->z_lock); - *aclp = zfs_acl_alloc(zfs_acl_version_zp(zp)); - (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; - zfs_acl_chmod(zp->z_zfsvfs, mode, *aclp); + if (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_DISCARD) + *aclp = zfs_acl_alloc(zfs_acl_version_zp(zp)); + else + error = zfs_acl_node_read(zp, B_TRUE, aclp, B_TRUE); + + if (error == 0) { + (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; + zfs_acl_chmod(ZTOV(zp)->v_type, mode, + (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), *aclp); + } mutex_exit(&zp->z_lock); mutex_exit(&zp->z_acl_lock); - ASSERT(*aclp); + + return (error); } /* @@ -1764,8 +1709,8 @@ if (acl_ids->z_aclp == NULL) { mutex_enter(&dzp->z_acl_lock); mutex_enter(&dzp->z_lock); - if (!(flag & IS_ROOT_NODE) && (ZTOV(dzp)->v_type == VDIR && - (dzp->z_pflags & ZFS_INHERIT_ACE)) && + if (!(flag & IS_ROOT_NODE) && + (dzp->z_pflags & ZFS_INHERIT_ACE) && !(dzp->z_pflags & ZFS_XATTR)) { VERIFY(0 == zfs_acl_node_read(dzp, B_TRUE, &paclp, B_FALSE)); @@ -1782,7 +1727,9 @@ if (need_chmod) { acl_ids->z_aclp->z_hints |= (vap->va_type == VDIR) ? ZFS_ACL_AUTO_INHERIT : 0; - zfs_acl_chmod(zfsvfs, acl_ids->z_mode, acl_ids->z_aclp); + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, + (zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED), + acl_ids->z_aclp); } } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (working copy) @@ -55,6 +55,7 @@ boolean_t z_fuid_dirty; /* need to sync fuid table ? */ struct zfs_fuid_info *z_fuid_replay; /* fuid info for replay */ zilog_t *z_log; /* intent log pointer */ + uint_t z_acl_mode; /* acl chmod/mode behavior */ uint_t z_acl_inherit; /* acl inheritance behavior */ zfs_case_t z_case; /* case-sense */ boolean_t z_utf8; /* utf8-only */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (working copy) @@ -217,7 +217,7 @@ extern int zfs_zaccess_rwx(struct znode *, mode_t, int, cred_t *); extern int zfs_zaccess_unix(struct znode *, mode_t, cred_t *); extern int zfs_acl_access(struct znode *, int, cred_t *); -void zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); +int zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); int zfs_zaccess_delete(struct znode *, struct znode *, cred_t *); int zfs_zaccess_rename(struct znode *, struct znode *, struct znode *, struct znode *, cred_t *cr); Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) @@ -89,7 +89,7 @@ ZFS_PROP_READONLY, ZFS_PROP_ZONED, ZFS_PROP_SNAPDIR, - ZFS_PROP_PRIVATE, /* not exposed to user, temporary */ + ZFS_PROP_ACLMODE, ZFS_PROP_ACLINHERIT, ZFS_PROP_CREATETXG, /* not exposed to the user */ ZFS_PROP_NAME, /* not exposed to the user */ --------------010907030303010503000607-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:16:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 014BA106566C; Mon, 20 Jun 2011 09:16:58 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 541748FC15; Mon, 20 Jun 2011 09:16:57 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AED68179252; Mon, 20 Jun 2011 11:16:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id UltTu+JW96Z7; Mon, 20 Jun 2011 11:16:54 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id BE1F4179215; Mon, 20 Jun 2011 11:16:50 +0200 (CEST) Message-ID: <4DFF1002.3070402@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:50 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------070809000803040501070106" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:58 -0000 This is a multi-part message in MIME format. --------------070809000803040501070106 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit zfs tries to allocate blocks evenly across all devices. This means when devices are imbalanced zfs will lots of CPU searching for space on devices which tend to be pretty full. https://www.illumos.org/issues/1051 https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 If there are no objections, I will commit this to -HEAD -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------070809000803040501070106 Content-Type: text/x-patch; name="13379.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13379.patch" Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c =================================================================== --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 223325) +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* @@ -5137,6 +5138,7 @@ */ kernel_init(FREAD | FWRITE); VERIFY(spa_open(zs->zs_pool, &spa, FTAG) == 0); + spa->spa_debug = B_TRUE; zs->zs_spa = spa; spa->spa_dedup_ditto = 2 * ZIO_DEDUPDITTO_MIN; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -1674,3 +1675,9 @@ return (0); } + +boolean_t +spa_debug_enabled(spa_t *spa) +{ + return (spa->spa_debug); +} Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -30,10 +31,29 @@ #include #include +/* + * Allow allocations to switch to gang blocks quickly. We do this to + * avoid having to load lots of space_maps in a given txg. There are, + * however, some cases where we want to avoid "fast" ganging and instead + * we want to do an exhaustive search of all metaslabs on this device. + * Currently we don't allow any gang or dump device related allocations + * to "fast" gang. + */ +#define CAN_FASTGANG(flags) \ + (!((flags) & (METASLAB_GANG_CHILD | METASLAB_GANG_HEADER | \ + METASLAB_GANG_AVOID))) + uint64_t metaslab_aliquot = 512ULL << 10; uint64_t metaslab_gang_bang = SPA_MAXBLOCKSIZE + 1; /* force gang blocks */ /* + * This value defines the number of allowed allocation failures per vdev. + * If a device reaches this threshold in a given txg then we consider skipping + * allocations on that device. + */ +int zfs_mg_alloc_failures; + +/* * Metaslab debugging: when set, keeps all space maps in core to verify frees. */ static int metaslab_debug = 0; @@ -671,7 +691,7 @@ metaslab_ndf_fragmented }; -space_map_ops_t *zfs_metaslab_ops = &metaslab_ndf_ops; +space_map_ops_t *zfs_metaslab_ops = &metaslab_df_ops; /* * ========================================================================== @@ -844,7 +864,7 @@ } static int -metaslab_activate(metaslab_t *msp, uint64_t activation_weight, uint64_t size) +metaslab_activate(metaslab_t *msp, uint64_t activation_weight) { metaslab_group_t *mg = msp->ms_group; space_map_t *sm = &msp->ms_map; @@ -877,13 +897,6 @@ mutex_exit(&mg->mg_lock); } - /* - * If we were able to load the map then make sure - * that this map is still able to satisfy our request. - */ - if (msp->ms_weight < size) - return (ENOSPC); - metaslab_group_sort(msp->ms_group, msp, msp->ms_weight | activation_weight); } @@ -1099,6 +1112,7 @@ metaslab_sync_reassess(metaslab_group_t *mg) { vdev_t *vd = mg->mg_vd; + int64_t failures = mg->mg_alloc_failures; /* * Re-evaluate all metaslabs which have lower offsets than the @@ -1115,6 +1129,8 @@ mutex_exit(&msp->ms_lock); } + atomic_add_64(&mg->mg_alloc_failures, -failures); + /* * Prefetch the next potential metaslabs */ @@ -1139,9 +1155,10 @@ } static uint64_t -metaslab_group_alloc(metaslab_group_t *mg, uint64_t size, uint64_t txg, - uint64_t min_distance, dva_t *dva, int d) +metaslab_group_alloc(metaslab_group_t *mg, uint64_t psize, uint64_t asize, + uint64_t txg, uint64_t min_distance, dva_t *dva, int d, int flags) { + spa_t *spa = mg->mg_vd->vdev_spa; metaslab_t *msp = NULL; uint64_t offset = -1ULL; avl_tree_t *t = &mg->mg_metaslab_tree; @@ -1162,11 +1179,17 @@ mutex_enter(&mg->mg_lock); for (msp = avl_first(t); msp; msp = AVL_NEXT(t, msp)) { - if (msp->ms_weight < size) { + if (msp->ms_weight < asize) { + spa_dbgmsg(spa, "%s: failed to meet weight " + "requirement: vdev %llu, txg %llu, mg %p, " + "msp %p, psize %llu, asize %llu, " + "failures %llu, weight %llu", + spa_name(spa), mg->mg_vd->vdev_id, txg, + mg, msp, psize, asize, + mg->mg_alloc_failures, msp->ms_weight); mutex_exit(&mg->mg_lock); return (-1ULL); } - was_active = msp->ms_weight & METASLAB_ACTIVE_MASK; if (activation_weight == METASLAB_WEIGHT_PRIMARY) break; @@ -1185,6 +1208,25 @@ if (msp == NULL) return (-1ULL); + /* + * If we've already reached the allowable number of failed + * allocation attempts on this metaslab group then we + * consider skipping it. We skip it only if we're allowed + * to "fast" gang, the physical size is larger than + * a gang block, and we're attempting to allocate from + * the primary metaslab. + */ + if (mg->mg_alloc_failures > zfs_mg_alloc_failures && + CAN_FASTGANG(flags) && psize > SPA_GANGBLOCKSIZE && + activation_weight == METASLAB_WEIGHT_PRIMARY) { + spa_dbgmsg(spa, "%s: skipping metaslab group: " + "vdev %llu, txg %llu, mg %p, psize %llu, " + "asize %llu, failures %llu", spa_name(spa), + mg->mg_vd->vdev_id, txg, mg, psize, asize, + mg->mg_alloc_failures); + return (-1ULL); + } + mutex_enter(&msp->ms_lock); /* @@ -1193,7 +1235,7 @@ * another thread may have changed the weight while we * were blocked on the metaslab lock. */ - if (msp->ms_weight < size || (was_active && + if (msp->ms_weight < asize || (was_active && !(msp->ms_weight & METASLAB_ACTIVE_MASK) && activation_weight == METASLAB_WEIGHT_PRIMARY)) { mutex_exit(&msp->ms_lock); @@ -1208,14 +1250,16 @@ continue; } - if (metaslab_activate(msp, activation_weight, size) != 0) { + if (metaslab_activate(msp, activation_weight) != 0) { mutex_exit(&msp->ms_lock); continue; } - if ((offset = space_map_alloc(&msp->ms_map, size)) != -1ULL) + if ((offset = space_map_alloc(&msp->ms_map, asize)) != -1ULL) break; + atomic_inc_64(&mg->mg_alloc_failures); + metaslab_passivate(msp, space_map_maxsize(&msp->ms_map)); mutex_exit(&msp->ms_lock); @@ -1224,7 +1268,7 @@ if (msp->ms_allocmap[txg & TXG_MASK].sm_space == 0) vdev_dirty(mg->mg_vd, VDD_METASLAB, msp, txg); - space_map_add(&msp->ms_allocmap[txg & TXG_MASK], offset, size); + space_map_add(&msp->ms_allocmap[txg & TXG_MASK], offset, asize); mutex_exit(&msp->ms_lock); @@ -1351,7 +1395,8 @@ asize = vdev_psize_to_asize(vd, psize); ASSERT(P2PHASE(asize, 1ULL << vd->vdev_ashift) == 0); - offset = metaslab_group_alloc(mg, asize, txg, distance, dva, d); + offset = metaslab_group_alloc(mg, psize, asize, txg, distance, + dva, d, flags); if (offset != -1ULL) { /* * If we've just selected this metaslab group, @@ -1363,18 +1408,24 @@ vdev_stat_t *vs = &vd->vdev_stat; int64_t vu, cu; - /* - * Determine percent used in units of 0..1024. - * (This is just to avoid floating point.) - */ - vu = (vs->vs_alloc << 10) / (vs->vs_space + 1); - cu = (mc->mc_alloc << 10) / (mc->mc_space + 1); + vu = (vs->vs_alloc * 100) / (vs->vs_space + 1); + cu = (mc->mc_alloc * 100) / (mc->mc_space + 1); /* - * Bias by at most +/- 25% of the aliquot. + * Calculate how much more or less we should + * try to allocate from this device during + * this iteration around the rotor. + * For example, if a device is 80% full + * and the pool is 20% full then we should + * reduce allocations by 60% on this device. + * + * mg_bias = (20 - 80) * 512K / 100 = -307K + * + * This reduces allocations by 307K for this + * iteration. */ mg->mg_bias = ((cu - vu) * - (int64_t)mg->mg_aliquot) / (1024 * 4); + (int64_t)mg->mg_aliquot) / 100; } if (atomic_add_64_nv(&mc->mc_aliquot, asize) >= @@ -1488,7 +1539,7 @@ mutex_enter(&msp->ms_lock); if ((txg != 0 && spa_writeable(spa)) || !msp->ms_map.sm_loaded) - error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY, 0); + error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY); if (error == 0 && !space_map_contains(&msp->ms_map, offset, size)) error = ENOENT; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_METASLAB_H @@ -47,6 +48,8 @@ #define METASLAB_HINTBP_FAVOR 0x0 #define METASLAB_HINTBP_AVOID 0x1 #define METASLAB_GANG_HEADER 0x2 +#define METASLAB_GANG_CHILD 0x4 +#define METASLAB_GANG_AVOID 0x8 extern int metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint64_t psize, blkptr_t *bp, int ncopies, uint64_t txg, blkptr_t *hintbp, int flags); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_SPA_IMPL_H @@ -196,6 +197,7 @@ kcondvar_t spa_suspend_cv; /* notification of resume */ uint8_t spa_suspended; /* pool is suspended */ uint8_t spa_claiming; /* pool is doing zil_claim() */ + boolean_t spa_debug; /* debug enabled? */ boolean_t spa_is_root; /* pool is root */ int spa_minref; /* num refs when first opened */ int spa_mode; /* FREAD | FWRITE */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (working copy) @@ -21,6 +21,7 @@ /* * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_METASLAB_IMPL_H @@ -52,6 +53,7 @@ avl_tree_t mg_metaslab_tree; uint64_t mg_aliquot; uint64_t mg_bonus_area; + uint64_t mg_alloc_failures; int64_t mg_bias; int64_t mg_activation_count; metaslab_class_t *mg_class; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_SPA_H @@ -698,6 +699,13 @@ #define dprintf_bp(bp, fmt, ...) #endif +extern boolean_t spa_debug_enabled(spa_t *spa); +#define spa_dbgmsg(spa, ...) \ +{ \ + if (spa_debug_enabled(spa)) \ + zfs_dbgmsg(__VA_ARGS__); \ +} + extern int spa_mode_global; /* mode, e.g. FREAD | FWRITE */ #ifdef __cplusplus Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -85,6 +86,7 @@ #ifdef _KERNEL extern vmem_t *zio_alloc_arena; #endif +extern int zfs_mg_alloc_failures; /* * An allocating zio is one that either currently has the DVA allocate @@ -160,6 +162,12 @@ zio_data_buf_cache[c - 1] = zio_data_buf_cache[c]; } + /* + * The zio write taskqs have 1 thread per cpu, allow 1/2 of the taskqs + * to fail 3 times per txg or 8 failures, whichever is greater. + */ + zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); + zio_inject_init(); } @@ -2135,6 +2143,7 @@ metaslab_class_t *mc = spa_normal_class(spa); blkptr_t *bp = zio->io_bp; int error; + int flags = 0; if (zio->io_gang_leader == NULL) { ASSERT(zio->io_child_type > ZIO_CHILD_GANG); @@ -2147,10 +2156,21 @@ ASSERT3U(zio->io_prop.zp_copies, <=, spa_max_replication(spa)); ASSERT3U(zio->io_size, ==, BP_GET_PSIZE(bp)); + /* + * The dump device does not support gang blocks so allocation on + * behalf of the dump device (i.e. ZIO_FLAG_NODATA) must avoid + * the "fast" gang feature. + */ + flags |= (zio->io_flags & ZIO_FLAG_NODATA) ? METASLAB_GANG_AVOID : 0; + flags |= (zio->io_flags & ZIO_FLAG_GANG_CHILD) ? + METASLAB_GANG_CHILD : 0; error = metaslab_alloc(spa, mc, zio->io_size, bp, - zio->io_prop.zp_copies, zio->io_txg, NULL, 0); + zio->io_prop.zp_copies, zio->io_txg, NULL, flags); if (error) { + spa_dbgmsg(spa, "%s: metaslab allocation failure: zio %p, " + "size %llu, error %d", spa_name(spa), zio, zio->io_size, + error); if (error == ENOSPC && zio->io_size > SPA_MINBLOCKSIZE) return (zio_write_gang_block(zio)); zio->io_error = error; --------------070809000803040501070106-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:17:27 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3420D106564A; Mon, 20 Jun 2011 09:17:27 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id AEAC18FC08; Mon, 20 Jun 2011 09:17:26 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 341721792E4; Mon, 20 Jun 2011 11:17:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vFgJzTwso-+g; Mon, 20 Jun 2011 11:17:22 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id AB5841792D4; Mon, 20 Jun 2011 11:17:21 +0200 (CEST) Message-ID: <4DFF1021.8050706@FreeBSD.org> Date: Mon, 20 Jun 2011 11:17:21 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------050704070709090909090007" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13387 (refcompressratio property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:17:27 -0000 This is a multi-part message in MIME format. --------------050704070709090909090007 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Add a "REFCOMPRESSRATIO" property, which is the compression ratio based on data referenced. For snapshots, this is the same as COMPRESSRATIO, but for filesystems/volumes, the COMPRESSRATIO is based on the data "USED" (ie, includes blocks in children, but not blocks shared with the origin). This is needed to figure out how much space a filesystem would use if it were not compressed (ignoring snapshots). https://www.illumos.org/issues/1092 https://www.illumos.org/projects/illumos-gate/repository/revisions/13387 If there are no objections, I will commit this to -HEAD -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------050704070709090909090007 Content-Type: text/x-patch; name="13387.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13387.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) @@ -6,6 +6,7 @@ .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License. You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. .\" See the License for the specific language governing permissions and limitations under the License. When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with .\" the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner] +.\" Copyright 2011 by Delphix. All rights reserved. .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" .SH NAME zfs \- configures ZFS file systems @@ -389,7 +390,7 @@ .ad .sp .6 .RS 4n -The compression ratio achieved for this dataset, expressed as a multiplier. Compression can be turned on by running: \fBzfs set compression=on \fIdataset\fR\fR. The default value is \fBoff\fR. +For non-snapshots, the compression ratio achieved for the \fBused\fR space of this dataset, expressed as a multiplier. The \fBused\fR property includes descendant datasets, and, for clones, does not include the space shared with the origin snapshot. For snapshots, the \fBcompressratio\fR is the same as the \fBrefcompressratio\fR property. Compression can be turned on by running: \fBzfs set compression=on \fIdataset\fR\fR. The default value is \fBoff\fR. .RE .sp @@ -453,6 +454,17 @@ .ne 2 .mk .na +\fB\fBrefcompressratio\fR\fR +.ad +.sp .6 +.RS 4n +The compression ratio achieved for the \fBreferenced\fR space of this dataset, expressed as a multiplier. See also the \fBcompressratio\fR property. +.RE + +.sp +.ne 2 +.mk +.na \fB\fBtype\fR\fR .ad .sp .6 @@ -1278,7 +1290,7 @@ Force an unmount of any file systems using the \fBunmount -f\fR command. This option has no effect on non-file systems or unmounted file systems. .RE -Extreme care should be taken when applying either the \fB-r\fR or the \fB-f\fR options, as they can destroy large portions of a pool and cause unexpected behavior for mounted file systems in use. +Extreme care should be taken when applying either the \fB-r\fR or the \fB-R\fR options, as they can destroy large portions of a pool and cause unexpected behavior for mounted file systems in use. .RE .sp Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 223325) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -22,6 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -2038,6 +2039,7 @@ } break; + case ZFS_PROP_REFRATIO: case ZFS_PROP_COMPRESSRATIO: if (get_numeric_property(zhp, prop, src, &source, &val) != 0) return (-1); Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c =================================================================== --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -311,6 +312,9 @@ zprop_register_number(ZFS_PROP_COMPRESSRATIO, "compressratio", 0, PROP_READONLY, ZFS_TYPE_DATASET, "<1.00x or higher if compressed>", "RATIO"); + zprop_register_number(ZFS_PROP_REFRATIO, "refcompressratio", 0, + PROP_READONLY, ZFS_TYPE_DATASET, + "<1.00x or higher if compressed>", "REFRATIO"); zprop_register_number(ZFS_PROP_VOLBLOCKSIZE, "volblocksize", ZVOL_DEFAULT_BLOCKSIZE, PROP_ONETIME, ZFS_TYPE_VOLUME, "512 to 128k, power of 2", "VOLBLOCK"); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -2150,7 +2151,7 @@ void dsl_dataset_stats(dsl_dataset_t *ds, nvlist_t *nv) { - uint64_t refd, avail, uobjs, aobjs; + uint64_t refd, avail, uobjs, aobjs, ratio; dsl_dir_stats(ds->ds_dir, nv); @@ -2177,6 +2178,11 @@ dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_DEFER_DESTROY, DS_IS_DEFER_DESTROY(ds) ? 1 : 0); + ratio = ds->ds_phys->ds_compressed_bytes == 0 ? 100 : + (ds->ds_phys->ds_uncompressed_bytes * 100 / + ds->ds_phys->ds_compressed_bytes); + dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_REFRATIO, ratio); + if (ds->ds_phys->ds_next_snap_obj) { /* * This is a snapshot; override the dd's space used with @@ -2184,10 +2190,7 @@ */ dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_USED, ds->ds_phys->ds_unique_bytes); - dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_COMPRESSRATIO, - ds->ds_phys->ds_compressed_bytes == 0 ? 100 : - (ds->ds_phys->ds_uncompressed_bytes * 100 / - ds->ds_phys->ds_compressed_bytes)); + dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_COMPRESSRATIO, ratio); } } Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) @@ -21,6 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -124,6 +125,7 @@ ZFS_PROP_DEDUP, ZFS_PROP_MLSLABEL, ZFS_PROP_SYNC, + ZFS_PROP_REFRATIO, ZFS_NUM_PROPS } zfs_prop_t; --------------050704070709090909090007-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 29 17:49:12 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00E2D106566B for ; Wed, 29 Jun 2011 17:49:12 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 9672F8FC16 for ; Wed, 29 Jun 2011 17:49:11 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BD2125CD; Wed, 29 Jun 2011 18:33:35 +0200 (CEST) Date: Wed, 29 Jun 2011 19:42:09 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110629174209.GB1741@garage.freebsd.pl> References: <4DFF1002.3070402@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="b5gNqxB1S1yM7hjW" Content-Disposition: inline In-Reply-To: <4DFF1002.3070402@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2011 17:49:12 -0000 --b5gNqxB1S1yM7hjW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 20, 2011 at 11:16:50AM +0200, Martin Matuska wrote: > zfs tries to allocate blocks evenly across all devices. This means when > devices are imbalanced zfs will lots of CPU searching for space on devices > which tend to be pretty full. >=20 > https://www.illumos.org/issues/1051 > https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 >=20 > If there are no objections, I will commit this to -HEAD Some comments inline. > @@ -5137,6 +5138,7 @@ > */ > kernel_init(FREAD | FWRITE); > VERIFY(spa_open(zs->zs_pool, &spa, FTAG) =3D=3D 0); > + spa->spa_debug =3D B_TRUE; > zs->zs_spa =3D spa; > =20 > spa->spa_dedup_ditto =3D 2 * ZIO_DEDUPDITTO_MIN; Do we want spa debug to be enabled by default? If we don't then please enable it only when DEBUG is defined. > /* > + * This value defines the number of allowed allocation failures per vdev. > + * If a device reaches this threshold in a given txg then we consider sk= ipping > + * allocations on that device. > + */ > +int zfs_mg_alloc_failures; In FreeBSD we probably want sysctl and loader tunable for this. I think it should be fine to make sysctl read/write, but I haven't looked very closely. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --b5gNqxB1S1yM7hjW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4LY/AACgkQForvXbEpPzS6xwCgu+AnY2WzueibMYyI+B+ifhxX gEAAoJYHHL+8S2yacFqk2p3Qh8TjpE/e =Fy5P -----END PGP SIGNATURE----- --b5gNqxB1S1yM7hjW-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 29 17:54:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B4D4106564A for ; Wed, 29 Jun 2011 17:54:13 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id CAA6B8FC16 for ; Wed, 29 Jun 2011 17:54:11 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 365ED5C0; Wed, 29 Jun 2011 18:27:57 +0200 (CEST) Date: Wed, 29 Jun 2011 19:36:30 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110629173630.GA1741@garage.freebsd.pl> References: <4DFF0FF3.1030308@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: <4DFF0FF3.1030308@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org, Edward Tomasz Napierala Subject: Re: [CFR][ZFS] merge Illumos rev. 13370 (resurrect "aclmode" property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2011 17:54:13 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 20, 2011 at 11:16:35AM +0200, Martin Matuska wrote: > Resurrect "aclmode" property for ZFS. > The Illumos changeset includes Issues submitted by trasz (279, 664). >=20 > https://www.illumos.org/issues/742 > https://www.illumos.org/issues/664 > https://www.illumos.org/issues/279 > https://www.illumos.org/projects/illumos-gate/repository/revisions/13370 >=20 > If there are no objections, I will commit this to -HEAD. No objection from me, although I'd prefer to wait for Edward's opinion. AFAIK he is also fine, but we should wait. It may take a short while as he just become a father:) > Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) > +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) > @@ -6,6 +6,7 @@ > .\" The contents of this file are subject to the terms of the Common Dev= elopment and Distribution License (the "License"). You may not use this fi= le except in compliance with the License. You can obtain a copy of the lice= nse at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensi= ng. > .\" See the License for the specific language governing permissions and= limitations under the License. When distributing Covered Code, include thi= s CDDL HEADER in each file and include the License file at usr/src/OPENSOLA= RIS.LICENSE. If applicable, add the following below this CDDL HEADER, with > .\" the fields enclosed by brackets "[]" replaced with your own identify= ing information: Portions Copyright [yyyy] [name of copyright owner] > +.\" Copyright 2011 Nexenta Systems, Inc. All rights reserved. > .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" > .SH NAME > zfs \- configures ZFS file systems > @@ -630,7 +631,7 @@ > .ad > .sp .6 > .RS 4n > -Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file syst= em with an \fBaclmode\fR property of \fBdiscard\fR deletes all \fBACL\fR en= tries that do not represent the mode of the file. An \fBaclmode\fR property= of \fBgroupmask\fR (the default) reduces user or group permissions. The pe= rmissions are reduced, such that they are no greater than the group permiss= ion bits, unless it is a user entry that has the same \fBUID\fR as the owne= r of the file or directory. In this case, the \fBACL\fR permissions are red= uced so that they are no greater than owner permission bits. A file system = with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no chang= es are made to the \fBACL\fR other than generating the necessary \fBACL\fR = entries to represent the new mode of the file or directory. > +Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file syst= em with an \fBaclmode\fR property of \fBdiscard\fR (the default) deletes al= l \fBACL\fR entries that do not represent the mode of the file. An \fBaclmo= de\fR property of \fBgroupmask\fR reduces permissions granted in all \fBALL= OW\fR entries found in the \fBACL\fR such that they are no greater than the= group permissions specified by \fBchmod\fR. A file system with an \fBaclm= ode\fR property of \fBpassthrough\fR indicates that no changes are made to = the \fBACL\fR other than creating or updating the necessary \fBACL\fR entri= es to represent the new mode of the file or directory. > .RE > =20 > .sp > @@ -2685,7 +2686,7 @@ > pool/home/bob readonly off default > pool/home/bob zoned off default > pool/home/bob snapdir hidden default > -pool/home/bob aclmode groupmask default > +pool/home/bob aclmode discard default > pool/home/bob aclinherit restricted default > pool/home/bob canmount on default > pool/home/bob shareiscsi off default > Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/acl/acl_common.c (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.c (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #include > @@ -376,7 +377,7 @@ > * by nfsace, assuming aclent_t -> nfsace semantics. > */ > static uint32_t > -mode_to_ace_access(mode_t mode, int isdir, int isowner, int isallow) > +mode_to_ace_access(mode_t mode, boolean_t isdir, int isowner, int isallo= w) > { > uint32_t access =3D 0; > int haswriteperm =3D 0; > @@ -419,7 +420,7 @@ > access |=3D ACE_DELETE_CHILD; > } > /* exec */ > - if (mode & 01) { > + if (mode & S_IXOTH) { > access |=3D ACE_EXECUTE; > } > =20 > @@ -670,7 +671,7 @@ > } > =20 > static int > -convert_aent_to_ace(aclent_t *aclentp, int aclcnt, int isdir, > +convert_aent_to_ace(aclent_t *aclentp, int aclcnt, boolean_t isdir, > ace_t **retacep, int *retacecnt) > { > ace_t *acep; > @@ -696,7 +697,7 @@ > dfaclcnt =3D aclcnt - i; > } > =20 > - if (dfaclcnt && isdir =3D=3D 0) { > + if (dfaclcnt && !isdir) { > return (EINVAL); > } > =20 > @@ -734,7 +735,7 @@ > } > =20 > static int > -ace_mask_to_mode(uint32_t mask, o_mode_t *modep, int isdir) > +ace_mask_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) > { > int error =3D 0; > o_mode_t mode =3D 0; > @@ -1031,7 +1032,7 @@ > } > =20 > static int > -ace_allow_to_mode(uint32_t mask, o_mode_t *modep, int isdir) > +ace_allow_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) > { > /* ACE_READ_ACL and ACE_READ_ATTRIBUTES must both be set */ > if ((mask & (ACE_READ_ACL | ACE_READ_ATTRIBUTES)) !=3D > @@ -1044,7 +1045,7 @@ > =20 > static int > acevals_to_aent(acevals_t *vals, aclent_t *dest, ace_list_t *list, > - uid_t owner, gid_t group, int isdir) > + uid_t owner, gid_t group, boolean_t isdir) > { > int error; > uint32_t flips =3D ACE_POSIX_SUPPORTED_BITS; > @@ -1084,7 +1085,7 @@ > =20 > static int > ace_list_to_aent(ace_list_t *list, aclent_t **aclentp, int *aclcnt, > - uid_t owner, gid_t group, int isdir) > + uid_t owner, gid_t group, boolean_t isdir) > { > int error =3D 0; > aclent_t *aent, *result =3D NULL; > @@ -1264,7 +1265,7 @@ > static int > ln_ace_to_aent(ace_t *ace, int n, uid_t owner, gid_t group, > aclent_t **aclentp, int *aclcnt, aclent_t **dfaclentp, int *dfaclcnt, > - int isdir) > + boolean_t isdir) > { > int error =3D 0; > ace_t *acep; > @@ -1459,7 +1460,7 @@ > } > =20 > static int > -convert_ace_to_aent(ace_t *acebufp, int acecnt, int isdir, > +convert_ace_to_aent(ace_t *acebufp, int acecnt, boolean_t isdir, > uid_t owner, gid_t group, aclent_t **retaclentp, int *retaclcnt) > { > int error =3D 0; > @@ -1501,7 +1502,7 @@ > =20 > =20 > int > -acl_translate(acl_t *aclp, int target_flavor, int isdir, uid_t owner, > +acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, uid_t own= er, > gid_t group) > { > int aclcnt; > @@ -1573,101 +1574,105 @@ > } > =20 > void > -acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, > - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyon= e) > +acl_trivial_access_masks(mode_t mode, boolean_t isdir, trivial_acl_t *ma= sks) > { > - *deny1 =3D *deny2 =3D *allow0 =3D *group =3D 0; > + uint32_t read_mask =3D ACE_READ_DATA; > + uint32_t write_mask =3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + uint32_t execute_mask =3D ACE_EXECUTE; > =20 > + (void) isdir; /* will need this later */ > + > + masks->deny1 =3D 0; > if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) > - *deny1 |=3D ACE_READ_DATA; > + masks->deny1 |=3D read_mask; > if (!(mode & S_IWUSR) && (mode & (S_IWGRP|S_IWOTH))) > - *deny1 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->deny1 |=3D write_mask; > if (!(mode & S_IXUSR) && (mode & (S_IXGRP|S_IXOTH))) > - *deny1 |=3D ACE_EXECUTE; > + masks->deny1 |=3D execute_mask; > =20 > + masks->deny2 =3D 0; > if (!(mode & S_IRGRP) && (mode & S_IROTH)) > - *deny2 =3D ACE_READ_DATA; > + masks->deny2 |=3D read_mask; > if (!(mode & S_IWGRP) && (mode & S_IWOTH)) > - *deny2 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->deny2 |=3D write_mask; > if (!(mode & S_IXGRP) && (mode & S_IXOTH)) > - *deny2 |=3D ACE_EXECUTE; > + masks->deny2 |=3D execute_mask; > =20 > + masks->allow0 =3D 0; > if ((mode & S_IRUSR) && (!(mode & S_IRGRP) && (mode & S_IROTH))) > - *allow0 |=3D ACE_READ_DATA; > + masks->allow0 |=3D read_mask; > if ((mode & S_IWUSR) && (!(mode & S_IWGRP) && (mode & S_IWOTH))) > - *allow0 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->allow0 |=3D write_mask; > if ((mode & S_IXUSR) && (!(mode & S_IXGRP) && (mode & S_IXOTH))) > - *allow0 |=3D ACE_EXECUTE; > + masks->allow0 |=3D execute_mask; > =20 > - *owner =3D ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| > + masks->owner =3D ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| > ACE_WRITE_NAMED_ATTRS|ACE_READ_ACL|ACE_READ_ATTRIBUTES| > ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE; > if (mode & S_IRUSR) > - *owner |=3D ACE_READ_DATA; > + masks->owner |=3D read_mask; > if (mode & S_IWUSR) > - *owner |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->owner |=3D write_mask; > if (mode & S_IXUSR) > - *owner |=3D ACE_EXECUTE; > + masks->owner |=3D execute_mask; > =20 > - *group =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| > + masks->group =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| > ACE_SYNCHRONIZE; > if (mode & S_IRGRP) > - *group |=3D ACE_READ_DATA; > + masks->group |=3D read_mask; > if (mode & S_IWGRP) > - *group |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->group |=3D write_mask; > if (mode & S_IXGRP) > - *group |=3D ACE_EXECUTE; > + masks->group |=3D execute_mask; > =20 > - *everyone =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| > + masks->everyone =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATT= RS| > ACE_SYNCHRONIZE; > if (mode & S_IROTH) > - *everyone |=3D ACE_READ_DATA; > + masks->everyone |=3D read_mask; > if (mode & S_IWOTH) > - *everyone |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->everyone |=3D write_mask; > if (mode & S_IXOTH) > - *everyone |=3D ACE_EXECUTE; > + masks->everyone |=3D execute_mask; > } > =20 > int > -acl_trivial_create(mode_t mode, ace_t **acl, int *count) > +acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count) > { > - uint32_t deny1, deny2; > - uint32_t allow0; > - uint32_t owner, group, everyone; > - int index =3D 0; > + int index =3D 0; > int error; > + trivial_acl_t masks; > =20 > *count =3D 3; > - acl_trivial_access_masks(mode, &allow0, &deny1, &deny2, &owner, &group, > - &everyone); > + acl_trivial_access_masks(mode, isdir, &masks); > =20 > - if (allow0) > + if (masks.allow0) > (*count)++; > - if (deny1) > + if (masks.deny1) > (*count)++; > - if (deny2) > + if (masks.deny2) > (*count)++; > =20 > if ((error =3D cacl_malloc((void **)acl, *count * sizeof (ace_t))) !=3D= 0) > return (error); > =20 > - if (allow0) { > - SET_ACE(acl, index, -1, allow0, ACE_ACCESS_ALLOWED_ACE_TYPE, > - ACE_OWNER); > + if (masks.allow0) { > + SET_ACE(acl, index, -1, masks.allow0, > + ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); > } > - if (deny1) { > - SET_ACE(acl, index, -1, deny1, ACE_ACCESS_DENIED_ACE_TYPE, > - ACE_OWNER); > + if (masks.deny1) { > + SET_ACE(acl, index, -1, masks.deny1, > + ACE_ACCESS_DENIED_ACE_TYPE, ACE_OWNER); > } > - if (deny2) { > - SET_ACE(acl, index, -1, deny2, ACE_ACCESS_DENIED_ACE_TYPE, > - ACE_GROUP|ACE_IDENTIFIER_GROUP); > + if (masks.deny2) { > + SET_ACE(acl, index, -1, masks.deny2, > + ACE_ACCESS_DENIED_ACE_TYPE, ACE_GROUP|ACE_IDENTIFIER_GROUP); > } > =20 > - SET_ACE(acl, index, -1, owner, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); > - SET_ACE(acl, index, -1, group, ACE_ACCESS_ALLOWED_ACE_TYPE, > + SET_ACE(acl, index, -1, masks.owner, ACE_ACCESS_ALLOWED_ACE_TYPE, > + ACE_OWNER); > + SET_ACE(acl, index, -1, masks.group, ACE_ACCESS_ALLOWED_ACE_TYPE, > ACE_IDENTIFIER_GROUP|ACE_GROUP); > - SET_ACE(acl, index, -1, everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, > + SET_ACE(acl, index, -1, masks.everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, > ACE_EVERYONE); > =20 > return (0); > Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/acl/acl_common.h (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.h (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #ifndef _ACL_COMMON_H > @@ -33,7 +34,14 @@ > extern "C" { > #endif > =20 > -extern ace_t trivial_acl[6]; > +typedef struct trivial_acl { > + uint32_t allow0; /* allow mask for bits only in owner */ > + uint32_t deny1; /* deny mask for bits not in owner */ > + uint32_t deny2; /* deny mask for bits not in group */ > + uint32_t owner; /* allow mask matching mode */ > + uint32_t group; /* allow mask matching mode */ > + uint32_t everyone; /* allow mask matching mode */ > +} trivial_acl_t; > =20 > extern int acltrivial(const char *); > extern void adjust_ace_pair(ace_t *pair, mode_t mode); > @@ -45,14 +53,14 @@ > #if !defined(_KERNEL) > extern acl_t *acl_alloc(acl_type_t); > extern void acl_free(acl_t *aclp); > -extern int acl_translate(acl_t *aclp, int target_flavor, > - int isdir, uid_t owner, gid_t group); > +extern int acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, > + uid_t owner, gid_t group); > #endif /* !_KERNEL */ > void ksort(caddr_t v, int n, int s, int (*f)()); > int cmp2acls(void *a, void *b); > -int acl_trivial_create(mode_t mode, ace_t **acl, int *count); > -void acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *d= eny1, > - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyon= e); > +int acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *c= ount); > +void acl_trivial_access_masks(mode_t mode, boolean_t isdir, > + trivial_acl_t *masks); > =20 > #ifdef __cplusplus > } > Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) > @@ -104,6 +104,13 @@ > { NULL } > }; > =20 > + static zprop_index_t acl_mode_table[] =3D { > + { "discard", ZFS_ACL_DISCARD }, > + { "groupmask", ZFS_ACL_GROUPMASK }, > + { "passthrough", ZFS_ACL_PASSTHROUGH }, > + { NULL } > + }; > + > static zprop_index_t acl_inherit_table[] =3D { > { "discard", ZFS_ACL_DISCARD }, > { "noallow", ZFS_ACL_NOALLOW }, > @@ -207,6 +214,9 @@ > zprop_register_index(ZFS_PROP_SNAPDIR, "snapdir", ZFS_SNAPDIR_HIDDEN, > PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > "hidden | visible", "SNAPDIR", snapdir_table); > + zprop_register_index(ZFS_PROP_ACLMODE, "aclmode", ZFS_ACL_DISCARD, > + PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > + "discard | groupmask | passthrough", "ACLMODE", acl_mode_table); > zprop_register_index(ZFS_PROP_ACLINHERIT, "aclinherit", > ZFS_ACL_RESTRICTED, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > "discard | noallow | restricted | passthrough | passthrough-x", > @@ -370,13 +380,6 @@ > zprop_register_hidden(ZFS_PROP_OBJSETID, "objsetid", PROP_TYPE_NUMBER, > PROP_READONLY, ZFS_TYPE_DATASET, "OBJSETID"); > =20 > - /* > - * Property to be removed once libbe is integrated > - */ > - zprop_register_hidden(ZFS_PROP_PRIVATE, "priv_prop", > - PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_FILESYSTEM, > - "PRIV_PROP"); > - > /* oddball properties */ > zprop_register_impl(ZFS_PROP_CREATION, "creation", PROP_TYPE_NUMBER, 0, > NULL, PROP_READONLY, ZFS_TYPE_DATASET, > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision = 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working c= opy) > @@ -3223,7 +3223,8 @@ > uint64_t acl_obj; > new_mode =3D (pmode & S_IFMT) | (vap->va_mode & ~S_IFMT); > =20 > - zfs_acl_chmod_setattr(zp, &aclp, new_mode); > + if (err =3D zfs_acl_chmod_setattr(zp, &aclp, new_mode)) > + goto out; > =20 > mutex_enter(&zp->z_lock); > if (!zp->z_is_sa && ((acl_obj =3D zfs_external_acl(zp)) !=3D 0)) { > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (revision= 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (working = copy) > @@ -364,6 +364,14 @@ > } > =20 > static void > +acl_mode_changed_cb(void *arg, uint64_t newval) > +{ > + zfsvfs_t *zfsvfs =3D arg; > + > + zfsvfs->z_acl_mode =3D newval; > +} > + > +static void > acl_inherit_changed_cb(void *arg, uint64_t newval) > { > zfsvfs_t *zfsvfs =3D arg; > @@ -488,6 +496,8 @@ > error =3D error ? error : dsl_prop_register(ds, > "snapdir", snapdir_changed_cb, zfsvfs); > error =3D error ? error : dsl_prop_register(ds, > + "aclmode", acl_mode_changed_cb, zfsvfs); > + error =3D error ? error : dsl_prop_register(ds, > "aclinherit", acl_inherit_changed_cb, zfsvfs); > error =3D error ? error : dsl_prop_register(ds, > "vscan", vscan_changed_cb, zfsvfs); > @@ -525,6 +535,7 @@ > (void) dsl_prop_unregister(ds, "setuid", setuid_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "exec", exec_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs); > + (void) dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, > zfsvfs); > (void) dsl_prop_unregister(ds, "vscan", vscan_changed_cb, zfsvfs); > @@ -1202,6 +1213,9 @@ > VERIFY(dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, > zfsvfs) =3D=3D 0); > =20 > + VERIFY(dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, > + zfsvfs) =3D=3D 0); > + > VERIFY(dsl_prop_unregister(ds, "aclinherit", > acl_inherit_changed_cb, zfsvfs) =3D=3D 0); > =20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (revision 22= 3325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (working cop= y) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #include > @@ -1327,76 +1328,9 @@ > return (sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); > } > =20 > -/* > - * Update access mask for prepended ACE > - * > - * This applies the "groupmask" value for aclmode property. > - */ > static void > -zfs_acl_prepend_fixup(zfs_acl_t *aclp, void *acep, void *origacep, > - mode_t mode, uint64_t owner) > +zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t trim, zfs_acl_t *a= clp) > { > - int rmask, wmask, xmask; > - int user_ace; > - uint16_t aceflags; > - uint32_t origmask, acepmask; > - uint64_t fuid; > - > - aceflags =3D aclp->z_ops.ace_flags_get(acep); > - fuid =3D aclp->z_ops.ace_who_get(acep); > - origmask =3D aclp->z_ops.ace_mask_get(origacep); > - acepmask =3D aclp->z_ops.ace_mask_get(acep); > - > - user_ace =3D (!(aceflags & > - (ACE_OWNER|ACE_GROUP|ACE_IDENTIFIER_GROUP))); > - > - if (user_ace && (fuid =3D=3D owner)) { > - rmask =3D S_IRUSR; > - wmask =3D S_IWUSR; > - xmask =3D S_IXUSR; > - } else { > - rmask =3D S_IRGRP; > - wmask =3D S_IWGRP; > - xmask =3D S_IXGRP; > - } > - > - if (origmask & ACE_READ_DATA) { > - if (mode & rmask) { > - acepmask &=3D ~ACE_READ_DATA; > - } else { > - acepmask |=3D ACE_READ_DATA; > - } > - } > - > - if (origmask & ACE_WRITE_DATA) { > - if (mode & wmask) { > - acepmask &=3D ~ACE_WRITE_DATA; > - } else { > - acepmask |=3D ACE_WRITE_DATA; > - } > - } > - > - if (origmask & ACE_APPEND_DATA) { > - if (mode & wmask) { > - acepmask &=3D ~ACE_APPEND_DATA; > - } else { > - acepmask |=3D ACE_APPEND_DATA; > - } > - } > - > - if (origmask & ACE_EXECUTE) { > - if (mode & xmask) { > - acepmask &=3D ~ACE_EXECUTE; > - } else { > - acepmask |=3D ACE_EXECUTE; > - } > - } > - aclp->z_ops.ace_mask_set(acep, acepmask); > -} > - > -static void > -zfs_acl_chmod(zfsvfs_t *zfsvfs, uint64_t mode, zfs_acl_t *aclp) > -{ > void *acep =3D NULL; > uint64_t who; > int new_count, new_bytes; > @@ -1407,30 +1341,31 @@ > zfs_acl_node_t *newnode; > size_t abstract_size =3D aclp->z_ops.ace_abstract_size(); > void *zacep; > - uint32_t owner, group, everyone; > - uint32_t deny1, deny2, allow0; > + boolean_t isdir; > + trivial_acl_t masks; > =20 > new_count =3D new_bytes =3D 0; > =20 > - acl_trivial_access_masks((mode_t)mode, &allow0, &deny1, &deny2, > - &owner, &group, &everyone); > + isdir =3D (vtype =3D=3D VDIR); > =20 > + acl_trivial_access_masks((mode_t)mode, isdir, &masks); > + > newnode =3D zfs_acl_node_alloc((abstract_size * 6) + aclp->z_acl_bytes); > =20 > zacep =3D newnode->z_acldata; > - if (allow0) { > - zfs_set_ace(aclp, zacep, allow0, ALLOW, -1, ACE_OWNER); > + if (masks.allow0) { > + zfs_set_ace(aclp, zacep, masks.allow0, ALLOW, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > - } if (deny1) { > - zfs_set_ace(aclp, zacep, deny1, DENY, -1, ACE_OWNER); > + } if (masks.deny1) { > + zfs_set_ace(aclp, zacep, masks.deny1, DENY, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > } > - if (deny2) { > - zfs_set_ace(aclp, zacep, deny2, DENY, -1, OWNING_GROUP); > + if (masks.deny2) { > + zfs_set_ace(aclp, zacep, masks.deny2, DENY, -1, OWNING_GROUP); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > @@ -1449,10 +1384,17 @@ > continue; > } > =20 > + /* > + * If this ACL has any inheritable ACEs, mark that in > + * the hints (which are later masked into the pflags) > + * so create knows to do inheritance. > + */ > + if (isdir && (inherit_flags & > + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) > + aclp->z_hints |=3D ZFS_INHERIT_ACE; > + > if ((type !=3D ALLOW && type !=3D DENY) || > (inherit_flags & ACE_INHERIT_ONLY_ACE)) { > - if (inherit_flags) > - aclp->z_hints |=3D ZFS_INHERIT_ACE; > switch (type) { > case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: > case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: > @@ -1465,20 +1407,13 @@ > =20 > /* > * Limit permissions to be no greater than > - * group permissions > + * group permissions. > + * The "aclinherit" and "aclmode" properties > + * affect policy for create and chmod(2), > + * respectively. > */ > - if (type =3D=3D ALLOW && zfsvfs->z_acl_inherit =3D=3D ZFS_ACL_RESTRIC= TED) { > - if (!(mode & S_IRGRP)) > - access_mask &=3D ~ACE_READ_DATA; > - if (!(mode & S_IWGRP)) > - access_mask &=3D > - ~(ACE_WRITE_DATA|ACE_APPEND_DATA); > - if (!(mode & S_IXGRP)) > - access_mask &=3D ~ACE_EXECUTE; > - access_mask &=3D > - ~(ACE_WRITE_OWNER|ACE_WRITE_ACL| > - ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS); > - } > + if ((type =3D=3D ALLOW) && trim) > + access_mask &=3D masks.group; > } > zfs_set_ace(aclp, zacep, access_mask, type, who, iflags); > ace_size =3D aclp->z_ops.ace_size(acep); > @@ -1486,11 +1421,11 @@ > new_count++; > new_bytes +=3D ace_size; > } > - zfs_set_ace(aclp, zacep, owner, 0, -1, ACE_OWNER); > + zfs_set_ace(aclp, zacep, masks.owner, 0, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > - zfs_set_ace(aclp, zacep, group, 0, -1, OWNING_GROUP); > + zfs_set_ace(aclp, zacep, masks.group, 0, -1, OWNING_GROUP); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > - zfs_set_ace(aclp, zacep, everyone, 0, -1, ACE_EVERYONE); > + zfs_set_ace(aclp, zacep, masks.everyone, 0, -1, ACE_EVERYONE); > =20 > new_count +=3D 3; > new_bytes +=3D abstract_size * 3; > @@ -1502,17 +1437,27 @@ > list_insert_tail(&aclp->z_acl, newnode); > } > =20 > -void > +int > zfs_acl_chmod_setattr(znode_t *zp, zfs_acl_t **aclp, uint64_t mode) > { > + int error =3D 0; > + > mutex_enter(&zp->z_acl_lock); > mutex_enter(&zp->z_lock); > - *aclp =3D zfs_acl_alloc(zfs_acl_version_zp(zp)); > - (*aclp)->z_hints =3D zp->z_pflags & V4_ACL_WIDE_FLAGS; > - zfs_acl_chmod(zp->z_zfsvfs, mode, *aclp); > + if (zp->z_zfsvfs->z_acl_mode =3D=3D ZFS_ACL_DISCARD) > + *aclp =3D zfs_acl_alloc(zfs_acl_version_zp(zp)); > + else > + error =3D zfs_acl_node_read(zp, B_TRUE, aclp, B_TRUE); > + > + if (error =3D=3D 0) { > + (*aclp)->z_hints =3D zp->z_pflags & V4_ACL_WIDE_FLAGS; > + zfs_acl_chmod(ZTOV(zp)->v_type, mode, > + (zp->z_zfsvfs->z_acl_mode =3D=3D ZFS_ACL_GROUPMASK), *aclp); > + } > mutex_exit(&zp->z_lock); > mutex_exit(&zp->z_acl_lock); > - ASSERT(*aclp); > + > + return (error); > } > =20 > /* > @@ -1764,8 +1709,8 @@ > if (acl_ids->z_aclp =3D=3D NULL) { > mutex_enter(&dzp->z_acl_lock); > mutex_enter(&dzp->z_lock); > - if (!(flag & IS_ROOT_NODE) && (ZTOV(dzp)->v_type =3D=3D VDIR && > - (dzp->z_pflags & ZFS_INHERIT_ACE)) && > + if (!(flag & IS_ROOT_NODE) && > + (dzp->z_pflags & ZFS_INHERIT_ACE) && > !(dzp->z_pflags & ZFS_XATTR)) { > VERIFY(0 =3D=3D zfs_acl_node_read(dzp, B_TRUE, > &paclp, B_FALSE)); > @@ -1782,7 +1727,9 @@ > if (need_chmod) { > acl_ids->z_aclp->z_hints |=3D (vap->va_type =3D=3D VDIR) ? > ZFS_ACL_AUTO_INHERIT : 0; > - zfs_acl_chmod(zfsvfs, acl_ids->z_mode, acl_ids->z_aclp); > + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, > + (zfsvfs->z_acl_inherit =3D=3D ZFS_ACL_RESTRICTED), > + acl_ids->z_aclp); > } > } > =20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (revi= sion 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (work= ing copy) > @@ -55,6 +55,7 @@ > boolean_t z_fuid_dirty; /* need to sync fuid table ? */ > struct zfs_fuid_info *z_fuid_replay; /* fuid info for replay */ > zilog_t *z_log; /* intent log pointer */ > + uint_t z_acl_mode; /* acl chmod/mode behavior */ > uint_t z_acl_inherit; /* acl inheritance behavior */ > zfs_case_t z_case; /* case-sense */ > boolean_t z_utf8; /* utf8-only */ > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (revisio= n 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (working= copy) > @@ -217,7 +217,7 @@ > extern int zfs_zaccess_rwx(struct znode *, mode_t, int, cred_t *); > extern int zfs_zaccess_unix(struct znode *, mode_t, cred_t *); > extern int zfs_acl_access(struct znode *, int, cred_t *); > -void zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); > +int zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); > int zfs_zaccess_delete(struct znode *, struct znode *, cred_t *); > int zfs_zaccess_rename(struct znode *, struct znode *, > struct znode *, struct znode *, cred_t *cr); > Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) > @@ -89,7 +89,7 @@ > ZFS_PROP_READONLY, > ZFS_PROP_ZONED, > ZFS_PROP_SNAPDIR, > - ZFS_PROP_PRIVATE, /* not exposed to user, temporary */ > + ZFS_PROP_ACLMODE, > ZFS_PROP_ACLINHERIT, > ZFS_PROP_CREATETXG, /* not exposed to the user */ > ZFS_PROP_NAME, /* not exposed to the user */ >=20 --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --G4iJoqBmSsgzjUCe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4LYp4ACgkQForvXbEpPzT18QCgzMvS3Qun9iemGIKFLQ0MyqEA mTsAn0Cn7B86/h3dMxxIQSmUnP2XILhJ =RSEU -----END PGP SIGNATURE----- --G4iJoqBmSsgzjUCe-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 6 12:45:16 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E406C106566C; Wed, 6 Jul 2011 12:45:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 7DF988FC0A; Wed, 6 Jul 2011 12:45:16 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A8CF615C525; Wed, 6 Jul 2011 14:45:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Hs+au5mHY7Nx; Wed, 6 Jul 2011 14:45:13 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 13D5F15C51A; Wed, 6 Jul 2011 14:45:13 +0200 (CEST) Message-ID: <4E1458D8.8000602@FreeBSD.org> Date: Wed, 06 Jul 2011 14:45:12 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4DFF1002.3070402@FreeBSD.org> <20110629174209.GB1741@garage.freebsd.pl> In-Reply-To: <20110629174209.GB1741@garage.freebsd.pl> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 12:45:17 -0000 Dňa 29.06.2011 19:42, Pawel Jakub Dawidek wrote / napísal(a): > On Mon, Jun 20, 2011 at 11:16:50AM +0200, Martin Matuska wrote: >> zfs tries to allocate blocks evenly across all devices. This means when >> devices are imbalanced zfs will lots of CPU searching for space on devices >> which tend to be pretty full. >> >> https://www.illumos.org/issues/1051 >> https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 >> >> If there are no objections, I will commit this to -HEAD > Some comments inline. > >> @@ -5137,6 +5138,7 @@ >> */ >> kernel_init(FREAD | FWRITE); >> VERIFY(spa_open(zs->zs_pool, &spa, FTAG) == 0); >> + spa->spa_debug = B_TRUE; >> zs->zs_spa = spa; >> >> spa->spa_dedup_ditto = 2 * ZIO_DEDUPDITTO_MIN; > Do we want spa debug to be enabled by default? If we don't then please > enable it only when DEBUG is defined. > We are here in ztest.c, spa_debug (new boolean) is disabled by default. >> /* >> + * This value defines the number of allowed allocation failures per vdev. >> + * If a device reaches this threshold in a given txg then we consider skipping >> + * allocations on that device. >> + */ >> +int zfs_mg_alloc_failures; > In FreeBSD we probably want sysctl and loader tunable for this. > I think it should be fine to make sysctl read/write, but I haven't > looked very closely. zio.c: This is called in zio_init(), called by spa_init(): /* * The zio write taskqs have 1 thread per cpu, allow 1/2 of the taskqs * to fail 3 times per txg or 8 failures, whichever is greater. */ zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); Looks like it is set on runtime so we have to change the code to make it properly tunable and set a default value. Maye comment out the code above and directly initialize with: int zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); The setup might look like this: TUNABLE_INT("vfs.zfs.mg_alloc_failures", &zfs_mg_alloc_failures); SYSCTL_INT(_vfs_zfs, OID_AUTO, mg_alloc_failures, CTLFLAG_RW, &zfs_mg_alloc_failures, 0, "Number of allowed allocation failures per vdev"); Another question is if it should be under vfs.zfs. or vfs.zfs.vdev. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 12 13:23:40 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 477C3106564A for ; Tue, 12 Jul 2011 13:23:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id D92118FC1B for ; Tue, 12 Jul 2011 13:23:39 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5B7E313F for ; Tue, 12 Jul 2011 15:23:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WA856HSb4ldg for ; Tue, 12 Jul 2011 15:23:37 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id DC7C13134 for ; Tue, 12 Jul 2011 15:23:36 +0200 (CEST) Message-ID: <4E1C4AD8.4080101@FreeBSD.org> Date: Tue, 12 Jul 2011 15:23:36 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Temporary clones not destroyed on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2011 13:23:40 -0000 In kern/157728, I describe a problem (with reproduction steps and a temporary workaround) of temporary clones not destroyed on incremental snapshot receive. This does apparently not happen on OpenIndiana or Solaris. Does anyone have a clue what could cause this bug in our ZFS port? URL:http://www.freebsd.org/cgi/query-pr.cgi?pr=157728 Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Thu Jul 14 01:36:34 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2C9C1065674; Thu, 14 Jul 2011 01:36:34 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 873468FC0C; Thu, 14 Jul 2011 01:36:34 +0000 (UTC) Received: by iyb11 with SMTP id 11so7738816iyb.13 for ; Wed, 13 Jul 2011 18:36:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=2L6xlxMNHlWPcEgQbDnSh8lBkhgaQFMAp4ka3vvSVaE=; b=RohtWt5I0vOfZe06URLWMyeRVbgYepiMVvuQtUaTIcRjRCeEwEg2P5tGmNbtzJgusl mSpoHcHHi6nKieRXG8G/vCAKO5RzCXD7tZYiPYRZTiva4552ryucYnCJdhohJEZUwdJx e2v+gDNP4ET0SWtRXAlEpLwkEjGfwewUTn/YY= Received: by 10.42.221.4 with SMTP id ia4mr2065760icb.111.1310605499111; Wed, 13 Jul 2011 18:04:59 -0700 (PDT) Received: from DataIX.net (adsl-99-181-128-71.dsl.klmzmi.sbcglobal.net [99.181.128.71]) by mx.google.com with ESMTPS id q4sm93768ibb.66.2011.07.13.18.04.57 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 13 Jul 2011 18:04:57 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id p6E14sUW078445 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 Jul 2011 21:04:55 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id p6E14sq5078444; Wed, 13 Jul 2011 21:04:54 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Wed, 13 Jul 2011 21:04:54 -0400 From: Jason Hellenthal To: Martin Matuska Message-ID: <20110714010454.GA73973@DataIX.net> References: <4E1C4AD8.4080101@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pWyiEgJYm5f9v55/" Content-Disposition: inline In-Reply-To: <4E1C4AD8.4080101@FreeBSD.org> Cc: zfs-devel@freebsd.org Subject: Re: Temporary clones not destroyed on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2011 01:36:35 -0000 --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 12, 2011 at 03:23:36PM +0200, Martin Matuska wrote: > In kern/157728, I describe a problem (with reproduction steps and a > temporary workaround) of temporary clones not destroyed on incremental > snapshot receive. > This does apparently not happen on OpenIndiana or Solaris. >=20 > Does anyone have a clue what could cause this bug in our ZFS port? >=20 > URL:http://www.freebsd.org/cgi/query-pr.cgi?pr=3D157728 >=20 This bug may be related to a very similiar one that was reported to me last night. The user was attempting to rename a populated dataset with less than ~50 snapshots (maybe far less) did not get an exact number but his steps to reproduce this is listed below for reference. zfs umount pool/(...)/dataset # This fails, dataset somehow in use. zfs umount -f pool/(...)/dataset # This works, problem waiting here. zfs rename pool/(...)/dataset pool/(...)/dataset2 # Live-Lock There was no coredump at this point that could be obtained and the backtrace information he had was shaky at best. So ... I had him remove all the snapshots before dataset renaming... zfs destroy pool/(...)/dataset@(...) zfs umount pool/(...)/dataset # Not clear if this worked zfs umount -f pool/(...)/dataset # Could have worked. zfs rename pool/(...)/dataset pool/(...)/dataset2 # Worked. He probably could have gone without the (umount) here but I suspect thats why he was using it because the rename probably failed with EBUSY or some error. The backtrace he gave here: (pasted in IRC) translation needed. 25674 100871 zfs initial thread mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_wait_synced+0x85 dsl_sync_task_group_wait+0x128 dsl_sync_task_do+0x5 4 dsl_dir_rename+0x8f dsl_dataset_rename+0x272 zfsdev_ioctl+0xe6 devfs_ioctl_f+0x7b kern_ioctl+0x102 ioctl+0xfd syscallenter+0x1e5 syscall+0x4b Xfast_syscall+0xe2 By the way he explained it, this felt like a locking problem or something not being free'd from something similiar to what we experienced before from the .zfs/* directories causing panics. Martin, He also explained that he was using a version of mfsBSD image to create this system too that he continued to run if its of any help. IIRC this is the same thing that was MFC'd to stable/8. --pWyiEgJYm5f9v55/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJOHkC2AAoJEJBXh4mJ2FR+itIH/0LmPtPD86RwYm6Je4FQ7CKB L/IVLiyZMMIHRqFITYqszRMs8w+6Y26RAAM/UTEpHZm7jLVhRH585av6ZaXDUUZz mcfs8sgePnXNmJVrEktoekoH3Dq22iTpX6nzd2ZX136hsTEBGChl0vB9Os2x/H3d /dTxRqJzWbllg2y2mAGdj+ItvTbkv/epYwHLc4zg4YGF1Guv6sLZUL9z/tCWQPiR fuBNEiFFXD512WsEpZYanCqmWcaK2y21Ad1OeT/C7rYQHUyrBmqghg7jGo7WtbBY /A8HuAdfxTkQWNiDcwlLgjcQhI25QDass24qxItwhgyJ2innDjJHNAMm1/Hju1w= =oI+Z -----END PGP SIGNATURE----- --pWyiEgJYm5f9v55/-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 25 20:26:37 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CC951065674 for ; Mon, 25 Jul 2011 20:26:37 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 8269B8FC1A for ; Mon, 25 Jul 2011 20:26:37 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 4F35214310; Mon, 25 Jul 2011 13:26:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1311625597; bh=K+/6FOxa1iitLik5Rj27a5CYymkNXydcmfDVaYM1rd4=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:Subject: Content-Type; b=zpJFDqpKz0+xXttitcTanLvnaaTaFmqn5bkQAEd3/XXo/iXERODlG6OHB8a0+p5Yz jOD8EvpuXcMrHFncNwbVb4VBkj413vppwPyOdwKwVd7WlCKYx9UOtvlIcK2SCaQSqo dXIK6dRY21nNCJ4x6Mj7CDb3TpbdBEGpDFPkTCkA= Message-ID: <4E2DD17C.60606@delphij.net> Date: Mon, 25 Jul 2011 13:26:36 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------060708020906060603030803" Cc: Subject: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 20:26:37 -0000 This is a multi-part message in MIME format. --------------060708020906060603030803 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Here is a patch that makes libzfs use zfs_ioctl consistently. Some functionality in libzfs calls ioctl() directly with raw ZFS ioctls, which causes some kernel message like: WARNING pid 30031 (initial thread): ioctl sign-extension ioctl ffffffffd5985a3a Objections? I have kept #sun'ed out blocks out. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iQEcBAEBCAAGBQJOLdF8AAoJEATO+BI/yjfBI0AH/04tS03WW0vIW/ywXp/FsHV6 bkkZQw7rjN2dnenHBTg3DNcjYlXpZOgRb/mnuwMsTxm/Xf/RVekfCLGoljwv7NNK FUTozx7jVBvwAhjArvUTsVjpblnm2b+mS9TCWh9BWVhIr5N6sMALZxZkNvVTUVn+ ZXotIaqQPSVNdEC3BuvJsxyHAEutTPF8TvMojKhaRhxTPGGcrjQfVLHHR/YMLhY5 ED/7gJsvSTAi+gmtHWgF9q5oqN9sCE/Juy5XD0HcXok5dZ2cfjYYFacKbbjpXQhn eYj8zSVPna0avCJ0YEivEO5LCEF9MLgTMGFJtV38mRandNdbU5UFNqA+m4WJeiM= =zWDs -----END PGP SIGNATURE----- --------------060708020906060603030803 Content-Type: text/plain; name="zfs-ioctl.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs-ioctl.diff" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6 ZnNfZGF0YXNldC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGNkZGwvY29udHJpYi9vcGVuc29sYXJp cy9saWIvbGliemZzL2NvbW1vbi9saWJ6ZnNfZGF0YXNldC5jCShyZXZpc2lvbiAyMjQzMzcp CisrKyBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvbGliL2xpYnpmcy9jb21tb24vbGliemZz X2RhdGFzZXQuYwkod29ya2luZyBjb3B5KQpAQCAtMjM4Miw2ICsyMzgyLDcgQEAgemZzX3By b3BfZ2V0X3VzZXJxdW90YV9jb21tb24oemZzX2hhbmRsZV90ICp6aHAsIGMKIHsKIAlpbnQg ZXJyOwogCXpmc19jbWRfdCB6YyA9IHsgMCB9OworCWxpYnpmc19oYW5kbGVfdCAqaGRsID0g emhwLT56ZnNfaGRsOwogCiAJKHZvaWQpIHN0cm5jcHkoemMuemNfbmFtZSwgemhwLT56ZnNf bmFtZSwgc2l6ZW9mICh6Yy56Y19uYW1lKSk7CiAKQEAgLTIzOTIsNyArMjM5Myw3IEBAIHpm c19wcm9wX2dldF91c2VycXVvdGFfY29tbW9uKHpmc19oYW5kbGVfdCAqemhwLCBjCiAJaWYg KGVycikKIAkJcmV0dXJuIChlcnIpOwogCi0JZXJyID0gaW9jdGwoemhwLT56ZnNfaGRsLT5s aWJ6ZnNfZmQsIFpGU19JT0NfVVNFUlNQQUNFX09ORSwgJnpjKTsKKwllcnIgPSB6ZnNfaW9j dGwoaGRsLCBaRlNfSU9DX1VTRVJTUEFDRV9PTkUsICZ6Yyk7CiAJaWYgKGVycikKIAkJcmV0 dXJuIChlcnIpOwogCkBAIC0yNDU4LDExICsyNDU5LDEyIEBAIHpmc19kb19saXN0X2lvY3Rs KHpmc19oYW5kbGVfdCAqemhwLCB1bnNpZ25lZCBsb25nCiB7CiAJaW50IHJjOwogCXVpbnQ2 NF90CW9yaWdfY29va2llOworCWxpYnpmc19oYW5kbGVfdCAqaGRsID0gemhwLT56ZnNfaGRs OwogCiAJb3JpZ19jb29raWUgPSB6Yy0+emNfY29va2llOwogdG9wOgogCSh2b2lkKSBzdHJs Y3B5KHpjLT56Y19uYW1lLCB6aHAtPnpmc19uYW1lLCBzaXplb2YgKHpjLT56Y19uYW1lKSk7 Ci0JcmMgPSBpb2N0bCh6aHAtPnpmc19oZGwtPmxpYnpmc19mZCwgYXJnLCB6Yyk7CisJcmMg PSB6ZnNfaW9jdGwoaGRsLCBhcmcsIHpjKTsKIAogCWlmIChyYyA9PSAtMSkgewogCQlzd2l0 Y2ggKGVycm5vKSB7CkBAIC0zODAwLDcgKzM4MDIsNyBAQCB6ZnNfZGVsZWdfc2hhcmVfbmZz KGxpYnpmc19oYW5kbGVfdCAqaGRsLCBjaGFyICpkYQogCXpjLnpjX3NoYXJlLnpfZXhwb3J0 ZGF0YSA9ICh1aW50NjRfdCkodWludHB0cl90KWV4cG9ydDsKIAl6Yy56Y19zaGFyZS56X3No YXJldHlwZSA9IG9wZXJhdGlvbjsKIAl6Yy56Y19zaGFyZS56X3NoYXJlbWF4ID0gc2hhcmVt YXg7Ci0JZXJyb3IgPSBpb2N0bChoZGwtPmxpYnpmc19mZCwgWkZTX0lPQ19TSEFSRSwgJnpj KTsKKwllcnJvciA9IHpmc19pb2N0bChoZGwsIFpGU19JT0NfU0hBUkUsICZ6Yyk7CiAJcmV0 dXJuIChlcnJvcik7CiB9CiAKQEAgLTM5MjQsNiArMzkyNiw3IEBAIHpmc191c2Vyc3BhY2Uo emZzX2hhbmRsZV90ICp6aHAsIHpmc191c2VycXVvdGFfcHJvCiAgICAgemZzX3VzZXJzcGFj ZV9jYl90IGZ1bmMsIHZvaWQgKmFyZykKIHsKIAl6ZnNfY21kX3QgemMgPSB7IDAgfTsKKwls aWJ6ZnNfaGFuZGxlX3QgKmhkbCA9IHpocC0+emZzX2hkbDsKIAlpbnQgZXJyb3I7CiAJemZz X3VzZXJhY2N0X3QgYnVmWzEwMF07CiAKQEAgLTM5MzcsNyArMzk0MCw3IEBAIHpmc191c2Vy c3BhY2UoemZzX2hhbmRsZV90ICp6aHAsIHpmc191c2VycXVvdGFfcHJvCiAJCXpmc191c2Vy YWNjdF90ICp6dWEgPSBidWY7CiAKIAkJemMuemNfbnZsaXN0X2RzdF9zaXplID0gc2l6ZW9m IChidWYpOwotCQllcnJvciA9IGlvY3RsKHpocC0+emZzX2hkbC0+bGliemZzX2ZkLAorCQll cnJvciA9IHpmc19pb2N0bChoZGwsCiAJCSAgICBaRlNfSU9DX1VTRVJTUEFDRV9NQU5ZLCAm emMpOwogCQlpZiAoZXJyb3IgfHwgemMuemNfbnZsaXN0X2RzdF9zaXplID09IDApCiAJCQli cmVhazsKQEAgLTQzMTYsNyArNDMxOSw3IEBAIHpmc19qYWlsKHpmc19oYW5kbGVfdCAqemhw LCBpbnQgamFpbGlkLCBpbnQgYXR0YWNoCiAJemMuemNfamFpbGlkID0gamFpbGlkOwogCiAJ Y21kID0gYXR0YWNoID8gWkZTX0lPQ19KQUlMIDogWkZTX0lPQ19VTkpBSUw7Ci0JaWYgKChy ZXQgPSBpb2N0bChoZGwtPmxpYnpmc19mZCwgY21kLCAmemMpKSAhPSAwKQorCWlmICgocmV0 ID0gemZzX2lvY3RsKGhkbCwgY21kLCAmemMpKSAhPSAwKQogCQl6ZnNfc3RhbmRhcmRfZXJy b3IoaGRsLCBlcnJubywgZXJyYnVmKTsKIAogCXJldHVybiAocmV0KTsK --------------060708020906060603030803-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 12:31:40 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F31C106566B; Tue, 26 Jul 2011 12:31:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id B20778FC16; Tue, 26 Jul 2011 12:31:39 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id DECF4167197; Tue, 26 Jul 2011 14:31:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DuQ-4GajIGZY; Tue, 26 Jul 2011 14:31:33 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 46661167187; Tue, 26 Jul 2011 14:31:33 +0200 (CEST) Message-ID: <4E2EB3A4.60007@FreeBSD.org> Date: Tue, 26 Jul 2011 14:31:32 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li X-Priority: 2 (High) X-Enigmail-Version: 1.2pre Content-Type: multipart/mixed; boundary="------------020005050609060404020702" Cc: zfs-devel@FreeBSD.org Subject: [PATCH] Illumos #883 ZIL reuse during remount can lead to data corruption X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:31:40 -0000 This is a multi-part message in MIME format. --------------020005050609060404020702 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Illumos developers have fixed a problem in ZIL that may lead to data corruption during remount. A detailed description of this problem is available at: https://www.illumos.org/issues/883 In my opinion this issue applies to FreeBSD, too, and we should fix this ASAP. Please review my patch (1:1 merge from illumos, no changes). References: illumos-gate revision: 13380:161b964a0e10 https://github.com/illumos/illumos-gate/commit/c9ba2a43cb76c223d115e021fdabd2c066e020ed -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020005050609060404020702 Content-Type: text/x-patch; name="illumos-zil.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="illumos-zil.patch" Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c =================================================================== --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 224409) +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) @@ -205,6 +205,7 @@ */ typedef struct ztest_ds { objset_t *zd_os; + rwlock_t zd_zilog_lock; zilog_t *zd_zilog; uint64_t zd_seq; ztest_od_t *zd_od; /* debugging aid */ @@ -238,6 +239,7 @@ ztest_func_t ztest_zap; ztest_func_t ztest_zap_parallel; ztest_func_t ztest_zil_commit; +ztest_func_t ztest_zil_remount; ztest_func_t ztest_dmu_read_write_zcopy; ztest_func_t ztest_dmu_objset_create_destroy; ztest_func_t ztest_dmu_prealloc; @@ -273,6 +275,7 @@ { ztest_zap_parallel, 100, &zopt_always }, { ztest_split_pool, 1, &zopt_always }, { ztest_zil_commit, 1, &zopt_incessant }, + { ztest_zil_remount, 1, &zopt_sometimes }, { ztest_dmu_read_write_zcopy, 1, &zopt_often }, { ztest_dmu_objset_create_destroy, 1, &zopt_often }, { ztest_dsl_prop_get_set, 1, &zopt_often }, @@ -986,6 +989,7 @@ zd->zd_seq = 0; dmu_objset_name(os, zd->zd_name); + VERIFY(rwlock_init(&zd->zd_zilog_lock, USYNC_THREAD, NULL) == 0); VERIFY(_mutex_init(&zd->zd_dirobj_lock, USYNC_THREAD, NULL) == 0); for (int l = 0; l < ZTEST_OBJECT_LOCKS; l++) @@ -1965,6 +1969,8 @@ if (ztest_random(2) == 0) io_type = ZTEST_IO_WRITE_TAG; + (void) rw_rdlock(&zd->zd_zilog_lock); + switch (io_type) { case ZTEST_IO_WRITE_TAG: @@ -2000,6 +2006,8 @@ break; } + (void) rw_unlock(&zd->zd_zilog_lock); + umem_free(data, blocksize); } @@ -2054,6 +2062,8 @@ { zilog_t *zilog = zd->zd_zilog; + (void) rw_rdlock(&zd->zd_zilog_lock); + zil_commit(zilog, ztest_random(ZTEST_OBJECTS)); /* @@ -2065,9 +2075,34 @@ ASSERT(zd->zd_seq <= zilog->zl_commit_lr_seq); zd->zd_seq = zilog->zl_commit_lr_seq; mutex_exit(&zilog->zl_lock); + + (void) rw_unlock(&zd->zd_zilog_lock); } /* + * This function is designed to simulate the operations that occur during a + * mount/unmount operation. We hold the dataset across these operations in an + * attempt to expose any implicit assumptions about ZIL management. + */ +/* ARGSUSED */ +void +ztest_zil_remount(ztest_ds_t *zd, uint64_t id) +{ + objset_t *os = zd->zd_os; + + (void) rw_wrlock(&zd->zd_zilog_lock); + + /* zfsvfs_teardown() */ + zil_close(zd->zd_zilog); + + /* zfsvfs_setup() */ + VERIFY(zil_open(os, ztest_get_data) == zd->zd_zilog); + zil_replay(os, zd, ztest_replay_vector); + + (void) rw_unlock(&zd->zd_zilog_lock); +} + +/* * Verify that we can't destroy an active pool, create an existing pool, * or create a pool with a bad vdev spec. */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision 224409) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -567,7 +568,7 @@ if (!list_is_empty(&zilog->zl_lwb_list)) { ASSERT(zh->zh_claim_txg == 0); - ASSERT(!keep_first); + VERIFY(!keep_first); while ((lwb = list_head(&zilog->zl_lwb_list)) != NULL) { list_remove(&zilog->zl_lwb_list, lwb); if (lwb->lwb_buf != NULL) @@ -1668,20 +1669,9 @@ void zil_free(zilog_t *zilog) { - lwb_t *head_lwb; - zilog->zl_stop_sync = 1; - /* - * After zil_close() there should only be one lwb with a buffer. - */ - head_lwb = list_head(&zilog->zl_lwb_list); - if (head_lwb) { - ASSERT(head_lwb == list_tail(&zilog->zl_lwb_list)); - list_remove(&zilog->zl_lwb_list, head_lwb); - zio_buf_free(head_lwb->lwb_buf, head_lwb->lwb_sz); - kmem_cache_free(zil_lwb_cache, head_lwb); - } + ASSERT(list_is_empty(&zilog->zl_lwb_list)); list_destroy(&zilog->zl_lwb_list); avl_destroy(&zilog->zl_vdev_tree); @@ -1721,6 +1711,10 @@ { zilog_t *zilog = dmu_objset_zil(os); + ASSERT(zilog->zl_clean_taskq == NULL); + ASSERT(zilog->zl_get_data == NULL); + ASSERT(list_is_empty(&zilog->zl_lwb_list)); + zilog->zl_get_data = get_data; zilog->zl_clean_taskq = taskq_create("zil_clean", 1, minclsyspri, 2, 2, TASKQ_PREPOPULATE); @@ -1734,7 +1728,7 @@ void zil_close(zilog_t *zilog) { - lwb_t *tail_lwb; + lwb_t *lwb; uint64_t txg = 0; zil_commit(zilog, 0); /* commit all itx */ @@ -1746,9 +1740,9 @@ * destroy the zl_clean_taskq. */ mutex_enter(&zilog->zl_lock); - tail_lwb = list_tail(&zilog->zl_lwb_list); - if (tail_lwb != NULL) - txg = tail_lwb->lwb_max_txg; + lwb = list_tail(&zilog->zl_lwb_list); + if (lwb != NULL) + txg = lwb->lwb_max_txg; mutex_exit(&zilog->zl_lock); if (txg) txg_wait_synced(zilog->zl_dmu_pool, txg); @@ -1756,6 +1750,19 @@ taskq_destroy(zilog->zl_clean_taskq); zilog->zl_clean_taskq = NULL; zilog->zl_get_data = NULL; + + /* + * We should have only one LWB left on the list; remove it now. + */ + mutex_enter(&zilog->zl_lock); + lwb = list_head(&zilog->zl_lwb_list); + if (lwb != NULL) { + ASSERT(lwb == list_tail(&zilog->zl_lwb_list)); + list_remove(&zilog->zl_lwb_list, lwb); + zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); + kmem_cache_free(zil_lwb_cache, lwb); + } + mutex_exit(&zilog->zl_lock); } /* --------------020005050609060404020702-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 12:34:37 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B83D1065670 for ; Tue, 26 Jul 2011 12:34:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id BFF158FC0C for ; Tue, 26 Jul 2011 12:34:36 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id D1018167328; Tue, 26 Jul 2011 14:34:35 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id MwOB66KoK1Hg; Tue, 26 Jul 2011 14:34:33 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id F2413167316; Tue, 26 Jul 2011 14:34:32 +0200 (CEST) Message-ID: <4E2EB457.4000706@FreeBSD.org> Date: Tue, 26 Jul 2011 14:34:31 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: d@delphij.net References: <4E2DD17C.60606@delphij.net> In-Reply-To: <4E2DD17C.60606@delphij.net> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:34:37 -0000 If it is also sucessfully tested on sparc64 (and/or powerpc64) I am ok with this patch. Dňa 25.07.2011 22:26, Xin LI wrote / napísal(a): > Hi, > > Here is a patch that makes libzfs use zfs_ioctl consistently. Some > functionality in libzfs calls ioctl() directly with raw ZFS ioctls, > which causes some kernel message like: > > WARNING pid 30031 (initial thread): ioctl sign-extension ioctl > ffffffffd5985a3a > > Objections? I have kept #sun'ed out blocks out. > > Cheers, > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 13:21:21 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F296D106564A for ; Tue, 26 Jul 2011 13:21:20 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id A59DF8FC0A for ; Tue, 26 Jul 2011 13:21:20 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 5F30962B; Tue, 26 Jul 2011 15:21:18 +0200 (CEST) Date: Tue, 26 Jul 2011 15:21:16 +0200 From: Pawel Jakub Dawidek To: d@delphij.net Message-ID: <20110726132116.GC1656@garage.freebsd.pl> References: <4E2DD17C.60606@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4ZLFUWh1odzi/v6L" Content-Disposition: inline In-Reply-To: <4E2DD17C.60606@delphij.net> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 13:21:21 -0000 --4ZLFUWh1odzi/v6L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 25, 2011 at 01:26:36PM -0700, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 >=20 > Hi, >=20 > Here is a patch that makes libzfs use zfs_ioctl consistently. Some > functionality in libzfs calls ioctl() directly with raw ZFS ioctls, > which causes some kernel message like: >=20 > WARNING pid 30031 (initial thread): ioctl sign-extension ioctl > ffffffffd5985a3a >=20 > Objections? I have kept #sun'ed out blocks out. zfs_ioctl() does something with history records, I think. Are you sure zfs_ioctl() is 1-to-1 equivalent of plain ioctl()? Some minor nits inline. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 224337) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -2382,6 +2382,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c > { > int err; > zfs_cmd_t zc =3D { 0 }; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > =20 > (void) strncpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); > =20 > @@ -2392,7 +2393,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c > if (err) > return (err); > =20 > - err =3D ioctl(zhp->zfs_hdl->libzfs_fd, ZFS_IOC_USERSPACE_ONE, &zc); > + err =3D zfs_ioctl(hdl, ZFS_IOC_USERSPACE_ONE, &zc); Do we really need additional variable that is used only once? Wouldn't it be better to just replace ioctl(zhp->zfs_hdl->libzfs_fd) with zfs_ioctl(zhp->zfs_hdl)? > @@ -2458,11 +2459,12 @@ zfs_do_list_ioctl(zfs_handle_t *zhp, unsigned long > { > int rc; > uint64_t orig_cookie; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > =20 > orig_cookie =3D zc->zc_cookie; > top: > (void) strlcpy(zc->zc_name, zhp->zfs_name, sizeof (zc->zc_name)); > - rc =3D ioctl(zhp->zfs_hdl->libzfs_fd, arg, zc); > + rc =3D zfs_ioctl(hdl, arg, zc); Same here. > @@ -3924,6 +3926,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro > zfs_userspace_cb_t func, void *arg) > { > zfs_cmd_t zc =3D { 0 }; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > int error; > zfs_useracct_t buf[100]; > =20 > @@ -3937,7 +3940,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro > zfs_useracct_t *zua =3D buf; > =20 > zc.zc_nvlist_dst_size =3D sizeof (buf); > - error =3D ioctl(zhp->zfs_hdl->libzfs_fd, > + error =3D zfs_ioctl(hdl, > ZFS_IOC_USERSPACE_MANY, &zc); And here. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --4ZLFUWh1odzi/v6L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4uv0wACgkQForvXbEpPzTlGgCg7WXprmBbYNaMkeVpLEA9vWXY g7YAnitMSxnr+IHKHNYHvJOniEIEiiZH =cK5y -----END PGP SIGNATURE----- --4ZLFUWh1odzi/v6L-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 20:42:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26787106564A; Tue, 26 Jul 2011 20:42:58 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C1538FC12; Tue, 26 Jul 2011 20:42:57 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 334F3165FCD; Tue, 26 Jul 2011 22:42:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id FO7mVGG6Mb1Y; Tue, 26 Jul 2011 22:42:52 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id D702C165FB8; Tue, 26 Jul 2011 22:42:51 +0200 (CEST) Message-ID: <4E2F26CD.30801@FreeBSD.org> Date: Tue, 26 Jul 2011 22:42:53 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E2DD17C.60606@delphij.net> <20110726132116.GC1656@garage.freebsd.pl> In-Reply-To: <20110726132116.GC1656@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 20:42:58 -0000 Dn(a 26. 7. 2011 15:21, Pawel Jakub Dawidek wrote / napísal(a): > On Mon, Jul 25, 2011 at 01:26:36PM -0700, Xin LI wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> Hi, >> >> Here is a patch that makes libzfs use zfs_ioctl consistently. Some >> functionality in libzfs calls ioctl() directly with raw ZFS ioctls, >> which causes some kernel message like: >> >> WARNING pid 30031 (initial thread): ioctl sign-extension ioctl >> ffffffffd5985a3a >> >> Objections? I have kept #sun'ed out blocks out. > > zfs_ioctl() does something with history records, I think. Are you sure > zfs_ioctl() is 1-to-1 equivalent of plain ioctl()? > > Some minor nits inline. Yes, zfs_ioctl deals with history and calls ioctl the very same way: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c: int zfs_ioctl(libzfs_handle_t *hdl, unsigned long request, zfs_cmd_t *zc) { int error; zc->zc_history = (uint64_t)(uintptr_t)hdl->libzfs_log_str; error = ioctl(hdl->libzfs_fd, request, zc); if (hdl->libzfs_log_str) { free(hdl->libzfs_log_str); hdl->libzfs_log_str = NULL; } zc->zc_history = 0; return (error); } Now to ioctl(): we are not calling ioctl directly. Ioctl() is actually zcmd_ioctl() - see bottom of the following code: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h: static int zfs_kernel_version = 0; /* * This is FreeBSD version of ioctl, because Solaris' ioctl() updates * zc_nvlist_dst_size even if an error is returned, on FreeBSD if an * error is returned zc_nvlist_dst_size won't be updated. */ static __inline int zcmd_ioctl(int fd, unsigned long cmd, zfs_cmd_t *zc) { size_t oldsize, zfs_kernel_version_size; int version, ret, cflag = ZFS_CMD_COMPAT_NONE; zfs_kernel_version_size = sizeof(zfs_kernel_version); if (zfs_kernel_version == 0) { sysctlbyname("vfs.zfs.version.spa", &zfs_kernel_version, &zfs_kernel_version_size, NULL, 0); } if (zfs_kernel_version == SPA_VERSION_15 || zfs_kernel_version == SPA_VERSION_14 || zfs_kernel_version == SPA_VERSION_13) cflag = ZFS_CMD_COMPAT_V15; oldsize = zc->zc_nvlist_dst_size; ret = zcmd_ioctl_compat(fd, cmd, zc, cflag); if (ret == 0 && oldsize < zc->zc_nvlist_dst_size) { ret = -1; errno = ENOMEM; } return (ret); } #define ioctl(fd, cmd, zc) zcmd_ioctl((fd), (cmd), (zc)) This function calls my zcmd_ioctl_compat in: sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c Probably there is a problem in this function (my compat code)? But for ZFS_CMD_COMPAT_NONE it is just passed to the kernel as it was before. > > >> Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c >> =================================================================== >> --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 224337) >> +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) >> @@ -2382,6 +2382,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c >> { >> int err; >> zfs_cmd_t zc = { 0 }; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> >> (void) strncpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); >> >> @@ -2392,7 +2393,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c >> if (err) >> return (err); >> >> - err = ioctl(zhp->zfs_hdl->libzfs_fd, ZFS_IOC_USERSPACE_ONE, &zc); >> + err = zfs_ioctl(hdl, ZFS_IOC_USERSPACE_ONE, &zc); > > Do we really need additional variable that is used only once? > Wouldn't it be better to just replace ioctl(zhp->zfs_hdl->libzfs_fd) > with zfs_ioctl(zhp->zfs_hdl)? I agree, we don't and just for one use we waste resources. > >> @@ -2458,11 +2459,12 @@ zfs_do_list_ioctl(zfs_handle_t *zhp, unsigned long >> { >> int rc; >> uint64_t orig_cookie; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> >> orig_cookie = zc->zc_cookie; >> top: >> (void) strlcpy(zc->zc_name, zhp->zfs_name, sizeof (zc->zc_name)); >> - rc = ioctl(zhp->zfs_hdl->libzfs_fd, arg, zc); >> + rc = zfs_ioctl(hdl, arg, zc); > > Same here. > >> @@ -3924,6 +3926,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro >> zfs_userspace_cb_t func, void *arg) >> { >> zfs_cmd_t zc = { 0 }; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> int error; >> zfs_useracct_t buf[100]; >> >> @@ -3937,7 +3940,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro >> zfs_useracct_t *zua = buf; >> >> zc.zc_nvlist_dst_size = sizeof (buf); >> - error = ioctl(zhp->zfs_hdl->libzfs_fd, >> + error = zfs_ioctl(hdl, >> ZFS_IOC_USERSPACE_MANY, &zc); > > And here. > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 20:55:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48A6B106564A for ; Tue, 26 Jul 2011 20:55:13 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id F35448FC13 for ; Tue, 26 Jul 2011 20:55:12 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 546F2183602; Tue, 26 Jul 2011 22:55:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id qiMBjk0DMulz; Tue, 26 Jul 2011 22:55:10 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 015051835F1; Tue, 26 Jul 2011 22:55:09 +0200 (CEST) Message-ID: <4E2F29AF.5030905@FreeBSD.org> Date: Tue, 26 Jul 2011 22:55:11 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: d@delphij.net References: <4E2DD17C.60606@delphij.net> In-Reply-To: <4E2DD17C.60606@delphij.net> X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------030604010008080804060700" Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 20:55:13 -0000 This is a multi-part message in MIME format. --------------030604010008080804060700 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I have chatted with delphij@ and he said his problem was in zfs jail / unjail. The problem is in zfs_jail(), cmd passed to ioctl() is initialized as int instead of unsigned long. Patch suggested for fixing this problem is attached. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------030604010008080804060700 Content-Type: text/plain; name="libzfs_dataset.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="libzfs_dataset.c.patch" Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 224409) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -4289,7 +4289,8 @@ libzfs_handle_t *hdl = zhp->zfs_hdl; zfs_cmd_t zc = { 0 }; char errbuf[1024]; - int cmd, ret; + unsigned long cmd; + int ret; if (attach) { (void) snprintf(errbuf, sizeof (errbuf), --------------030604010008080804060700-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 05:34:52 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BC26106564A; Wed, 27 Jul 2011 05:34:52 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C630C8FC0A; Wed, 27 Jul 2011 05:34:51 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 9CD6892B; Wed, 27 Jul 2011 07:34:49 +0200 (CEST) Date: Wed, 27 Jul 2011 07:34:46 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110727053445.GA4808@garage.freebsd.pl> References: <4E2DD17C.60606@delphij.net> <4E2F29AF.5030905@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XsQoSWH+UP9D9v3l" Content-Disposition: inline In-Reply-To: <4E2F29AF.5030905@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 05:34:52 -0000 --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 10:55:11PM +0200, Martin Matuska wrote: > I have chatted with delphij@ and he said his problem was in zfs jail / > unjail. >=20 > The problem is in zfs_jail(), cmd passed to ioctl() is initialized as > int instead of unsigned long. > Patch suggested for fixing this problem is attached. Looks good to me. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 224409) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -4289,7 +4289,8 @@ > libzfs_handle_t *hdl =3D zhp->zfs_hdl; > zfs_cmd_t zc =3D { 0 }; > char errbuf[1024]; > - int cmd, ret; > + unsigned long cmd; > + int ret; > =20 > if (attach) { > (void) snprintf(errbuf, sizeof (errbuf), --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --XsQoSWH+UP9D9v3l Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4vo3UACgkQForvXbEpPzStaACg7WF3sxP735kTAQXQWgHk/PY4 mQgAoJTRmoFT8iHi3LFmtETpkN0vrOJJ =/L3p -----END PGP SIGNATURE----- --XsQoSWH+UP9D9v3l-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 09:09:23 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6711B106564A; Wed, 27 Jul 2011 09:09:23 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id AB7208FC0A; Wed, 27 Jul 2011 09:09:22 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 1DD6B9B2; Wed, 27 Jul 2011 11:09:21 +0200 (CEST) Date: Wed, 27 Jul 2011 11:09:18 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110727090918.GB1642@garage.freebsd.pl> References: <4E2EB3A4.60007@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+pHx0qQiF2pBVqBT" Content-Disposition: inline In-Reply-To: <4E2EB3A4.60007@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org, Xin Li Subject: Re: [PATCH] Illumos #883 ZIL reuse during remount can lead to data corruption X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 09:09:23 -0000 --+pHx0qQiF2pBVqBT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 02:31:32PM +0200, Martin Matuska wrote: > Illumos developers have fixed a problem in ZIL that may lead to data > corruption during remount. >=20 > A detailed description of this problem is available at: > https://www.illumos.org/issues/883 >=20 > In my opinion this issue applies to FreeBSD, too, and we should fix this > ASAP. > Please review my patch (1:1 merge from illumos, no changes). >=20 > References: > illumos-gate revision: 13380:161b964a0e10 > https://github.com/illumos/illumos-gate/commit/c9ba2a43cb76c223d115e021fd= abd2c066e020ed This looks like a very important fix. There were several reports of people not being able to import their pools because of space map corruption and it seems like this will fix the underlying problem. I'd really like to see this committed and merged to stable/8. > Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 224409) > +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) > @@ -205,6 +205,7 @@ > */ > typedef struct ztest_ds { > objset_t *zd_os; > + rwlock_t zd_zilog_lock; > zilog_t *zd_zilog; > uint64_t zd_seq; > ztest_od_t *zd_od; /* debugging aid */ > @@ -238,6 +239,7 @@ > ztest_func_t ztest_zap; > ztest_func_t ztest_zap_parallel; > ztest_func_t ztest_zil_commit; > +ztest_func_t ztest_zil_remount; > ztest_func_t ztest_dmu_read_write_zcopy; > ztest_func_t ztest_dmu_objset_create_destroy; > ztest_func_t ztest_dmu_prealloc; > @@ -273,6 +275,7 @@ > { ztest_zap_parallel, 100, &zopt_always }, > { ztest_split_pool, 1, &zopt_always }, > { ztest_zil_commit, 1, &zopt_incessant }, > + { ztest_zil_remount, 1, &zopt_sometimes }, > { ztest_dmu_read_write_zcopy, 1, &zopt_often }, > { ztest_dmu_objset_create_destroy, 1, &zopt_often }, > { ztest_dsl_prop_get_set, 1, &zopt_often }, > @@ -986,6 +989,7 @@ > zd->zd_seq =3D 0; > dmu_objset_name(os, zd->zd_name); > =20 > + VERIFY(rwlock_init(&zd->zd_zilog_lock, USYNC_THREAD, NULL) =3D=3D 0); > VERIFY(_mutex_init(&zd->zd_dirobj_lock, USYNC_THREAD, NULL) =3D=3D 0); > =20 > for (int l =3D 0; l < ZTEST_OBJECT_LOCKS; l++) > @@ -1965,6 +1969,8 @@ > if (ztest_random(2) =3D=3D 0) > io_type =3D ZTEST_IO_WRITE_TAG; > =20 > + (void) rw_rdlock(&zd->zd_zilog_lock); > + > switch (io_type) { > =20 > case ZTEST_IO_WRITE_TAG: > @@ -2000,6 +2006,8 @@ > break; > } > =20 > + (void) rw_unlock(&zd->zd_zilog_lock); > + > umem_free(data, blocksize); > } > =20 > @@ -2054,6 +2062,8 @@ > { > zilog_t *zilog =3D zd->zd_zilog; > =20 > + (void) rw_rdlock(&zd->zd_zilog_lock); > + > zil_commit(zilog, ztest_random(ZTEST_OBJECTS)); > =20 > /* > @@ -2065,9 +2075,34 @@ > ASSERT(zd->zd_seq <=3D zilog->zl_commit_lr_seq); > zd->zd_seq =3D zilog->zl_commit_lr_seq; > mutex_exit(&zilog->zl_lock); > + > + (void) rw_unlock(&zd->zd_zilog_lock); > } > =20 > /* > + * This function is designed to simulate the operations that occur durin= g a > + * mount/unmount operation. We hold the dataset across these operations= in an > + * attempt to expose any implicit assumptions about ZIL management. > + */ > +/* ARGSUSED */ > +void > +ztest_zil_remount(ztest_ds_t *zd, uint64_t id) > +{ > + objset_t *os =3D zd->zd_os; > + > + (void) rw_wrlock(&zd->zd_zilog_lock); > + > + /* zfsvfs_teardown() */ > + zil_close(zd->zd_zilog); > + > + /* zfsvfs_setup() */ > + VERIFY(zil_open(os, ztest_get_data) =3D=3D zd->zd_zilog); > + zil_replay(os, zd, ztest_replay_vector); > + > + (void) rw_unlock(&zd->zd_zilog_lock); > +} > + > +/* > * Verify that we can't destroy an active pool, create an existing pool, > * or create a pool with a bad vdev spec. > */ > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision 224409) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright (c) 2011 by Delphix. All rights reserved. > */ > =20 > /* Portions Copyright 2010 Robert Milkowski */ > @@ -567,7 +568,7 @@ > =20 > if (!list_is_empty(&zilog->zl_lwb_list)) { > ASSERT(zh->zh_claim_txg =3D=3D 0); > - ASSERT(!keep_first); > + VERIFY(!keep_first); > while ((lwb =3D list_head(&zilog->zl_lwb_list)) !=3D NULL) { > list_remove(&zilog->zl_lwb_list, lwb); > if (lwb->lwb_buf !=3D NULL) > @@ -1668,20 +1669,9 @@ > void > zil_free(zilog_t *zilog) > { > - lwb_t *head_lwb; > - > zilog->zl_stop_sync =3D 1; > =20 > - /* > - * After zil_close() there should only be one lwb with a buffer. > - */ > - head_lwb =3D list_head(&zilog->zl_lwb_list); > - if (head_lwb) { > - ASSERT(head_lwb =3D=3D list_tail(&zilog->zl_lwb_list)); > - list_remove(&zilog->zl_lwb_list, head_lwb); > - zio_buf_free(head_lwb->lwb_buf, head_lwb->lwb_sz); > - kmem_cache_free(zil_lwb_cache, head_lwb); > - } > + ASSERT(list_is_empty(&zilog->zl_lwb_list)); > list_destroy(&zilog->zl_lwb_list); > =20 > avl_destroy(&zilog->zl_vdev_tree); > @@ -1721,6 +1711,10 @@ > { > zilog_t *zilog =3D dmu_objset_zil(os); > =20 > + ASSERT(zilog->zl_clean_taskq =3D=3D NULL); > + ASSERT(zilog->zl_get_data =3D=3D NULL); > + ASSERT(list_is_empty(&zilog->zl_lwb_list)); > + > zilog->zl_get_data =3D get_data; > zilog->zl_clean_taskq =3D taskq_create("zil_clean", 1, minclsyspri, > 2, 2, TASKQ_PREPOPULATE); > @@ -1734,7 +1728,7 @@ > void > zil_close(zilog_t *zilog) > { > - lwb_t *tail_lwb; > + lwb_t *lwb; > uint64_t txg =3D 0; > =20 > zil_commit(zilog, 0); /* commit all itx */ > @@ -1746,9 +1740,9 @@ > * destroy the zl_clean_taskq. > */ > mutex_enter(&zilog->zl_lock); > - tail_lwb =3D list_tail(&zilog->zl_lwb_list); > - if (tail_lwb !=3D NULL) > - txg =3D tail_lwb->lwb_max_txg; > + lwb =3D list_tail(&zilog->zl_lwb_list); > + if (lwb !=3D NULL) > + txg =3D lwb->lwb_max_txg; > mutex_exit(&zilog->zl_lock); > if (txg) > txg_wait_synced(zilog->zl_dmu_pool, txg); > @@ -1756,6 +1750,19 @@ > taskq_destroy(zilog->zl_clean_taskq); > zilog->zl_clean_taskq =3D NULL; > zilog->zl_get_data =3D NULL; > + > + /* > + * We should have only one LWB left on the list; remove it now. > + */ > + mutex_enter(&zilog->zl_lock); > + lwb =3D list_head(&zilog->zl_lwb_list); > + if (lwb !=3D NULL) { > + ASSERT(lwb =3D=3D list_tail(&zilog->zl_lwb_list)); > + list_remove(&zilog->zl_lwb_list, lwb); > + zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); > + kmem_cache_free(zil_lwb_cache, lwb); > + } > + mutex_exit(&zilog->zl_lock); > } > =20 > /* --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --+pHx0qQiF2pBVqBT Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4v1b4ACgkQForvXbEpPzSvIgCg2tAl6I84bMPr/Gb+/6OFCI0O 910An3kcKh0RgFKtj+EnWkiCdxVF5LvR =b2ZW -----END PGP SIGNATURE----- --+pHx0qQiF2pBVqBT-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 17:07:45 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED62F106566C; Wed, 27 Jul 2011 17:07:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id D46EE8FC0C; Wed, 27 Jul 2011 17:07:44 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id A2ECF1496D; Wed, 27 Jul 2011 10:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1311786464; bh=GiOHvCl09OcXBfsjppKVVap81mmQ26r9iz7OnBeQeYA=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=JVbfaRenxGc4oQuYkdDATQ3ma0OU98OdDxzP2Fq+GHTn5bnlPCjsLep3OdsA5sM+s TwUuwEjc5pKkUv6shCY+u/zo3b08Ayr7OHP6snjqy40IMrgWKVGLVNTLtreqP8cICE wzb9KCVSBwoyopN7XyPPd8w2O8/4EkN9J+2s+fB8= Message-ID: <4E3045E0.7030108@delphij.net> Date: Wed, 27 Jul 2011 10:07:44 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska References: <4E2DD17C.60606@delphij.net> <4E2F29AF.5030905@FreeBSD.org> In-Reply-To: <4E2F29AF.5030905@FreeBSD.org> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 17:07:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 07/26/11 13:55, Martin Matuska wrote: > I have chatted with delphij@ and he said his problem was in zfs jail > / unjail. > > The problem is in zfs_jail(), cmd passed to ioctl() is initialized > as int instead of unsigned long. Patch suggested for fixing this > problem is attached. Sorry for the delay and this solves the problem more neatly. Would you please propose your version to re@? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iQEcBAEBCAAGBQJOMEXfAAoJEATO+BI/yjfBf5EH/R12+1eMmB55+/zmLBOOimRf FA7XdBKhs556zP2iBw/2PWLON/2KANFTpPZnBZY9J2vO74CFYXP7e35UFNd9M7WS yYjkTu+E48k5GPqCk1OQw5Upx2ASrQQTwp2L0TzVlR1/O6APs4NaYUoODjGsWy48 IGpRkUO4NGUpNR8oCGRAiuY6nH/9OuT91r1EPBB3zg4sr0zxl15gcYqDxFYAU8fE 1yr+Vp64s9UA1l8Ast0/fjs/b1dOC2sZ+ErCFRdWpPlte014Uplh/dCCprZsk+dk PBcMbOa6IlzIbtSYIWKzIN6Q4iEjldjOnyy5Xdzt1iRSMRrzvociQF4qaRD67D0= =Pycb -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 20:08:47 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C982106566C; Wed, 27 Jul 2011 20:08:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 107BB8FC13; Wed, 27 Jul 2011 20:08:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id D2DB9167161; Wed, 27 Jul 2011 22:08:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id GrKtm9UJsmNz; Wed, 27 Jul 2011 22:08:43 +0200 (CEST) Received: from [10.9.8.3] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id D7ABF167149; Wed, 27 Jul 2011 22:08:42 +0200 (CEST) Message-ID: <4E30704C.9070606@FreeBSD.org> Date: Wed, 27 Jul 2011 22:08:44 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------000705020007040007080007" Cc: zfs-devel@FreeBSD.ORG, Kostik Belousov Subject: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 20:08:47 -0000 This is a multi-part message in MIME format. --------------000705020007040007080007 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Please review my attached patch. The patch fixes mount/unmount inside a jail for filesystems with the VFCF_JAIL flag security.jail.mount_allowed=1 is required. For "enforce_statfs == 2" it makes no sense to allow mount/unmount inside a jail so enforce_statfs == 2 implies mount_allowed = 0 The filesystems mounted inside a jail have now a corect f_mntonname. Tested with: zfs - works! (both enforce_statfs=0 and enforce_statfs=1) nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=0 and enforce_statfs=1) tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=0 and enforce_statfs=1) I assume other filesystems are going to work correctly, too (e.g nfs). Fore the future, I suggest a option to allow mounting specific filesystems in a jail (e.g. zfs, nullfs, tmpfs). I consider nullfs mounts harmless inside a jail. With jailed nullfs and tmpfs we can run tinderbox in a jail! Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000705020007040007080007 Content-Type: text/plain; name="jail_mount.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="jail_mount.patch" Index: src/sys/kern/kern_jail.c =================================================================== --- src/sys/kern/kern_jail.c (revision 224297) +++ src/sys/kern/kern_jail.c (working copy) @@ -3858,7 +3858,8 @@ case PRIV_VFS_UNMOUNT: case PRIV_VFS_MOUNT_NONUSER: case PRIV_VFS_MOUNT_OWNER: - if (cred->cr_prison->pr_allow & PR_ALLOW_MOUNT) + if (cred->cr_prison->pr_allow & PR_ALLOW_MOUNT && + cred->cr_prison->pr_enforce_statfs != 2) return (0); else return (EPERM); Index: src/sys/kern/vfs_mount.c =================================================================== --- src/sys/kern/vfs_mount.c (revision 224297) +++ src/sys/kern/vfs_mount.c (working copy) @@ -1007,6 +1007,7 @@ struct vfsconf *vfsp; struct nameidata nd; struct vnode *vp; + char realfspath[MNAMELEN]; int error; /* @@ -1023,6 +1024,21 @@ } /* + * If we are jailed, construct real filesystem path + */ + if (jailed(td->td_ucred) && + strcmp(td->td_ucred->cr_prison->pr_path, "/") != 0) { + if (strlen(td->td_ucred->cr_prison->pr_path) + + strlen(fspath) >= MNAMELEN) + return (ENAMETOOLONG); + strlcpy(realfspath, td->td_ucred->cr_prison->pr_path, + sizeof(realfspath)); + strlcat(realfspath, fspath, sizeof(realfspath)); + } else { + strlcpy(realfspath, fspath, sizeof(realfspath)); + } + + /* * Do not allow NFS export or MNT_SUIDDIR by unprivileged users. */ if (fsflags & MNT_EXPORTED) { @@ -1070,7 +1086,7 @@ NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; if ((fsflags & MNT_UPDATE) == 0) { - error = vfs_domount_first(td, vfsp, fspath, vp, fsflags, + error = vfs_domount_first(td, vfsp, realfspath, vp, fsflags, optlist); } else { error = vfs_domount_update(td, vp, fsflags, optlist); @@ -1107,6 +1123,7 @@ struct mount *mp; char *pathbuf; int error, id0, id1; + char realfspath[MNAMELEN]; AUDIT_ARG_VALUE(uap->flags); if (jailed(td->td_ucred) || usermount == 0) { @@ -1139,10 +1156,23 @@ } mtx_unlock(&mountlist_mtx); } else { - AUDIT_ARG_UPATH1(td, pathbuf); + /* + * If we are jailed and enforce_statfs=1 + * construct real filesystem path + */ + if (jailed(td->td_ucred) && + td->td_ucred->cr_prison->pr_enforce_statfs == 1 && + strcmp(td->td_ucred->cr_prison->pr_path, "/") != 0) { + strlcpy(realfspath, td->td_ucred->cr_prison->pr_path, + sizeof(realfspath)); + strlcat(realfspath, pathbuf, sizeof(realfspath)); + } else { + strlcpy(realfspath, pathbuf, sizeof(realfspath)); + } + AUDIT_ARG_UPATH1(td, realfspath); mtx_lock(&mountlist_mtx); TAILQ_FOREACH_REVERSE(mp, &mountlist, mntlist, mnt_list) { - if (strcmp(mp->mnt_stat.f_mntonname, pathbuf) == 0) + if (strcmp(mp->mnt_stat.f_mntonname, realfspath) == 0) break; } mtx_unlock(&mountlist_mtx); --------------000705020007040007080007-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 14:11:42 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FDCB1065672; Thu, 28 Jul 2011 14:11:42 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id E4E1A8FC15; Thu, 28 Jul 2011 14:11:41 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id CE024167157; Thu, 28 Jul 2011 16:11:40 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 1Ye+jXhIbmzV; Thu, 28 Jul 2011 16:11:38 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id E3EF916714E; Thu, 28 Jul 2011 16:11:37 +0200 (CEST) Message-ID: <4E316E19.9040309@FreeBSD.org> Date: Thu, 28 Jul 2011 16:11:37 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-current@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: multipart/mixed; boundary="------------000605040303080100060003" X-Mailman-Approved-At: Thu, 28 Jul 2011 14:39:21 +0000 Cc: Subject: [PATCH] updated /etc/rc.d/jail and added ZFS support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 14:11:42 -0000 This is a multi-part message in MIME format. --------------000605040303080100060003 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch allows better fine-tuning of jails started via /etc/rc.d, uses the new jail(8) flags (-c -m), the persist parameter and adds ZFS support. Patch is fully backward compatible. Please review, comment and/or test my attached patch. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000605040303080100060003 Content-Type: text/x-patch; name="etc_jail.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="etc_jail.patch" Index: etc/rc.d/jail =================================================================== --- etc/rc.d/jail (revision 224471) +++ etc/rc.d/jail (working copy) @@ -43,6 +43,7 @@ eval _ip=\"\$jail_${_j}_ip\" eval _interface=\"\${jail_${_j}_interface:-${jail_interface}}\" eval _exec=\"\$jail_${_j}_exec\" + eval _params=\"\$jail_${_j}_params\" i=0 while : ; do @@ -83,6 +84,8 @@ i=$((i + 1)) done + eval _zfs=\"\${jail_${_j}_zfs:-}\" + if [ -n "${_exec}" ]; then # simple/backward-compatible execution _exec_start="${_exec}" @@ -98,6 +101,9 @@ fi # The default jail ruleset will be used by rc.subr if none is specified. + if [ -n "jail_devfs_ruleset" -a -n "_zfs" ]; then + jail_devfs_ruleset="devfsrules_jail_zfs" + fi eval _ruleset=\"\${jail_${_j}_devfs_ruleset:-${jail_devfs_ruleset}}\" eval _devfs=\"\${jail_${_j}_devfs_enable:-${jail_devfs_enable}}\" [ -z "${_devfs}" ] && _devfs="NO" @@ -200,6 +206,58 @@ if [ -z "${_rootdir}" ]; then err 3 "$name: No root directory has been defined for ${_j}" fi + + # Security-related parameters + eval _enforce_statfs=\"\$jail_${_j}_enforce_statfs\" + eval _allow_set_hostname=\"\$jail_${_j}_allow_set_hostname\" + eval _allow_sysvipc=\"\$jail_${_j}_allow_sysvipc\" + eval _allow_raw_sockets=\"\$jail_${_j}_allow_raw_sockets\" + eval _allow_chflags=\"\$jail_${_j}_allow_chflags\" + eval _allow_mount=\"\$jail_${_j}_allow_mount\" + eval _allow_socket_af=\"\$jail_${_j}_allow_socket_af\" + eval _allow_quotas=\"\$jail_${_j}_allow_quotas:-0\" + + if [ -z "${_enforce_statfs}" ]; then + _enforce_statfs=`${SYSCTL} -n security.jail.enforce_statfs` + fi + + if [ -z "${_allow_set_hostname}" ]; then + _allow_set_hostname=`${SYSCTL} -n security.jail.set_hostname_allowed` + fi + + if [ -z "${_allow_sysvipc}" ]; then + _allow_sysvipc=`${SYSCTL} -n security.jail.sysvipc_allowed` + fi + + if [ -z "${_allow_raw_sockets}" ]; then + _allow_raw_sockets=`${SYSCTL} -n security.jail.allow_raw_sockets` + fi + + if [ -z "${_allow_chflags}" ]; then + _allow_chflags=`${SYSCTL} -n security.jail.chflags_allowed` + fi + + if [ -z "${_allow_mount}" ]; then + _allow_mount=`${SYSCTL} -n security.jail.mount_allowed` + fi + + if [ -z "${_allow_socket_af}" ]; then + _tmpval=`${SYSCTL} -n security.jail.socket_unixiproute_only` + if [ "${_tmpval}" = "0" ]; then + _allow_socket_af=1 + else + _allow_socket_af=0 + fi + fi + + _security_params="enforce_statfs=${_enforce_statfs} \ + allow.set_hostname=${_allow_set_hostname} \ + allow.sysvipc=${_allow_sysvipc} \ + allow.raw_sockets=${_allow_raw_sockets} \ + allow.chflags=${_allow_chflags} \ + allow.mount=${_allow_mount} \ + allow.socket_af=${_allow_socket_af} \ + allow.quotas=${allow_quotas}" } # set_sysctl rc_knob mib msg @@ -345,6 +403,36 @@ mount -a -F "${_fstab}" } +# jail_zfs_jailin +# Make zfs datasets manageable from inside a jail +# the "jailed" dataset property must be set to "on" +jail_zfs_jailin() +{ + if [ -n "${_zfs}" ]; then + for _ds in ${_zfs}; do + _jailed=`zfs get -H jailed ${_ds} 2>/dev/null | awk '{ print $3 }'` + if [ "$_jailed" = "on" ]; then + zfs jail "${_jail_id}" ${_ds} 2>/dev/null + fi + done + fi +} + +# jail_zfs_jailout +# Unjail zfs datasets +# the "jailed" dataset property must be set to "on" +jail_zfs_jailout() +{ + if [ -n "${_zfs}" ]; then + for _ds in ${_zfs}; do + _jailed=`zfs get -H jailed ${_ds} 2>/dev/null | awk '{ print $3 }'` + if [ "$_jailed" = "on" ]; then + zfs unjail "${_jail_id}" ${_ds} 2>/dev/null + fi + done + fi +} + # jail_show_addresses jail # Debug print the input for the given _multi aliases # for a jail for init_variables(). @@ -483,10 +571,27 @@ *) ;; esac - # Append address to list of addresses for the jail command. - case "${_addrl}" in - "") _addrl="${_addr}" ;; - *) _addrl="${_addrl},${_addr}" ;; + case "${_type}" in + inet) + # Append address to list of ipv4 addresses for the + # jail command. + case "${_addrl}" in + "") _addrl="${_addr}" ;; + *) _addrl="${_addrl},${_addr}" ;; + esac + ;; + inet6) + # Append address to list of ipv6 addresses for the + # jail command. + case "${_addrl6}" in + "") _addrl6="${_addr}" ;; + *) _addrl6="${_addrl6},${_addr}" ;; + esac + ;; + *) warn "Could not determine address family. Not going" \ + "to set address '${_addr}' for ${_jail}." + continue + ;; esac # Configure interface alias if requested by a given interface @@ -494,14 +599,7 @@ case "${_iface}" in "") continue ;; esac - case "${_type}" in - inet) ;; - inet6) ;; - *) warn "Could not determine address family. Not going" \ - "to ${_action} address '${_addr}' for ${_jail}." - continue - ;; - esac + case "${_action}" in add) ifconfig ${_iface} ${_type} ${_addr}${_mask} alias ;; @@ -576,6 +674,7 @@ continue; fi _addrl="" + _addrl6="" jail_ips "add" if [ -n "${_fib}" ]; then _setfib="setfib -F '${_fib}'" @@ -644,42 +743,54 @@ i=$((i + 1)) done - eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \ - \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 \ - ${_tmp_jail} 2>&1 \ + ${_consolelog} - echo ${_jail_id} > /var/run/jail_${_jail}.id + echo -n " $_hostname" + tail +2 ${_tmp_jail} >${_consolelog} + echo ${_jail_id} > /var/run/jail_${_jail}.id - i=0 - while : ; do - eval out=\"\${_exec_poststart${i}:-''}\" - [ -z "$out" ] && break - ${out} - i=$((i + 1)) - done - else - jail_umount_fs - jail_ips "del" - echo " cannot start jail \"${_jail}\": " - tail +2 ${_tmp_jail} + i=0 + while : ; do + eval out=\"\${_exec_poststart${i}:-''}\" + [ -z "$out" ] && break + ${out} + i=$((i + 1)) + done + else + jail_zfs_jailout + jail -m jid="${_jail_id}" persist=0 + jail_umount_fs + jail_ips "del" + echo " cannot start jail \"${_jail}\": " + tail +2 ${_tmp_jail} + fi + rm -f ${_tmp_jail} fi - rm -f ${_tmp_jail} done rmdir ${_tmp_dir} echo '.' @@ -707,6 +818,7 @@ eval env -i /usr/sbin/jexec ${_jail_id} ${_exec_stop} \ >> ${_consolelog} 2>&1 fi + jail_zfs_jailout killall -j ${_jail_id} -TERM > /dev/null 2>&1 sleep 1 killall -j ${_jail_id} -KILL > /dev/null 2>&1 Index: etc/defaults/devfs.rules =================================================================== --- etc/defaults/devfs.rules (revision 224471) +++ etc/defaults/devfs.rules (working copy) @@ -83,3 +83,9 @@ add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login + +# Jail with zfs support +# +[devfsrules_jail_zfs=5] +add include $devfsrules_jail +add path zfs unhide Index: etc/defaults/rc.conf =================================================================== --- etc/defaults/rc.conf (revision 224471) +++ etc/defaults/rc.conf (working copy) @@ -695,6 +695,21 @@ #jail_example_mount_enable="NO" # mount/umount jail's fs #jail_example_fstab="" # fstab(5) for mount/umount #jail_example_flags="-l -U root" # flags for jail(8) +#jail_example_params="" # additional parameters for jail(8) +#jail_example_enforce_statfs="" # jail(8) enforce_statfs parameter +#jail_example_allow_set_hostname="" # jail(8) allow.set_hostname parameter +#jail_example_allow_sysvipc="" # jail(8) allow.sysvipc parameter +#jail_example_allow_raw_sockets="" # jail(8) allow.raw_sockets parameter +#jail_example_allow_chflags="" # jail(8) allow.chflags parameter +#jail_example_allow_mount="" # jail(8) allow.mount parameter +#jail_example_allow_socket_af="" # jail(8) allow.socket_af parameter +#jail_example_allow_quotas="" # jail(8) allow.quotas parameter +#jail_example_zfs="" # Space-separated list of ZFS datasets to be + # managed from this jail. For proper operation, + # allow_mount must be defined and enforce_statfs + # must be lower than 2. The "jailed" property + # must be set to "on" on these datasets before starting + # the jail. ############################################################## ### Define source_rc_confs, the mechanism used by /etc/rc.* ## --------------000605040303080100060003-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 16:07:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17B83106564A; Thu, 28 Jul 2011 16:07:58 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C18448FC15; Thu, 28 Jul 2011 16:07:57 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 7730CA3; Thu, 28 Jul 2011 18:07:55 +0200 (CEST) Date: Thu, 28 Jul 2011 18:07:52 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110728160751.GA1704@garage.freebsd.pl> References: <4E30704C.9070606@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2oS5YaxWCcQjTEyO" Content-Disposition: inline In-Reply-To: <4E30704C.9070606@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@FreeBSD.ORG Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 16:07:58 -0000 --2oS5YaxWCcQjTEyO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 27, 2011 at 10:08:44PM +0200, Martin Matuska wrote: > Please review my attached patch. >=20 > The patch fixes mount/unmount inside a jail for filesystems with the > VFCF_JAIL flag > security.jail.mount_allowed=3D1 is required. > For "enforce_statfs =3D=3D 2" it makes no sense to allow mount/unmount > inside a jail so enforce_statfs =3D=3D 2 implies mount_allowed =3D 0 > The filesystems mounted inside a jail have now a corect f_mntonname. >=20 > Tested with: > zfs - works! (both enforce_statfs=3D0 and enforce_statfs=3D1) > nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > enforce_statfs=3D1) > tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > enforce_statfs=3D1) >=20 > I assume other filesystems are going to work correctly, too (e.g nfs). > Fore the future, I suggest a option to allow mounting specific > filesystems in a jail (e.g. zfs, nullfs, tmpfs). >=20 > I consider nullfs mounts harmless inside a jail. >=20 > With jailed nullfs and tmpfs we can run tinderbox in a jail! In you patch you depend on fact that full path to mount directory is passed to the nmount(2) system call. This doesn't have to be true. I changed mount(8) to call realpath(3) in mount directory, but I see no reason someone calling nmount(2) directly with "./foo" mount dir. I think the proper way is to build full path from within the kernel using vn_fullpath_global(). --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --2oS5YaxWCcQjTEyO Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4xiVcACgkQForvXbEpPzRfGwCdE4+FvceZi2LZvA+nFBEl/nHJ OeEAnRwVfIED/tFkkWBN3DLqU3pFfUMQ =COGe -----END PGP SIGNATURE----- --2oS5YaxWCcQjTEyO-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:38:16 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DEDB106564A; Thu, 28 Jul 2011 17:38:16 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 531A48FC18; Thu, 28 Jul 2011 17:38:16 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 9CEBFE0; Thu, 28 Jul 2011 19:38:14 +0200 (CEST) Date: Thu, 28 Jul 2011 19:38:11 +0200 From: Pawel Jakub Dawidek To: Kostik Belousov Message-ID: <20110728173811.GB1704@garage.freebsd.pl> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LpQ9ahxlCli8rRTG" Content-Disposition: inline In-Reply-To: <20110728172539.GU17489@deviant.kiev.zoral.com.ua> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:38:16 -0000 --LpQ9ahxlCli8rRTG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: > On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > > In you patch you depend on fact that full path to mount directory is > > passed to the nmount(2) system call. This doesn't have to be true. > > I changed mount(8) to call realpath(3) in mount directory, but I see no > > reason someone calling nmount(2) directly with "./foo" mount dir. > >=20 > > I think the proper way is to build full path from within the kernel > > using vn_fullpath_global(). >=20 > It indeed may work if the supplied vnode is a directory, since it > will fall back to the dotdot lookup loop if the namecache is purged. Exactly. Mount directory by definition is always a directory, so it should be reliable. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --LpQ9ahxlCli8rRTG Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4xnoIACgkQForvXbEpPzS3OACgyy0E1Ycqz1AbucDo0oBeP3wc CAsAn2x7PpreyfSoyUbYdhWCoElxZFCf =QuAm -----END PGP SIGNATURE----- --LpQ9ahxlCli8rRTG-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:40:05 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D795D106566C for ; Thu, 28 Jul 2011 17:40:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6B06B8FC16 for ; Thu, 28 Jul 2011 17:40:05 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6SHPd4B019140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6SHPd1S015954; Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6SHPd21015953; Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Jul 2011 20:25:39 +0300 From: Kostik Belousov To: Pawel Jakub Dawidek Message-ID: <20110728172539.GU17489@deviant.kiev.zoral.com.ua> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xK0oco9ieugCOMaM" Content-Disposition: inline In-Reply-To: <20110728160751.GA1704@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:40:06 -0000 --xK0oco9ieugCOMaM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > On Wed, Jul 27, 2011 at 10:08:44PM +0200, Martin Matuska wrote: > > Please review my attached patch. > >=20 > > The patch fixes mount/unmount inside a jail for filesystems with the > > VFCF_JAIL flag > > security.jail.mount_allowed=3D1 is required. > > For "enforce_statfs =3D=3D 2" it makes no sense to allow mount/unmount > > inside a jail so enforce_statfs =3D=3D 2 implies mount_allowed =3D 0 > > The filesystems mounted inside a jail have now a corect f_mntonname. > >=20 > > Tested with: > > zfs - works! (both enforce_statfs=3D0 and enforce_statfs=3D1) > > nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 a= nd > > enforce_statfs=3D1) > > tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > > enforce_statfs=3D1) > >=20 > > I assume other filesystems are going to work correctly, too (e.g nfs). > > Fore the future, I suggest a option to allow mounting specific > > filesystems in a jail (e.g. zfs, nullfs, tmpfs). > >=20 > > I consider nullfs mounts harmless inside a jail. > >=20 > > With jailed nullfs and tmpfs we can run tinderbox in a jail! >=20 > In you patch you depend on fact that full path to mount directory is > passed to the nmount(2) system call. This doesn't have to be true. > I changed mount(8) to call realpath(3) in mount directory, but I see no > reason someone calling nmount(2) directly with "./foo" mount dir. >=20 > I think the proper way is to build full path from within the kernel > using vn_fullpath_global(). It indeed may work if the supplied vnode is a directory, since it will fall back to the dotdot lookup loop if the namecache is purged. --xK0oco9ieugCOMaM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4xm5IACgkQC3+MBN1Mb4hVvgCgkf3/1L/oVW34LWyKmZrOx5AQ eSUAn3Y3Wsu8eeBlv50qSjttesfZr6Tc =wZk4 -----END PGP SIGNATURE----- --xK0oco9ieugCOMaM-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:41:50 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50F2F106566C; Thu, 28 Jul 2011 17:41:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id BE3198FC0A; Thu, 28 Jul 2011 17:41:49 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6SHfgoW020892 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6SHfg1r016562; Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6SHfgaM016561; Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Jul 2011 20:41:42 +0300 From: Kostik Belousov To: Pawel Jakub Dawidek Message-ID: <20110728174142.GW17489@deviant.kiev.zoral.com.ua> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> <20110728173811.GB1704@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0N88lRoFd5qAfd1F" Content-Disposition: inline In-Reply-To: <20110728173811.GB1704@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@FreeBSD.org, Martin Matuska Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:41:50 -0000 --0N88lRoFd5qAfd1F Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 07:38:11PM +0200, Pawel Jakub Dawidek wrote: > On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: > > On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > > > In you patch you depend on fact that full path to mount directory is > > > passed to the nmount(2) system call. This doesn't have to be true. > > > I changed mount(8) to call realpath(3) in mount directory, but I see = no > > > reason someone calling nmount(2) directly with "./foo" mount dir. > > >=20 > > > I think the proper way is to build full path from within the kernel > > > using vn_fullpath_global(). > >=20 > > It indeed may work if the supplied vnode is a directory, since it > > will fall back to the dotdot lookup loop if the namecache is purged. >=20 > Exactly. Mount directory by definition is always a directory, so it > should be reliable. It is so on FreeBSD, but is not true for a random Unix in existence. E.g., Solaris does allow to mount over the file, and it is useful. --0N88lRoFd5qAfd1F Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4xn1UACgkQC3+MBN1Mb4hgAwCghciKG94NNKKnyHc/bbycCrwa adUAniMwbywpnJO16fpRpFFPXUeuX885 =+BhQ -----END PGP SIGNATURE----- --0N88lRoFd5qAfd1F-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 29 10:30:33 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3938C106566C; Fri, 29 Jul 2011 10:30:33 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4CF8FC0C; Fri, 29 Jul 2011 10:30:32 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AA870145301; Fri, 29 Jul 2011 12:30:31 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id oMZ6YuP4141m; Fri, 29 Jul 2011 12:30:29 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 3909C1452DB; Fri, 29 Jul 2011 12:30:26 +0200 (CEST) Message-ID: <4E328BC5.5030001@FreeBSD.org> Date: Fri, 29 Jul 2011 12:30:29 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> <20110728173811.GB1704@garage.freebsd.pl> In-Reply-To: <20110728173811.GB1704@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------020101090001040705070308" Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2011 10:30:33 -0000 This is a multi-part message in MIME format. --------------020101090001040705070308 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dňa 28. 7. 2011 19:38, Pawel Jakub Dawidek wrote / napísal(a): > On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: >> On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: >>> In you patch you depend on fact that full path to mount directory is >>> passed to the nmount(2) system call. This doesn't have to be true. >>> I changed mount(8) to call realpath(3) in mount directory, but I see no >>> reason someone calling nmount(2) directly with "./foo" mount dir. >>> >>> I think the proper way is to build full path from within the kernel >>> using vn_fullpath_global(). >> It indeed may work if the supplied vnode is a directory, since it >> will fall back to the dotdot lookup loop if the namecache is purged. > Exactly. Mount directory by definition is always a directory, so it > should be reliable. > I have reworked and tested the patch to use vn_fullpath_global(), patch attached. The vp to global path translation is done in vfs_domount_first() (static function) that doesn't take a fspath argument anymore. Please review, comment and/or test my attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020101090001040705070308 Content-Type: text/plain; name="jail_mount.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="jail_mount.patch" SW5kZXg6IHN5cy9rZXJuL2tlcm5famFpbC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9rZXJu L2tlcm5famFpbC5jCShyZXZpc2lvbiAyMjQ0NzEpCisrKyBzeXMva2Vybi9rZXJuX2phaWwu Ywkod29ya2luZyBjb3B5KQpAQCAtMzg1OCw3ICszODU4LDggQEAKIAljYXNlIFBSSVZfVkZT X1VOTU9VTlQ6CiAJY2FzZSBQUklWX1ZGU19NT1VOVF9OT05VU0VSOgogCWNhc2UgUFJJVl9W RlNfTU9VTlRfT1dORVI6Ci0JCWlmIChjcmVkLT5jcl9wcmlzb24tPnByX2FsbG93ICYgUFJf QUxMT1dfTU9VTlQpCisJCWlmIChjcmVkLT5jcl9wcmlzb24tPnByX2FsbG93ICYgUFJfQUxM T1dfTU9VTlQgJiYKKwkJCWNyZWQtPmNyX3ByaXNvbi0+cHJfZW5mb3JjZV9zdGF0ZnMgIT0g MikKIAkJCXJldHVybiAoMCk7CiAJCWVsc2UKIAkJCXJldHVybiAoRVBFUk0pOwpJbmRleDog c3lzL2tlcm4vdmZzX21vdW50LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2tlcm4vdmZzX21v dW50LmMJKHJldmlzaW9uIDIyNDQ3MSkKKysrIHN5cy9rZXJuL3Zmc19tb3VudC5jCSh3b3Jr aW5nIGNvcHkpCkBAIC03NDUsNyArNzQ1LDYgQEAKIHZmc19kb21vdW50X2ZpcnN0KAogCXN0 cnVjdCB0aHJlYWQgKnRkLAkJLyogQ2FsbGluZyB0aHJlYWQuICovCiAJc3RydWN0IHZmc2Nv bmYgKnZmc3AsCQkvKiBGaWxlIHN5c3RlbSB0eXBlLiAqLwotCWNoYXIgKmZzcGF0aCwJCQkv KiBNb3VudCBwYXRoLiAqLwogCXN0cnVjdCB2bm9kZSAqdnAsCQkvKiBWbm9kZSB0byBiZSBj b3ZlcmVkLiAqLwogCWludCBmc2ZsYWdzLAkJCS8qIEZsYWdzIGNvbW1vbiB0byBhbGwgZmls ZXN5c3RlbXMuICovCiAJc3RydWN0IHZmc29wdGxpc3QgKipvcHRsaXN0CS8qIE9wdGlvbnMg bG9jYWwgdG8gdGhlIGZpbGVzeXN0ZW0uICovCkBAIC03NTUsMTEgKzc1NCwyMyBAQAogCXN0 cnVjdCBtb3VudCAqbXA7CiAJc3RydWN0IHZub2RlICpuZXdkcDsKIAlpbnQgZXJyb3I7CisJ Y2hhciAqZnNwYXRoLCAqZmJ1ZjsKIAogCW10eF9hc3NlcnQoJkdpYW50LCBNQV9PV05FRCk7 CiAJQVNTRVJUX1ZPUF9FTE9DS0VEKHZwLCBfX2Z1bmNfXyk7CiAJS0FTU0VSVCgoZnNmbGFn cyAmIE1OVF9VUERBVEUpID09IDAsICgiTU5UX1VQREFURSBzaG91bGRuJ3QgYmUgaGVyZSIp KTsKIAorCS8qIENvbnN0cnVjdCBnbG9iYWwgZmlsZXN5c3RlbSBwYXRoIGZyb20gdm5vZGUg Ki8KKwllcnJvciA9IHZuX2Z1bGxwYXRoX2dsb2JhbCh0ZCwgdnAsICZmc3BhdGgsICZmYnVm KTsKKwlpZiAoZXJyb3IgPT0gMCAmJiBzdHJsZW4oZnNwYXRoKSA+PSBNTkFNRUxFTikKKwkJ ZXJyb3IgPSBFTkFNRVRPT0xPTkc7CisJaWYgKGVycm9yICE9IDApIHsKKwkJdnB1dCh2cCk7 CisJCWlmIChmYnVmICE9IE5VTEwpCisJCQlmcmVlKGZidWYsIE1fVEVNUCk7CisJCXJldHVy bihlcnJvcik7CisJfQorCiAJLyoKIAkgKiBJZiB0aGUgdXNlciBpcyBub3Qgcm9vdCwgZW5z dXJlIHRoYXQgdGhleSBvd24gdGhlIGRpcmVjdG9yeQogCSAqIG9udG8gd2hpY2ggd2UgYXJl IGF0dGVtcHRpbmcgdG8gbW91bnQuCkBAIC03ODEsMTIgKzc5MiwxNCBAQAogCX0KIAlpZiAo ZXJyb3IgIT0gMCkgewogCQl2cHV0KHZwKTsKKwkJZnJlZShmYnVmLCBNX1RFTVApOwogCQly ZXR1cm4gKGVycm9yKTsKIAl9CiAJVk9QX1VOTE9DSyh2cCwgMCk7CiAKIAkvKiBBbGxvY2F0 ZSBhbmQgaW5pdGlhbGl6ZSB0aGUgZmlsZXN5c3RlbS4gKi8KIAltcCA9IHZmc19tb3VudF9h bGxvYyh2cCwgdmZzcCwgZnNwYXRoLCB0ZC0+dGRfdWNyZWQpOworCWZyZWUoZmJ1ZiwgTV9U RU1QKTsKIAkvKiBYWFhNQUM6IHBhc3MgdG8gdmZzX21vdW50X2FsbG9jPyAqLwogCW1wLT5t bnRfb3B0bmV3ID0gKm9wdGxpc3Q7CiAJLyogU2V0IHRoZSBtb3VudCBsZXZlbCBmbGFncy4g Ki8KQEAgLTEwNzAsNyArMTA4Myw3IEBACiAJTkRGUkVFKCZuZCwgTkRGX09OTFlfUE5CVUYp OwogCXZwID0gbmQubmlfdnA7CiAJaWYgKChmc2ZsYWdzICYgTU5UX1VQREFURSkgPT0gMCkg ewotCQllcnJvciA9IHZmc19kb21vdW50X2ZpcnN0KHRkLCB2ZnNwLCBmc3BhdGgsIHZwLCBm c2ZsYWdzLAorCQllcnJvciA9IHZmc19kb21vdW50X2ZpcnN0KHRkLCB2ZnNwLCB2cCwgZnNm bGFncywKIAkJICAgIG9wdGxpc3QpOwogCX0gZWxzZSB7CiAJCWVycm9yID0gdmZzX2RvbW91 bnRfdXBkYXRlKHRkLCB2cCwgZnNmbGFncywgb3B0bGlzdCk7CkBAIC0xMTA1LDcgKzExMTgs NyBAQAogCX0gKi8gKnVhcDsKIHsKIAlzdHJ1Y3QgbW91bnQgKm1wOwotCWNoYXIgKnBhdGhi dWY7CisJY2hhciAqcGF0aGJ1ZiwgKnJwYXRoYnVmOwogCWludCBlcnJvciwgaWQwLCBpZDE7 CiAKIAlBVURJVF9BUkdfVkFMVUUodWFwLT5mbGFncyk7CkBAIC0xMTM5LDEzICsxMTUyLDI4 IEBACiAJCX0KIAkJbXR4X3VubG9jaygmbW91bnRsaXN0X210eCk7CiAJfSBlbHNlIHsKLQkJ QVVESVRfQVJHX1VQQVRIMSh0ZCwgcGF0aGJ1Zik7CisJCS8qCisJCSAqIElmIHdlIGFyZSBq YWlsZWQgYW5kIGVuZm9yY2Vfc3RhdGZzID09IDEKKwkJICogY29uc3RydWN0IHJlYWwgZmls ZXN5c3RlbSBwYXRoCisJCSAqLworCQlycGF0aGJ1ZiA9IG1hbGxvYyhNTkFNRUxFTiwgTV9U RU1QLCBNX1dBSVRPSyk7CisJCWlmIChqYWlsZWQodGQtPnRkX3VjcmVkKSAmJgorCQkgICAg dGQtPnRkX3VjcmVkLT5jcl9wcmlzb24tPnByX2VuZm9yY2Vfc3RhdGZzID09IDEgJiYKKwkJ ICAgIHN0cmNtcCh0ZC0+dGRfdWNyZWQtPmNyX3ByaXNvbi0+cHJfcGF0aCwgIi8iKSAhPSAw KSB7CisJCQlzdHJsY3B5KHJwYXRoYnVmLCB0ZC0+dGRfdWNyZWQtPmNyX3ByaXNvbi0+cHJf cGF0aCwKKwkJCSAgICBNTkFNRUxFTik7CisJCQlzdHJsY2F0KHJwYXRoYnVmLCBwYXRoYnVm LCBNTkFNRUxFTik7CisJCX0gZWxzZSB7CisJCQlzdHJsY3B5KHJwYXRoYnVmLCBwYXRoYnVm LCBNTkFNRUxFTik7CisJCX0KKwkJQVVESVRfQVJHX1VQQVRIMSh0ZCwgcnBhdGhidWYpOwog CQltdHhfbG9jaygmbW91bnRsaXN0X210eCk7CiAJCVRBSUxRX0ZPUkVBQ0hfUkVWRVJTRSht cCwgJm1vdW50bGlzdCwgbW50bGlzdCwgbW50X2xpc3QpIHsKLQkJCWlmIChzdHJjbXAobXAt Pm1udF9zdGF0LmZfbW50b25uYW1lLCBwYXRoYnVmKSA9PSAwKQorCQkJaWYgKHN0cmNtcCht cC0+bW50X3N0YXQuZl9tbnRvbm5hbWUsIHJwYXRoYnVmKSA9PSAwKQogCQkJCWJyZWFrOwog CQl9CiAJCW10eF91bmxvY2soJm1vdW50bGlzdF9tdHgpOworCQlmcmVlKHJwYXRoYnVmLCBN X1RFTVApOwogCX0KIAlmcmVlKHBhdGhidWYsIE1fVEVNUCk7CiAJaWYgKG1wID09IE5VTEwp IHsK --------------020101090001040705070308-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 29 18:32:41 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FEAB106566C for ; Fri, 29 Jul 2011 18:32:41 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B48588FC08 for ; Fri, 29 Jul 2011 18:32:40 +0000 (UTC) Received: by wyg24 with SMTP id 24so516515wyg.13 for ; Fri, 29 Jul 2011 11:32:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.227.203.10 with SMTP id fg10mr2248017wbb.102.1311963049826; Fri, 29 Jul 2011 11:10:49 -0700 (PDT) Received: by 10.216.45.77 with HTTP; Fri, 29 Jul 2011 11:10:49 -0700 (PDT) Date: Fri, 29 Jul 2011 12:10:49 -0600 Message-ID: From: Will Andrews To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: ZFS sharenfs mangling X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2011 18:32:41 -0000 Hi, Why do the FreeBSD fsshare compatibility routines mangle the sharenfs value? As best as I can tell, this behavior only exists so that people don't have to quote their sharenfs value when setting it via 'zfs set'. However, this feature breaks using hyphens anywhere but in options. An user reported this problem last month: http://lists.freebsd.org/pipermail/freebsd-fs/2011-June/011784.html Removing translate_opts() (and the call from fsshare_main()) from cddl/compat/opensolaris/misc/fsshare.c fixed the problem for me, and had the desired result. The function could be tweaked to fix this problem, but I think it would be better to simply remove it. --Will. From owner-zfs-devel@FreeBSD.ORG Sun Jul 31 22:35:38 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2718B106566C; Sun, 31 Jul 2011 22:35:38 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id DC34F8FC0A; Sun, 31 Jul 2011 22:35:37 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 905DB1686D0; Mon, 1 Aug 2011 00:35:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id jI70UDYBgP-r; Mon, 1 Aug 2011 00:35:34 +0200 (CEST) Received: from [10.9.8.3] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 116941686C0; Mon, 1 Aug 2011 00:35:34 +0200 (CEST) Message-ID: <4E35D8B5.5080100@FreeBSD.org> Date: Mon, 01 Aug 2011 00:35:33 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------090600050800050909060601" Cc: zfs-devel@FreeBSD.ORG Subject: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 22:35:38 -0000 This is a multi-part message in MIME format. --------------090600050800050909060601 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit The txg_delay() function in txg.c uses the following initialization: int timeout = ddi_get_lbolt() + ticks; Later, we have: while (ddi_get_lbolt() < timeout && tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) (void) cv_timedwait(&tx->tx_quiesce_more_cv, &tx->tx_sync_lock, timeout - ddi_get_lbolt()); The function txg_delay() is called from: dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() In 24.855 days ddi_get_lbolt will be never smaller than timeout. Please review and/or comment the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090600050800050909060601 Content-Type: text/plain; name="txg.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="txg.c.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) @@ -488,7 +488,7 @@ txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) { tx_state_t *tx = &dp->dp_tx; - int timeout = ddi_get_lbolt() + ticks; + clock_t timeout = ddi_get_lbolt() + ticks; /* don't delay if this txg could transition to quiesing immediately */ if (tx->tx_open_txg > txg || --------------090600050800050909060601-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 1 13:35:57 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44647106564A for ; Mon, 1 Aug 2011 13:35:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 947808FC14 for ; Mon, 1 Aug 2011 13:35:56 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA26807; Mon, 01 Aug 2011 16:20:15 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E36A80E.8080909@FreeBSD.org> Date: Mon, 01 Aug 2011 16:20:14 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: Martin Matuska References: <4E35D8B5.5080100@FreeBSD.org> In-Reply-To: <4E35D8B5.5080100@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 13:35:57 -0000 on 01/08/2011 01:35 Martin Matuska said the following: > The txg_delay() function in txg.c uses the following initialization: > int timeout = ddi_get_lbolt() + ticks; > > Later, we have: > while (ddi_get_lbolt() < timeout && > tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > (void) cv_timedwait(&tx->tx_quiesce_more_cv, > &tx->tx_sync_lock, > timeout - ddi_get_lbolt()); > > The function txg_delay() is called from: > dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() > > In 24.855 days ddi_get_lbolt will be never smaller than timeout. > > Please review and/or comment the attached patch. > I agree with the patch - thank you for catching this bug! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 05:33:34 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4CB0106564A; Tue, 2 Aug 2011 05:33:34 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 67C208FC08; Tue, 2 Aug 2011 05:33:34 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 93CA0F81; Tue, 2 Aug 2011 07:33:31 +0200 (CEST) Date: Tue, 2 Aug 2011 07:33:24 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802053324.GG1685@garage.freebsd.pl> References: <4E35D8B5.5080100@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rV8arf8D5Dod9UkK" Content-Disposition: inline In-Reply-To: <4E35D8B5.5080100@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 05:33:34 -0000 --rV8arf8D5Dod9UkK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: > The txg_delay() function in txg.c uses the following initialization: > int timeout =3D ddi_get_lbolt() + ticks; >=20 > Later, we have: > while (ddi_get_lbolt() < timeout && > tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > (void) cv_timedwait(&tx->tx_quiesce_more_cv, > &tx->tx_sync_lock, > timeout - ddi_get_lbolt()); >=20 > The function txg_delay() is called from: > dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() >=20 > In 24.855 days ddi_get_lbolt will be never smaller than timeout. >=20 > Please review and/or comment the attached patch. Looks good to me. Can you elaborate a bit on consequences of such overflow? How the problem manifests itself? BTW. Is this something that affects IllumOS as well? If so, it would be nice to share with them. > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) > @@ -488,7 +488,7 @@ > txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) > { > tx_state_t *tx =3D &dp->dp_tx; > - int timeout =3D ddi_get_lbolt() + ticks; > + clock_t timeout =3D ddi_get_lbolt() + ticks; > =20 > /* don't delay if this txg could transition to quiesing immediately */ > if (tx->tx_open_txg > txg || --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --rV8arf8D5Dod9UkK Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43jCMACgkQForvXbEpPzQLeQCg3acNcUSKsXeyCcBQboKhvrf0 nzkAnjvW7+AAoxKN+wcC4D46PKjYoseq =6STl -----END PGP SIGNATURE----- --rV8arf8D5Dod9UkK-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:06:20 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F365E106564A; Tue, 2 Aug 2011 06:06:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD628FC12; Tue, 2 Aug 2011 06:06:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id B012B14BF3A; Tue, 2 Aug 2011 08:06:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WaZ+e4TDf84E; Tue, 2 Aug 2011 08:06:15 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 5D4D514BF2B; Tue, 2 Aug 2011 08:06:15 +0200 (CEST) Message-ID: <4E3793DA.7090506@FreeBSD.org> Date: Tue, 02 Aug 2011 08:06:18 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E35D8B5.5080100@FreeBSD.org> <20110802053324.GG1685@garage.freebsd.pl> In-Reply-To: <20110802053324.GG1685@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:06:20 -0000 Dňa 2. 8. 2011 7:33, Pawel Jakub Dawidek wrote / napísal(a): > On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: >> The txg_delay() function in txg.c uses the following initialization: >> int timeout = ddi_get_lbolt() + ticks; >> >> Later, we have: >> while (ddi_get_lbolt() < timeout && >> tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) >> (void) cv_timedwait(&tx->tx_quiesce_more_cv, >> &tx->tx_sync_lock, >> timeout - ddi_get_lbolt()); >> >> The function txg_delay() is called from: >> dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() >> >> In 24.855 days ddi_get_lbolt will be never smaller than timeout. >> >> Please review and/or comment the attached patch. > Looks good to me. Can you elaborate a bit on consequences of such > overflow? How the problem manifests itself? > > BTW. Is this something that affects IllumOS as well? If so, it would be > nice to share with them. > >> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c >> =================================================================== >> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) >> @@ -488,7 +488,7 @@ >> txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) >> { >> tx_state_t *tx = &dp->dp_tx; >> - int timeout = ddi_get_lbolt() + ticks; >> + clock_t timeout = ddi_get_lbolt() + ticks; >> >> /* don't delay if this txg could transition to quiesing immediately */ >> if (tx->tx_open_txg > txg || The overflow causes txg_delay() not delaying txg threads anymore. It is called from dsl_pool_tempreserve_space() in dsl_pool.c and from dsl_dir_tempreserve_space() in dsl_dir.c from busy transaction groups: dsl_pool.c: /* * If this transaction group is over 7/8ths capacity, delay * the caller 1 clock tick. This will slow down the "fill" * rate until the sync process can catch up with us. */ if (reserved && reserved > (write_limit - (write_limit >> 3))) txg_delay(dp, tx->tx_txg, 1); dsl_dir.c: err = arc_tempreserve_space(lsize, tx->tx_txg); if (err == 0) { struct tempreserve *tr; tr = kmem_zalloc(sizeof (struct tempreserve), KM_SLEEP); tr->tr_size = lsize; list_insert_tail(tr_list, tr); err = dsl_pool_tempreserve_space(dd->dd_pool, asize, tx); } else { if (err == EAGAIN) { txg_delay(dd->dd_pool, tx->tx_txg, 1); err = ERESTART; } dsl_pool_memory_pressure(dd->dd_pool); } In my interpretation, this overflow will increase thread concurrency and memory pressure on the pool and therefore slow down write speed. I have filed this yesterday as Illumos bug #1313. https://www.illumos.org/issues/1313 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:30:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA7B0106564A; Tue, 2 Aug 2011 06:30:14 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 684948FC13; Tue, 2 Aug 2011 06:30:14 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id D5E43FA3; Tue, 2 Aug 2011 08:30:12 +0200 (CEST) Date: Tue, 2 Aug 2011 08:30:07 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802063007.GB2789@garage.freebsd.pl> References: <4E35D8B5.5080100@FreeBSD.org> <20110802053324.GG1685@garage.freebsd.pl> <4E3793DA.7090506@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hHWLQfXTYDoKhP50" Content-Disposition: inline In-Reply-To: <4E3793DA.7090506@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:30:15 -0000 --hHWLQfXTYDoKhP50 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 02, 2011 at 08:06:18AM +0200, Martin Matuska wrote: > D=C5=88a 2. 8. 2011 7:33, Pawel Jakub Dawidek wrote / nap=C3=ADsal(a): > > On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: > >> The txg_delay() function in txg.c uses the following initialization: > >> int timeout =3D ddi_get_lbolt() + ticks; > >> > >> Later, we have: > >> while (ddi_get_lbolt() < timeout && > >> tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > >> (void) cv_timedwait(&tx->tx_quiesce_more_cv, > >> &tx->tx_sync_lock, > >> timeout - ddi_get_lbolt()); > >> > >> The function txg_delay() is called from: > >> dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() > >> > >> In 24.855 days ddi_get_lbolt will be never smaller than timeout. > >> > >> Please review and/or comment the attached patch. > > Looks good to me. Can you elaborate a bit on consequences of such > > overflow? How the problem manifests itself? > > > > BTW. Is this something that affects IllumOS as well? If so, it would be > > nice to share with them. > > > >> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224= 527) > >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) > >> @@ -488,7 +488,7 @@ > >> txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) > >> { > >> tx_state_t *tx =3D &dp->dp_tx; > >> - int timeout =3D ddi_get_lbolt() + ticks; > >> + clock_t timeout =3D ddi_get_lbolt() + ticks; > >> =20 > >> /* don't delay if this txg could transition to quiesing immediately = */ > >> if (tx->tx_open_txg > txg || > The overflow causes txg_delay() not delaying txg threads anymore. > It is called from dsl_pool_tempreserve_space() in dsl_pool.c and from > dsl_dir_tempreserve_space() in dsl_dir.c from busy transaction groups: >=20 > dsl_pool.c: > /* > * If this transaction group is over 7/8ths capacity, delay > * the caller 1 clock tick. This will slow down the "fill" > * rate until the sync process can catch up with us. > */ > if (reserved && reserved > (write_limit - (write_limit >> 3))) > txg_delay(dp, tx->tx_txg, 1); >=20 > dsl_dir.c: > err =3D arc_tempreserve_space(lsize, tx->tx_txg); > if (err =3D=3D 0) { > struct tempreserve *tr; >=20 > tr =3D kmem_zalloc(sizeof (struct tempreserve), KM_SLEEP); > tr->tr_size =3D lsize; > list_insert_tail(tr_list, tr); >=20 > err =3D dsl_pool_tempreserve_space(dd->dd_pool, asize, tx); > } else { > if (err =3D=3D EAGAIN) { > txg_delay(dd->dd_pool, tx->tx_txg, 1); > err =3D ERESTART; > } > dsl_pool_memory_pressure(dd->dd_pool); > } >=20 > In my interpretation, this overflow will increase thread concurrency and > memory pressure on the pool and therefore slow down write speed. >=20 > I have filed this yesterday as Illumos bug #1313. > https://www.illumos.org/issues/1313 Ok, thanks. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --hHWLQfXTYDoKhP50 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43mW8ACgkQForvXbEpPzSlqACgpZuBy5otS4BHXvp315Z3nx75 VY4An05rl8HVk7heeEDrMqde6XqA8puW =maWI -----END PGP SIGNATURE----- --hHWLQfXTYDoKhP50-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:39:06 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44ACD1065670; Tue, 2 Aug 2011 06:39:06 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id B979B8FC12; Tue, 2 Aug 2011 06:39:05 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 2A95114DF6F; Tue, 2 Aug 2011 08:39:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id mQD208TISvo9; Tue, 2 Aug 2011 08:39:02 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 94C8C14DF4E; Tue, 2 Aug 2011 08:39:02 +0200 (CEST) Message-ID: <4E379B89.9040409@FreeBSD.org> Date: Tue, 02 Aug 2011 08:39:05 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , zfs-devel@FreeBSD.ORG References: <20110802044718.GA8838@gw.zagrebin.ru> In-Reply-To: <20110802044718.GA8838@gw.zagrebin.ru> X-Enigmail-Version: 1.2 X-Forwarded-Message-Id: <20110802044718.GA8838@gw.zagrebin.ru> Content-Type: multipart/mixed; boundary="------------000101040805060109030304" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Fwd: ZFS v28: kernel panics while reading an extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:39:06 -0000 This is a multi-part message in MIME format. --------------000101040805060109030304 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I can confirm the bug found by Alexander Zagrebin in ZFS v28, he suggests the attached bugfix. Is the patch acceptable? How to reproduce the bug: setextattr user testattr 1 testfile zfs snapshot test@s1 getextattr user testattr .zfs/snapshot/s1/testfile PANIC ----- Original message ----- Subject: ZFS v28: kernel panics while reading an extended attribute Date: Tue, 2 Aug 2011 08:47:19 +0400 From: Alexander Zagrebin To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Cc: freebsd-current@freebsd.org Hi! It seems, I've found a bug in the ZFS v28 on the latest stable: if we have a snapshot with some files having an extended attributes, then attempt to read an extended attributes's value leads to a well reproducible kernel panic. The part of backtrace follows: #6 0xffffffff804bbe44 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff80950ea7 in zil_commit (zilog=0x0, foid=5795917) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1497 #8 0xffffffff80979e6b in zfs_freebsd_read (ap=Variable "ap" is not available.) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:622 #9 0xffffffff80979750 in zfs_getextattr (ap=0xffffff80dd5d8820) at vnode_if.h:384 #10 0xffffffff8038921b in extattr_get_vp (vp=0xffffff0056a01588, attrnamespace=1, attrname=0xffffff80dd5d89a0 "DOSATTRIB", data=Variable "data" is not available.) at vnode_if.h:1332 It seems that ZIL isn't available for snapshots, but zfs_freebsd_read doesn't check this when calling zil_commit. The attached patch fixes this issue. Can anybody confirm this? -- Alexander Zagrebin --------------000101040805060109030304 Content-Type: text/x-diff; name="patch-zfs_vnops.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="patch-zfs_vnops.c" --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c.orig 2011-08-01 23:04:07.358173627 +0400 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c 2011-08-02 00:10:02.674585604 +0400 @@ -618,7 +618,8 @@ zfs_read(vnode_t *vp, uio_t *uio, int io /* * If we're in FRSYNC mode, sync out this znode before reading it. */ - if (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + if (zfsvfs->z_log && + (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS)) zil_commit(zfsvfs->z_log, zp->z_id); /* --------------000101040805060109030304 Content-Type: text/plain; name="=?windows-1250?Q?=C8as=9D_prilo=9Eenej_spr=E1vy?=" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0*=windows-1250''%C8%61%73%9D%20%70%72%69%6C%6F%9E%65%6E%65%6A%20; filename*1*=%73%70%72%E1%76%79 _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --------------000101040805060109030304-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 07:52:21 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF8C5106568C; Tue, 2 Aug 2011 07:52:21 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 461E08FC1D; Tue, 2 Aug 2011 07:52:21 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id EEB9EFD5; Tue, 2 Aug 2011 09:52:18 +0200 (CEST) Date: Tue, 2 Aug 2011 09:52:13 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802075211.GC2789@garage.freebsd.pl> References: <20110802044718.GA8838@gw.zagrebin.ru> <4E379B89.9040409@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZmUaFz6apKcXQszQ" Content-Disposition: inline In-Reply-To: <4E379B89.9040409@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: Fwd: ZFS v28: kernel panics while reading an extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 07:52:23 -0000 --ZmUaFz6apKcXQszQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 02, 2011 at 08:39:05AM +0200, Martin Matuska wrote: > I can confirm the bug found by Alexander Zagrebin in ZFS v28, he > suggests the attached bugfix. > Is the patch acceptable? >=20 > How to reproduce the bug: > setextattr user testattr 1 testfile > zfs snapshot test@s1 > getextattr user testattr .zfs/snapshot/s1/testfile > PANIC This is because zfs_getextattr() calls VOP_READ(9) with IO_SYNC flag. I was wondering if we could trigger this panic from regular read(2), but regular read never passes IO_SYNC flag. So I think this fix is correct, but I'd also remove IO_SYNC flag from VOP_READ() call inside zfs_getextattr(), as I don't see the point in having it there. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --ZmUaFz6apKcXQszQ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43rKsACgkQForvXbEpPzSoYQCfVMtC23JY/xqeoq0WT8cdgmkL oecAn3SGSEOQsCaxAyx5khIQnuHuwPmT =I4uj -----END PGP SIGNATURE----- --ZmUaFz6apKcXQszQ-- From owner-zfs-devel@FreeBSD.ORG Thu Aug 4 07:03:51 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B1F51065670 for ; Thu, 4 Aug 2011 07:03:51 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 381038FC12 for ; Thu, 4 Aug 2011 07:03:50 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 5B3EB18C97B for ; Thu, 4 Aug 2011 09:03:49 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id EXhXDNlJcXUH for ; Thu, 4 Aug 2011 09:03:47 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 3D3DA18C961 for ; Thu, 4 Aug 2011 09:03:45 +0200 (CEST) Message-ID: <4E3A4450.5090100@FreeBSD.org> Date: Thu, 04 Aug 2011 09:03:44 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: New ZFS changes in Illumos (13314) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 07:03:51 -0000 Illumos has made another change to ZFS, this time dealing with two of their isuses: 734 taskq_dispatch_prealloc() desired 943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP New code (changeset 13414): http://hg.openindiana.org/illumos-gate/rev/b42c1f0432b6 Would be implementing this change to our code of value? Issue 734: https://www.illumos.org/issues/734 Description: We would like an in-kernel interface for non-dynamic taskq's where the caller supplies the storage for the taskq_ent_t. The reason for this is to avoid hitting the kmem_alloc()/kmem_free() calls on very hot code paths. The other motivation (perhaps bigger) is that if we can avoid the allocation, we can avoid having to deal with scheduling failures on hot code paths, or in situations where it might not be easy to recover (e.g. interrupt context) -- by providing the storage before hand, the taskq_dispatch_prealloc() can degenerate to mutex_enter, twiddle list pointers, cv_broadcast/signal, mutex_exit. Easy peasey. Issue 943: https://www.illumos.org/issues/943 Description: While digging through bug reports and such looking for hints toward an end-user deadlock, I happened upon a report suggesting that ZFS was calling taskq_dispatch with TQ_SLEEP while in interrupt context. Code inspection appears to bear this out. The call chain ends up being: sdintr --> ... --> biodone --> vdev_disk_io_intr --> zio_interrupt --> zio_taskq_dispatch --> taskq_dispatch, where zio_taskq_dispatch explicitly ors TQ_SLEEP into any flags argument: source:usr/src/uts/common/fs/zfs/zio.c#L1056 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sun Aug 7 08:52:55 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D1A6106566C; Sun, 7 Aug 2011 08:52:55 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5212D8FC0C; Sun, 7 Aug 2011 08:52:55 +0000 (UTC) Received: from delta.delphij.net (unknown [IPv6:2001:470:83bf:0:221:5cff:fe6a:37bb]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 1A5B81593A; Sun, 7 Aug 2011 01:52:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1312707175; bh=reIeV/7rzxiSTk5lCdUdAGb2Zc36cv4OsBb9yvKu2+M=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: Content-Type; b=5HNJUIqKhsBg5+5W6tuCabd+BmeEXSAYUOYz6QCq7N49pQELFzMS+CWLnQIKtq9Wb 5H9BPCfy9u15Y3hX3I3gOXsp6MirgQmsMiRdiHhyLANlAe8y3oeWOYaPpMSDfww7Qq Rj2smTU4By41fsIPWsQf96w6A2+jWgLmwkkrGmvw= Message-ID: <4E3E525D.1040906@delphij.net> Date: Sun, 07 Aug 2011 01:52:45 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------080503040005000208080600" Cc: Josh Paetzel , Xin LI , Doug White Subject: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2011 08:52:55 -0000 This is a multi-part message in MIME format. --------------080503040005000208080600 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I have observed this on a test system which have both L2ARC and ZIL device installed, and several vdev groups. This can be triggered by removing the l2arc and zil device (detaching l2arc would fail), blocking at 'spa_namespace_lock' After reboot, the pool can not be imported. 'zpool import' shows the zpool command stuck with 'spa_namespace_lock'. I have a theory about the issue but the patch was not tested, maybe someone who is more familiar with the code can shed me some light? Thread 1 (zpool) Thread 2 (l2arc_feed_thread) Calls vdev_open() vdev_open() can now assert RW_WRITER on spa_config; [*1] vdev_open() calls vdev_geom_open() vdev_geom_open() called with spa_namespace_lock held, sets lock=1 and drops spa_namespace_lock, DROP_GIANT(), g_topology_lock() Races in, proceed to l2arc_dev_get_next Acquire &spa_namespace_lock; [*2] Found a device and then spa_config_enter(RW_READER) spa_config_enter blocks waiting writer to finish Tries to obtain spa_namespace_lock [*3] This creates a deadlock situation. My proposed solution is to use spa_config_tryenter() instead of spa_config_enter() in arc.c (see attachment). Comments? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOPlJcAAoJEATO+BI/yjfB4UoH/j3pK5rQ4iF52HmUwzMzKKol UVXHmoUPbV1Ibhpajcz6LprSL7b/SI50a2yelIr+1ZjgC22Lrw6fFird90JfeXF7 YtQ1LyVVuDbMA5KfM6wD8linm3HYri88DQ2CDtUqZesZ1w7PH0XNYnRKRYcEGRem 1JG09BMl0EqGuvKSy+69UnE5dPTRnzrAQOe3xBB4LntZBeapwMy5F8gcBY5XP5Lh W2lBJFcdFX3Zh390fUi1dxY5uoXQLfO5gu+YCXF7Zie2PLl596ZUjAEA3CsE/9yw qWBSBGufMP4gK24j1EulxhmoIU993vGtqc3iZhs1TugfjGbHAlXiHQEcmJnaEk8= =zkTh -----END PGP SIGNATURE----- --------------080503040005000208080600 Content-Type: text/plain; name="arc.c.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="arc.c.diff" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv YXJjLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91 dHMvY29tbW9uL2ZzL3pmcy9hcmMuYwkocmV2aXNpb24gMjI0NjUyKQorKysgc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9hcmMuYwkod29ya2luZyBj b3B5KQpAQCAtNDIyNCw4ICs0MjI0LDEwIEBAIG91dDoKIAkgKiBHcmFiIHRoZSBjb25maWcg bG9jayB0byBwcmV2ZW50IHRoZSAnbmV4dCcgZGV2aWNlIGZyb20gYmVpbmcKIAkgKiByZW1v dmVkIHdoaWxlIHdlIGFyZSB3cml0aW5nIHRvIGl0LgogCSAqLwotCWlmIChuZXh0ICE9IE5V TEwpCi0JCXNwYV9jb25maWdfZW50ZXIobmV4dC0+bDJhZF9zcGEsIFNDTF9MMkFSQywgbmV4 dCwgUldfUkVBREVSKTsKKwlpZiAobmV4dCAhPSBOVUxMKSB7CisJCWlmICghc3BhX2NvbmZp Z190cnllbnRlcihuZXh0LT5sMmFkX3NwYSwgU0NMX0wyQVJDLCBuZXh0LCBSV19SRUFERVIp KQorCQkJbmV4dCA9IE5VTEw7CisJfQogCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9j ayk7CiAKIAlyZXR1cm4gKG5leHQpOwo= --------------080503040005000208080600-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 8 13:28:48 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68559106564A; Mon, 8 Aug 2011 13:28:48 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 12CE78FC16; Mon, 8 Aug 2011 13:28:47 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 9F5E17D8; Mon, 8 Aug 2011 15:28:46 +0200 (CEST) Date: Mon, 8 Aug 2011 15:28:37 +0200 From: Pawel Jakub Dawidek To: d@delphij.net Message-ID: <20110808132837.GH1640@garage.freebsd.pl> References: <4E3E525D.1040906@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fCcDWlUEdh43YKr8" Content-Disposition: inline In-Reply-To: <4E3E525D.1040906@delphij.net> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Josh Paetzel , zfs-devel@FreeBSD.ORG, gibbs@FreeBSD.org, Xin LI , Doug White Subject: Re: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2011 13:28:48 -0000 --fCcDWlUEdh43YKr8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 07, 2011 at 01:52:45AM -0700, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 >=20 > Hi, >=20 > I have observed this on a test system which have both L2ARC and ZIL > device installed, and several vdev groups. This can be triggered by > removing the l2arc and zil device (detaching l2arc would fail), blocking > at 'spa_namespace_lock' >=20 > After reboot, the pool can not be imported. 'zpool import' shows the > zpool command stuck with 'spa_namespace_lock'. >=20 > I have a theory about the issue but the patch was not tested, maybe > someone who is more familiar with the code can shed me some light? Yes, your theory is correct, but the problem is wider (there are other cases it might deadlock as we discussed with Justin (CCed)). The reason for it is that we have two GEOM classes in ZFS: ZVOL and VDEV_GEOM. ZVOL calls from GEOM into ZFS and VDEV_GEOM calls from ZFS into GEOM. This creates lock order reversals. While working on ZFSv28 I cleaned up locking, so that locks are always acquired in the following order: zfsdev_state_lock g_topology_lock spa_namespace_lock Unfortunately I missed spa_config lock, which is special and not handled by WITNESS and the problem you are seeing is indeed related to that lock (reversal between spa_namespace_lock and spa_config lock). I just tried to simplify the locking by eliminating the zfsdev_state_lock altogether and replacing it with the spa_namespace_lock. Xin, could you try the following patch: http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch Justin, what do you think? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --fCcDWlUEdh43YKr8 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4/5IUACgkQForvXbEpPzQzkwCg7v4w+z88LdCjwVlYnPdpmsYZ vjoAoIfbGMnUkGY9wM9ipjJYYEGETnvN =eKTo -----END PGP SIGNATURE----- --fCcDWlUEdh43YKr8-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 9 21:15:35 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BDA5106566B for ; Tue, 9 Aug 2011 21:15:35 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 9BFB28FC0A for ; Tue, 9 Aug 2011 21:15:34 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id C41A41914EA for ; Tue, 9 Aug 2011 23:15:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Hs-5gOokOacU for ; Tue, 9 Aug 2011 23:15:31 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 4F9B31914E4 for ; Tue, 9 Aug 2011 23:15:31 +0200 (CEST) Message-ID: <4E41A377.8060103@FreeBSD.org> Date: Tue, 09 Aug 2011 23:15:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: [BUG] cache corruption on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Aug 2011 21:15:35 -0000 Hi, a user has wrongly described a bug report (kern/156933) but has contacted me recently. I wrote a script that reproduces the bug, it has nothing to do with readonly=on. The following script corrupts cache on snapshot receive (not reproducible on Solaris): #!/bin/sh zpool destroy tpool dd if=/dev/zero of=/tmp/poolfile bs=1m count=64 zpool create tpool /tmp/poolfile zfs create tpool/ds1 zfs create tpool/ds2 echo "Test line 1" > /tpool/ds1/test.txt zfs snapshot tpool/ds1@s1 zfs send tpool/ds1@s1 | zfs recv -F tpool/ds2 echo "Test line 2" >> /tpool/ds1/test.txt zfs snapshot tpool/ds1@s2 # This causes corrupted FS cache after 2nd recv tail /tpool/ds2/test.txt # zfs send -i @s1 tpool/ds1@s2 | zfs recv -F tpool/ds2 md5 /tpool/ds1/test.txt md5 /tpool/ds2/test.txt If you umount + mount tpool/ds2, the file is correct again. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Aug 10 18:05:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DD7A106564A for ; Wed, 10 Aug 2011 18:05:13 +0000 (UTC) (envelope-from martin@matuska.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 47D6C8FC14 for ; Wed, 10 Aug 2011 18:05:12 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 5864B170322 for ; Wed, 10 Aug 2011 20:05:11 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id mRuzTQSH5lU5 for ; Wed, 10 Aug 2011 20:05:09 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id F412017030D for ; Wed, 10 Aug 2011 20:05:08 +0200 (CEST) Message-ID: <4E42C859.5000909@matuska.org> Date: Wed, 10 Aug 2011 20:05:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------020407000507000703080808" Cc: Subject: [PATCH] fix ZFS prefetch code X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2011 18:05:13 -0000 This is a multi-part message in MIME format. --------------020407000507000703080808 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch is a solution for kern/157728. The new zfs send/recv code doesn't play well with prefetching temporary clones. In Illumos, the prefetch code in zfs_ioc_dataset_list_next() is different from our code: /* * Pre-fetch the datasets. dmu_objset_prefetch() always returns 0 * but is not declared void because its called by dmu_objset_find(). */ if (zc->zc_cookie == 0) { uint64_t cookie = 0; int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) (void) dmu_objset_prefetch(p NULL); } Compared to our code: /* * Pre-fetch the datasets. dmu_objset_prefetch() always returns 0 * but is not declared void because its called by dmu_objset_find(). */ if (zc->zc_cookie == 0) { uint64_t cookie = 0; int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) (void) dmu_objset_prefetch(zc->zc_name, NULL); } I have done some debugging and Pawel's change is right, with "p" it doesn't work, dmu_objset_prefetch() rejects all entries on dmu_objset_hold() and everything is just skipped because names are incorrect) -> Solaris and Illumos do not prefetch datasets. How does this race happen? At the end of dmu_recv_existing_end() we destroy the temporary clone without checking any result code: (void) dsl_dataset_destroy(drc->drc_real_ds, dmu_recv_tag, B_FALSE); dsl_dataset_destroy() fails if there is an extra hold on the dataset - and dmu_objset_prefetch() is the function that places this hold. Because the prefetch code in Solaris doesn't work (is unfixed), the don't have this race. Now we have this code and comments in dmu_objset.c (dmu_objset_snapshot_one()): /* * If the objset starts with a '%', then ignore it unless it was * explicitly named (ie, not recursive). These hidden datasets * are always inconsistent, and by not opening them here, we can * avoid a race with dsl_dir_destroy_check(). */ cp = strrchr(name, '/'); if (cp && cp[1] == '%' && sn->recursive) return (0); (void) strcpy(sn->failed, name); /* * Check permissions if we are doing a recursive snapshot. The * permission checks for the starting dataset have already been * performed in zfs_secpolicy_snapshot() */ if (sn->recursive && (err = zfs_secpolicy_snapshot_perms(name, CRED()))) return (err); err = dmu_objset_hold(name, sn, &os); if (err != 0) return (err); /* * If the objset is in an inconsistent state (eg, in the process * of being destroyed), don't snapshot it. As with %hidden * datasets, we return EBUSY if this name was explicitly * requested (ie, not recursive), and otherwise ignore it. */ if (os->os_dsl_dataset->ds_phys->ds_flags & DS_FLAG_INCONSISTENT) { dmu_objset_rele(os, sn); return (sn->recursive ? 0 : EBUSY); } This is exactly the same race I am talking about. So when these datasets count as allways inconsistent, we have to do the same in dmu_objset_prefetch() and I suggest we don't prefetch inconsistent datasets. I am submitting this to Illumos as well. Please review attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020407000507000703080808 Content-Type: text/plain; name="dmu_objset.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmu_objset.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZG11X29ianNldC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNv bGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvZG11X29ianNldC5jCShyZXZpc2lvbiAyMjQ3NjAp CisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL2Rt dV9vYmpzZXQuYwkod29ya2luZyBjb3B5KQpAQCAtMTc2MCwxMCArMTc2MCwyOSBAQAogZG11 X29ianNldF9wcmVmZXRjaChjb25zdCBjaGFyICpuYW1lLCB2b2lkICphcmcpCiB7CiAJZHNs X2RhdGFzZXRfdCAqZHM7CisJY2hhciAqY3A7CiAKKwkvKgorCSAqIElmIHRoZSBvYmpzZXQg c3RhcnRzIHdpdGggYSAnJScsIHRoZW4gaWdub3JlIGl0LgorCSAqIFRoZXNlIGhpZGRlbiBk YXRhc2V0cyBhcmUgYWx3YXlzIGluY29uc2lzdGVudCBhbmQgYnkgbm90IG9wZW5pbmcKKwkg KiB0aGVtIGhlcmUsIHdlIGNhbiBhdm9pZCBhIHJhY2Ugd2l0aCBkc2xfZGlyX2Rlc3Ryb3lf Y2hlY2soKS4KKwkgKi8KKwljcCA9IHN0cnJjaHIobmFtZSwgJy8nKTsKKwlpZiAoY3AgJiYg Y3BbMV0gPT0gJyUnKQorCQlyZXR1cm4gKDApOworCiAJaWYgKGRzbF9kYXRhc2V0X2hvbGQo bmFtZSwgRlRBRywgJmRzKSkKIAkJcmV0dXJuICgwKTsKIAorCS8qCisJICogSWYgdGhlIG9i anNldCBpcyBpbiBhbiBpbmNvbnNpc3RlbnQgc3RhdGUgKGVnLCBpbiB0aGUgcHJvY2Vzcwor CSAqIG9mIGJlaW5nIGRlc3Ryb3llZCksIGRvbid0IHByZWZldGNoIGl0LgorCSAqLworCWlm IChkcy0+ZHNfcGh5cy0+ZHNfZmxhZ3MgJiBEU19GTEFHX0lOQ09OU0lTVEVOVCkgeworCQlk c2xfZGF0YXNldF9yZWxlKGRzLCBGVEFHKTsKKwkJcmV0dXJuICgwKTsKKwl9CisKIAlpZiAo IUJQX0lTX0hPTEUoJmRzLT5kc19waHlzLT5kc19icCkpIHsKIAkJbXV0ZXhfZW50ZXIoJmRz LT5kc19vcGVuaW5nX2xvY2spOwogCQlpZiAoZHMtPmRzX29ianNldCA9PSBOVUxMKSB7Cg== --------------020407000507000703080808-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 10 19:49:36 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88C15106564A; Wed, 10 Aug 2011 19:49:36 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 12F948FC0A; Wed, 10 Aug 2011 19:49:35 +0000 (UTC) Received: by vxh11 with SMTP id 11so1532550vxh.13 for ; Wed, 10 Aug 2011 12:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=JYAfSnHXd0+P82z/vPT9miQVbJ3grgbjP1tYh3+zEgQ=; b=N4zcv9OPcY58PJFHd4dzBDAlwXlFVXlXSzrRq0mKWUjy9/2KSPUcexUxiB3uih4LE+ OE3n1eeJ09k3aVaax8BU7Swpr0ijINNKY6/zX/S0mKtZwKO6poKpVXZvLT7/eskHWsju Y2psupvatvo7KMRbmI9RdDRc8e5rDvLpvGCjc= MIME-Version: 1.0 Received: by 10.52.90.135 with SMTP id bw7mr9052681vdb.279.1313004405764; Wed, 10 Aug 2011 12:26:45 -0700 (PDT) Received: by 10.52.110.166 with HTTP; Wed, 10 Aug 2011 12:26:45 -0700 (PDT) In-Reply-To: <20110808132837.GH1640@garage.freebsd.pl> References: <4E3E525D.1040906@delphij.net> <20110808132837.GH1640@garage.freebsd.pl> Date: Wed, 10 Aug 2011 12:26:45 -0700 Message-ID: From: Xin LI To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Josh Paetzel , zfs-devel@freebsd.org, Xin LI , Doug White Subject: Re: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2011 19:49:36 -0000 Hi, Pawel, > Xin, could you try the following patch: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0http://people.freebsd.org/~pjd/patches/zfsdev_= state_lock.patch For the record, our tests can no longer reproduce the issue I've mentioned before, with the patch applied. Could you please commit this patch against -HEAD? Cheers, --=20 Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-zfs-devel@FreeBSD.ORG Fri Aug 12 14:29:04 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 480F8106566B; Fri, 12 Aug 2011 14:29:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C16E38FC08; Fri, 12 Aug 2011 14:29:03 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 21D4F190E1E; Fri, 12 Aug 2011 16:29:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZJq4MevRWFXd; Fri, 12 Aug 2011 16:29:00 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 48289190E04; Fri, 12 Aug 2011 16:28:59 +0200 (CEST) Message-ID: <4E4538B0.8080508@FreeBSD.org> Date: Fri, 12 Aug 2011 16:29:04 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li , Andriy Gapon X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------040508030707020806090701" Cc: zfs-devel@freebsd.org Subject: [PATCH] zfs prefetch patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2011 14:29:04 -0000 This is a multi-part message in MIME format. --------------040508030707020806090701 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Please review my attached patch, I have decided to go for the minimalistic version for 9.0-RELEASE. Since upgrade to zfs v28 I have been experiencing temporary clones left over (not deleted) by incremental zfs receive. This made the parent snapshots undeletable. This has been also reported by several users in mailing lists. I have investigated this issue, and the cause is a race between dmu_objset_prefetch() invoked from zfs_ioc_dataset_list_next() and dsl_dir_destroy_check() indirectly invoked from dmu_recv_existing_end() via dsl_dataset_destroy(). In addition, the calling of dmu_objset_prefetch() from with internal datasets, temporary clones and invisible datasets is useless as it either fails (internal datsets) or we are not going to process these datasets later (they are skipped in the following code). Last, fix calling of dmu_objset_prefetch() from zvol_create_minors() by using absolute instead of relative dataset name. PR: kern/157728 (reported on Jun 9, 2011) Filed as Illumos bug #1346 https://www.illumos.org/issues/1346 Thank you very much. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------040508030707020806090701 Content-Type: text/plain; name="zfs_prefetch.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_prefetch.patch" Index: zfs_ioctl.c =================================================================== --- zfs_ioctl.c (revision 224794) +++ zfs_ioctl.c (working copy) @@ -1964,7 +1964,8 @@ zfs_ioc_dataset_list_next(zfs_cmd_t *zc) int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) - (void) dmu_objset_prefetch(zc->zc_name, NULL); + if (dataset_name_hidden(zc->zc_name) == B_FALSE) + (void) dmu_objset_prefetch(zc->zc_name, NULL); } do { Index: zvol.c =================================================================== --- zvol.c (revision 224794) +++ zvol.c (working copy) @@ -2197,12 +2197,9 @@ zvol_create_minors(const char *name) p = osname + strlen(osname); len = MAXPATHLEN - (p - osname); - if (strchr(name, '/') == NULL) { - /* Prefetch only for pool name. */ - cookie = 0; - while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) - (void) dmu_objset_prefetch(p, NULL); - } + /* Prefetch only for pool name. */ + if (strchr(name, '/') == NULL) + (void) dmu_objset_prefetch(name, NULL); cookie = 0; while (dmu_dir_list_next(os, MAXPATHLEN - (p - osname), p, NULL, --------------040508030707020806090701 Content-Type: text/plain; name="zfs_prefetch.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_prefetch.txt" Rml4IHJhY2UgYmV0d2VlbiBkbXVfb2Jqc2V0X3ByZWZldGNoKCkgaW52b2tlZCBmcm9tCnpm c19pb2NfZGF0YXNldF9saXN0X25leHQoKSBhbmQgZHNsX2Rpcl9kZXN0cm95X2NoZWNrKCkg aW5kaXJlY3RseQppbnZva2VkIGZyb20gZG11X3JlY3ZfZXhpc3RpbmdfZW5kKCkgdmlhIGRz bF9kYXRhc2V0X2Rlc3Ryb3koKSBieSBub3QKcHJlZmV0Y2hpbmcgdGVtcG9yYXJ5IGNsb25l cywgYXMgdGhlc2UgY291bnQgYXMgYWx3YXlzIGluY29uc2lzdGVudC4KSW4gYWRkaXRpb24s IGRvIG5vdCBwcmVmZXRjaCBoaWRkZW4gZGF0YXNldHMgYXQgYWxsIGFzIHdlIGFyZSBub3QK Z29pbmcgdG8gcHJvY2VzcyB0aGVzZSBsYXRlci4gWzFdCgpGaXggY2FsbGluZyBvZiBkbXVf b2Jqc2V0X3ByZWZldGNoKCkgZnJvbSB6dm9sX2NyZWF0ZV9taW5vcnMoKQpieSB1c2luZyBh YnNvbHV0ZSBpbnN0ZWFkIG9mIHJlbGF0aXZlIGRhdGFzZXQgbmFtZS4KCkZpbGVkIGFzIEls bHVtb3MgQnVnIDEzNDYgWzFdCgpQUjoJCQkJa2Vybi8xNTc3MjggWzFdClRlc3RlZCBieToJ CUJvcmphIE1hcmNvcyA8Ym9yamFtQHNhcmVuZXQuZXM+IFsxXSwgbW0KQXBwcm92ZWQgYnk6 CXJlICgpCk1GQyBhZnRlcjoJCTEgd2VlayA= --------------040508030707020806090701-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 13:03:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8846E1065673; Wed, 24 Aug 2011 13:03:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 21A088FC14; Wed, 24 Aug 2011 13:03:15 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 3EF6C197CA3; Wed, 24 Aug 2011 15:03:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CHw1yIMx1S6U; Wed, 24 Aug 2011 15:03:11 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 74BBA197C97; Wed, 24 Aug 2011 15:03:09 +0200 (CEST) Message-ID: <4E54F689.4010600@FreeBSD.org> Date: Wed, 24 Aug 2011 15:03:05 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------090805040003050802020905" Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: [PATCH] fix for kern/160035 for review X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 13:03:15 -0000 This is a multi-part message in MIME format. --------------090805040003050802020905 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch fixes kern/160035 and kern/156933. http://www.freebsd.org/cgi/query-pr.cgi?pr=160035 The function zfs_rezget() is called as part of zfs_resume_fs(). Suspending and resuming a ZFS filesystem is used in three cases: 1.) zfs rollback 2.) snapshot receive on a mounted fs (zfs recv -F) 3.) zfs upgrade To make sure we don't have mmaped wrong data we have to invalidate buffers for all vnodes. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090805040003050802020905 Content-Type: text/plain; name="zfs_znode.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_znode.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv emZzX3pub2RlLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfem5vZGUuYwkocmV2aXNpb24gMjI1MTQwKQor Kysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNf em5vZGUuYwkod29ya2luZyBjb3B5KQpAQCAtMTM0Myw2ICsxMzQzLDkgQEAgemZzX3Jlemdl dCh6bm9kZV90ICp6cCkKIAogCXpwLT56X3VubGlua2VkID0gKHpwLT56X2xpbmtzID09IDAp OwogCXpwLT56X2Jsa3N6ID0gZG9pLmRvaV9kYXRhX2Jsb2NrX3NpemU7CisKKwlpZiAodmlu dmFsYnVmKFpUT1YoenApLCBWX1NBVkUsIDAsIDApICE9IDApCisJCXZpbnZhbGJ1ZihaVE9W KHpwKSwgMCwgMCwgMCk7CiAJaWYgKHpwLT56X3NpemUgIT0gc2l6ZSAmJiBaVE9WKHpwKSAh PSBOVUxMKQogCQl2bm9kZV9wYWdlcl9zZXRzaXplKFpUT1YoenApLCB6cC0+el9zaXplKTsK IAo= --------------090805040003050802020905-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:38:03 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B17B1065673; Wed, 24 Aug 2011 17:38:03 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C44E68FC0C; Wed, 24 Aug 2011 17:38:02 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5B53197341; Wed, 24 Aug 2011 19:38:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DufxESP22zL3; Wed, 24 Aug 2011 19:38:00 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id B1FE019732C; Wed, 24 Aug 2011 19:37:58 +0200 (CEST) Message-ID: <4E5536F6.9020200@FreeBSD.org> Date: Wed, 24 Aug 2011 19:37:58 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824134514.GR17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------000207000804020509030203" Cc: zfs-devel@freebsd.org, Pawel Jakub Dawidek Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:38:03 -0000 This is a multi-part message in MIME format. --------------000207000804020509030203 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 24. 8. 2011 15:45, Kostik Belousov wrote: > On Wed, Aug 24, 2011 at 03:14:47PM +0200, Martin Matuska wrote: >> Hi Kostik, >> >> could you please review the attached patch? >> >> It fixes kern/160035 and kern/156933. >> http://www.freebsd.org/cgi/query-pr.cgi?pr=160035 >> >> The function zfs_rezget() is called as part of zfs_resume_fs(). >> Suspending and resuming a ZFS filesystem is used in three cases: >> 1.) zfs rollback >> 2.) snapshot receive on a mounted fs (zfs recv -F) >> 3.) zfs upgrade >> >> In case 1.) and 2.) the filesystem usually changes (changed/deleted/new files/directories). >> >> To make sure we don't have mmaped wrong data we have to invalidate >> buffers for all vnodes of the resuming fs. >> >> Thank you very much, >> Martin >> > Uh. Does ZFS use the buffer cache at all ? > > Could it be that all you need is a call to vm_object_page_remove(vp->v_object, > 0, 0) ? Yes, that seems to be enough. Updated patch attached. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000207000804020509030203 Content-Type: text/plain; name="zfs_znode.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_znode.c.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1343,6 +1343,12 @@ zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; + + if (ZTOV(zp)->v_object != NULL) { + VM_OBJECT_LOCK(ZTOV(zp)->v_object); + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); + } if (zp->z_size != size && ZTOV(zp) != NULL) vnode_pager_setsize(ZTOV(zp), zp->z_size); --------------000207000804020509030203-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:49:18 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 893981065673; Wed, 24 Aug 2011 17:49:18 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 3D58E8FC08; Wed, 24 Aug 2011 17:49:18 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id B85FDF7C; Wed, 24 Aug 2011 19:49:15 +0200 (CEST) Date: Wed, 24 Aug 2011 19:48:58 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110824174858.GB1688@garage.freebsd.pl> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wq9mPyueHGvFACwf" Content-Disposition: inline In-Reply-To: <4E5536F6.9020200@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:49:18 -0000 --wq9mPyueHGvFACwf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 07:37:58PM +0200, Martin Matuska wrote: > On 24. 8. 2011 15:45, Kostik Belousov wrote: > > Uh. Does ZFS use the buffer cache at all ? > > > > Could it be that all you need is a call to vm_object_page_remove(vp->v_= object, > > 0, 0) ? > Yes, that seems to be enough. > Updated patch attached. > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1343,6 +1343,12 @@ > =20 > zp->z_unlinked =3D (zp->z_links =3D=3D 0); > zp->z_blksz =3D doi.doi_data_block_size; > + > + if (ZTOV(zp)->v_object !=3D NULL) { > + VM_OBJECT_LOCK(ZTOV(zp)->v_object); > + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); > + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); > + } > if (zp->z_size !=3D size && ZTOV(zp) !=3D NULL) > vnode_pager_setsize(ZTOV(zp), zp->z_size); As you can see after the code you added we check for ZTOV(zp) not being NULL. If it is possible it can be NULL, you need to check it also before referencing it. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --wq9mPyueHGvFACwf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk5VOYkACgkQForvXbEpPzTiSACghZCeeHEi/kXR3t/W7Ukl++4J 0rQAoJsJqga+w6Qi9BZrb/R1JxlNzOnQ =316C -----END PGP SIGNATURE----- --wq9mPyueHGvFACwf-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:51:31 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A591065673; Wed, 24 Aug 2011 17:51:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A26D68FC18; Wed, 24 Aug 2011 17:51:30 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p7OHpPn6067339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p7OHpPDp075876; Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p7OHpPYX075875; Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 24 Aug 2011 20:51:25 +0300 From: Kostik Belousov To: Martin Matuska Message-ID: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XzfsYBK/mU6LvfOy" Content-Disposition: inline In-Reply-To: <4E5536F6.9020200@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:51:32 -0000 --XzfsYBK/mU6LvfOy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 07:37:58PM +0200, Martin Matuska wrote: > On 24. 8. 2011 15:45, Kostik Belousov wrote: > > On Wed, Aug 24, 2011 at 03:14:47PM +0200, Martin Matuska wrote: > >> Hi Kostik, > >> > >> could you please review the attached patch? > >> > >> It fixes kern/160035 and kern/156933. > >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D160035 > >> > >> The function zfs_rezget() is called as part of zfs_resume_fs(). > >> Suspending and resuming a ZFS filesystem is used in three cases: > >> 1.) zfs rollback > >> 2.) snapshot receive on a mounted fs (zfs recv -F) > >> 3.) zfs upgrade > >> > >> In case 1.) and 2.) the filesystem usually changes (changed/deleted/ne= w files/directories). > >> > >> To make sure we don't have mmaped wrong data we have to invalidate > >> buffers for all vnodes of the resuming fs. > >> > >> Thank you very much, > >> Martin=20 > >> > > Uh. Does ZFS use the buffer cache at all ? > > > > Could it be that all you need is a call to vm_object_page_remove(vp->v_= object, > > 0, 0) ? > Yes, that seems to be enough. > Updated patch attached. >=20 > --=20 > Martin Matuska > FreeBSD committer > http://blog.vx.sk >=20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1343,6 +1343,12 @@ > =20 > zp->z_unlinked =3D (zp->z_links =3D=3D 0); > zp->z_blksz =3D doi.doi_data_block_size; > + > + if (ZTOV(zp)->v_object !=3D NULL) { > + VM_OBJECT_LOCK(ZTOV(zp)->v_object); > + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); > + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); > + } > if (zp->z_size !=3D size && ZTOV(zp) !=3D NULL) > vnode_pager_setsize(ZTOV(zp), zp->z_size); > =20 I recommend you to create a local variable of the type vm_object_t and use it, instead of dereferencing the long chain of pointers. Also, consider moving ffs_pages_remove out from ffs_inode() into vfs_vnops.= c, calling it e.g. vn_pages_remove(), and use it instead. --XzfsYBK/mU6LvfOy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk5VOh0ACgkQC3+MBN1Mb4jz4gCfdJNbpYqOeMk6Fdj6OlvnD0bK hocAoJUD2+1D9YXLv/kkiMClzZht1CJs =kF+I -----END PGP SIGNATURE----- --XzfsYBK/mU6LvfOy-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 19:14:15 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E5B1106567F; Wed, 24 Aug 2011 19:14:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 4980B8FC20; Wed, 24 Aug 2011 19:14:15 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 490B019625A; Wed, 24 Aug 2011 21:14:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 2X3CHf7wgtIP; Wed, 24 Aug 2011 21:14:11 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id D1ACD19624A; Wed, 24 Aug 2011 21:14:10 +0200 (CEST) Message-ID: <4E554D82.805@FreeBSD.org> Date: Wed, 24 Aug 2011 21:14:10 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------010201060300020505080503" Cc: zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 19:14:15 -0000 This is a multi-part message in MIME format. --------------010201060300020505080503 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 24. 8. 2011 19:51, Kostik Belousov wrote: > I recommend you to create a local variable of the type vm_object_t > and use it, instead of dereferencing the long chain of pointers. > > Also, consider moving ffs_pages_remove out from ffs_inode() into vfs_vnops.c, > calling it e.g. vn_pages_remove(), and use it instead. Thank you very much for your comments. A new version with function moved to zfs_vnops.c and considering pjd's remark is attached. The function is thematically placed after static update_pages() and before static mappedread_sf(). -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------010201060300020505080503 Content-Type: text/plain; name="zfs_remove_pages.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_remove_pages.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1253,6 +1253,8 @@ again: return (err); } +extern void zfs_remove_pages(vnode_t *vp); + int zfs_rezget(znode_t *zp) { @@ -1343,8 +1345,11 @@ zfs_rezget(znode_t *zp) zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; - if (zp->z_size != size && ZTOV(zp) != NULL) - vnode_pager_setsize(ZTOV(zp), zp->z_size); + if (ZTOV(zp) != NULL) { + zfs_remove_pages(ZTOV(zp)); + if (zp->z_size != size) + vnode_pager_setsize(ZTOV(zp), zp->z_size); + } ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -420,6 +420,21 @@ update_pages(vnode_t *vp, int64_t start, int len, } /* + * Remove all mapped pages from a vnode + */ +void +zfs_remove_pages(vnode_t *vp) +{ + vm_object_t obj = vp->v_object; + + if (obj != NULL) { + VM_OBJECT_LOCK(obj); + vm_object_page_remove(obj, 0, 0, 0); + VM_OBJECT_UNLOCK(obj); + } +} + +/* * Read with UIO_NOCOPY flag means that sendfile(2) requests * ZFS to populate a range of page cache pages with data. * --------------010201060300020505080503-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 21:07:50 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF9681065672; Wed, 24 Aug 2011 21:07:50 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA1B8FC18; Wed, 24 Aug 2011 21:07:50 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id EC2A6197625; Wed, 24 Aug 2011 23:07:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id W2GpVcgUpJ8U; Wed, 24 Aug 2011 23:07:46 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 5313819761A; Wed, 24 Aug 2011 23:07:46 +0200 (CEST) Message-ID: <4E556822.1030004@FreeBSD.org> Date: Wed, 24 Aug 2011 23:07:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------070308010107030404050803" Cc: zfs-devel@freebsd.org Subject: Patch review reguest: vn_pages_remove() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 21:07:50 -0000 This is a multi-part message in MIME format. --------------070308010107030404050803 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit A generalization of vn_pages_remove() is attached + ZTOV(vp) suggested change by pjd. Is this we have been talking about? Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------070308010107030404050803 Content-Type: text/plain; name="vn_pages_remove.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="vn_pages_remove.patch" Index: sys/ufs/ffs/ffs_softdep.c =================================================================== --- sys/ufs/ffs/ffs_softdep.c (revision 225140) +++ sys/ufs/ffs/ffs_softdep.c (working copy) @@ -6541,7 +6541,7 @@ trunc_pages(ip, length, extblocks, flags) fs = ip->i_fs; extend = OFF_TO_IDX(lblktosize(fs, -extblocks)); if ((flags & IO_EXT) != 0) - ffs_pages_remove(vp, extend, 0); + vn_pages_remove(vp, extend, 0); if ((flags & IO_NORMAL) == 0) return; BO_LOCK(&vp->v_bufobj); @@ -6567,7 +6567,7 @@ trunc_pages(ip, length, extblocks, flags) end = OFF_TO_IDX(lblktosize(fs, lbn)); } else end = extend; - ffs_pages_remove(vp, OFF_TO_IDX(OFF_MAX), end); + vn_pages_remove(vp, OFF_TO_IDX(OFF_MAX), end); } /* Index: sys/ufs/ffs/ffs_extern.h =================================================================== --- sys/ufs/ffs/ffs_extern.h (revision 225140) +++ sys/ufs/ffs/ffs_extern.h (working copy) @@ -79,7 +79,6 @@ int ffs_isfreeblock(struct fs *, u_char *, ufs1_da void ffs_load_inode(struct buf *, struct inode *, struct fs *, ino_t); int ffs_mountroot(void); void ffs_oldfscompat_write(struct fs *, struct ufsmount *); -void ffs_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end); int ffs_reallocblks(struct vop_reallocblks_args *); int ffs_realloccg(struct inode *, ufs2_daddr_t, ufs2_daddr_t, ufs2_daddr_t, int, int, int, struct ucred *, struct buf **); Index: sys/ufs/ffs/ffs_inode.c =================================================================== --- sys/ufs/ffs/ffs_inode.c (revision 225140) +++ sys/ufs/ffs/ffs_inode.c (working copy) @@ -120,18 +120,6 @@ ffs_update(vp, waitfor) } } -void -ffs_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end) -{ - vm_object_t object; - - if ((object = vp->v_object) == NULL) - return; - VM_OBJECT_LOCK(object); - vm_object_page_remove(object, start, end, 0); - VM_OBJECT_UNLOCK(object); -} - #define SINGLE 0 /* index of single indirect block */ #define DOUBLE 1 /* index of double indirect block */ #define TRIPLE 2 /* index of triple indirect block */ @@ -219,7 +207,7 @@ ffs_truncate(vp, length, flags, cred, td) (void) chkdq(ip, -extblocks, NOCRED, 0); #endif vinvalbuf(vp, V_ALT, 0, 0); - ffs_pages_remove(vp, + vn_pages_remove(vp, OFF_TO_IDX(lblktosize(fs, -extblocks)), 0); osize = ip->i_din2->di_extsize; ip->i_din2->di_blocks -= extblocks; Index: sys/kern/vfs_vnops.c =================================================================== --- sys/kern/vfs_vnops.c (revision 225152) +++ sys/kern/vfs_vnops.c (working copy) @@ -64,6 +64,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include + static fo_rdwr_t vn_read; static fo_rdwr_t vn_write; static fo_truncate_t vn_truncate; @@ -1398,3 +1401,15 @@ vn_chown(struct file *fp, uid_t uid, gid_t gid, st VFS_UNLOCK_GIANT(vfslocked); return (error); } + +void +vn_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end) +{ + vm_object_t object; + + if ((object = vp->v_object) == NULL) + return; + VM_OBJECT_LOCK(object); + vm_object_page_remove(object, start, end, 0); + VM_OBJECT_UNLOCK(object); +} Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1259,6 +1259,7 @@ zfs_rezget(znode_t *zp) zfsvfs_t *zfsvfs = zp->z_zfsvfs; dmu_object_info_t doi; dmu_buf_t *db; + vnode_t *vp = ZTOV(zp); uint64_t obj_num = zp->z_id; uint64_t mode, size; sa_bulk_attr_t bulk[8]; @@ -1334,8 +1335,8 @@ zfs_rezget(znode_t *zp) * that for example regular file was replaced with directory * which has the same object number. */ - if (ZTOV(zp) != NULL && - ZTOV(zp)->v_type != IFTOVT((mode_t)zp->z_mode)) { + if (vp != NULL && + vp->v_type != IFTOVT((mode_t)zp->z_mode)) { zfs_znode_dmu_fini(zp); ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); return (EIO); @@ -1343,8 +1344,11 @@ zfs_rezget(znode_t *zp) zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; - if (zp->z_size != size && ZTOV(zp) != NULL) - vnode_pager_setsize(ZTOV(zp), zp->z_size); + if (vp != NULL) { + vn_pages_remove(vp, 0, 0); + if (zp->z_size != size) + vnode_pager_setsize(vp, zp->z_size); + } ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); Index: sys/sys/param.h =================================================================== --- sys/sys/param.h (revision 225140) +++ sys/sys/param.h (working copy) @@ -58,7 +58,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 900041 /* Master, propagated to newvers */ +#define __FreeBSD_version 900042 /* Master, propagated to newvers */ #ifdef _KERNEL #define P_OSREL_SIGSEGV 700004 Index: sys/sys/vnode.h =================================================================== --- sys/sys/vnode.h (revision 225140) +++ sys/sys/vnode.h (working copy) @@ -640,6 +640,7 @@ int _vn_lock(struct vnode *vp, int flags, char *fi int vn_open(struct nameidata *ndp, int *flagp, int cmode, struct file *fp); int vn_open_cred(struct nameidata *ndp, int *flagp, int cmode, u_int vn_open_flags, struct ucred *cred, struct file *fp); +void vn_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end); int vn_pollrecord(struct vnode *vp, struct thread *p, int events); int vn_rdwr(enum uio_rw rw, struct vnode *vp, void *base, int len, off_t offset, enum uio_seg segflg, int ioflg, --------------070308010107030404050803-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 21:26:47 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 512801065673; Wed, 24 Aug 2011 21:26:47 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 03F8A8FC0C; Wed, 24 Aug 2011 21:26:46 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id CBD87C3; Wed, 24 Aug 2011 23:26:44 +0200 (CEST) Date: Wed, 24 Aug 2011 23:26:27 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110824212626.GJ1688@garage.freebsd.pl> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> <4E556822.1030004@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BEa57a89OpeoUzGD" Content-Disposition: inline In-Reply-To: <4E556822.1030004@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: Patch review reguest: vn_pages_remove() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 21:26:47 -0000 --BEa57a89OpeoUzGD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 11:07:46PM +0200, Martin Matuska wrote: > A generalization of vn_pages_remove() is attached + ZTOV(vp) suggested > change by pjd. > Is this we have been talking about? [...] > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1259,6 +1259,7 @@ zfs_rezget(znode_t *zp) > zfsvfs_t *zfsvfs =3D zp->z_zfsvfs; > dmu_object_info_t doi; > dmu_buf_t *db; > + vnode_t *vp =3D ZTOV(zp); Could you move the assignment just before it is used? We put a hold on znode and I'd prefer not to obtain its vnode pointer before we do that. Just in case. > uint64_t obj_num =3D zp->z_id; > uint64_t mode, size; > sa_bulk_attr_t bulk[8]; > @@ -1334,8 +1335,8 @@ zfs_rezget(znode_t *zp) > * that for example regular file was replaced with directory > * which has the same object number. > */ I'll move it here: vp =3D ZTOV(zp); > - if (ZTOV(zp) !=3D NULL && > - ZTOV(zp)->v_type !=3D IFTOVT((mode_t)zp->z_mode)) { > + if (vp !=3D NULL && > + vp->v_type !=3D IFTOVT((mode_t)zp->z_mode)) { Those two lines should now fit into one. Other than that looks good. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --BEa57a89OpeoUzGD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk5VbIIACgkQForvXbEpPzSd3wCeOn8hrtJqtp4Gq2HYhplbSYCC kjUAoLPoBpdk+99tguJ+N+iihBilecZ2 =VVUk -----END PGP SIGNATURE----- --BEa57a89OpeoUzGD-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 30 08:56:11 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75C9A106564A for ; Tue, 30 Aug 2011 08:56:11 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 10CF98FC13 for ; Tue, 30 Aug 2011 08:56:11 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 4085418D4C3 for ; Tue, 30 Aug 2011 10:56:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id jBccCGwa18Vp for ; Tue, 30 Aug 2011 10:56:04 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 9C1D518D4A3 for ; Tue, 30 Aug 2011 10:56:04 +0200 (CEST) Message-ID: <4E5CA59F.7010705@FreeBSD.org> Date: Tue, 30 Aug 2011 10:55:59 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: List of recently fixed Oracle bugs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Aug 2011 08:56:11 -0000 Oracle has upgraded ZFS in Solaris 10 to version 29 (still not including deduplication). Here is a list of fixed bugs that I was manage to get that have been not fixed in OpenSolaris or Illumos (e.g. disabled zfs vdev cache). Some of the bugs are not relevant to FreeBSD as we have already fixed the issues a different way. 6651136 zfs_link_destroy() should use reader vn_vfsrlock instead of writer vn_vfswlock http://wesunsolve.net/bugid/id/6651136 6998684 deadlock between zfs_inactive() and zfs_write()/as_fault() http://wesunsolve.net/bugid/id/6998684 6914204 zfs commands hang trying to get spa_errlog_lock owned by sync_thread stalled by ZIO error http://wesunsolve.net/bugid/id/6914204 6920295 possible deadlock between ZFS range lock and address space a_lock http://wesunsolve.net/bugid/id/6920295 6931697 ZFS should keep writing data to vdevs with an active hot spare http://wesunsolve.net/bugid/id/6931697 7019356 extra VN_RELE in zfs_setattr() if user has exceeded user quota and file has extended attributes http://wesunsolve.net/bugid/id/7019356 7019358 small race in zfs_sa_upgrade http://wesunsolve.net/bugid/id/7019358 7019370 zfs_aclset_common could cause creation of unnecessary SA layout http://wesunsolve.net/bugid/id/7019370 6700597 zfs send -R | zfs receive -d will fail if any of the file systems have non default mountpoints http://wesunsolve.net/bugid/id/6700597 6883722 want 'zfs recv -o prop=value' to set initial property values of received dataset http://wesunsolve.net/bugid/id/6883722 6886341 want a 'zfs send' option to ignore local property settings when restoring from a backup http://wesunsolve.net/bugid/id/6886341 6900937 ZFS hang waiting for exclusive access to the dataset http://wesunsolve.net/bugid/id/6900937 6947432 zfs list could accept "fs" as synonym for "filesystem" http://wesunsolve.net/bugid/id/6947432 6961707 ZFS rpool panic at zfs: allocating allocated segment http://wesunsolve.net/bugid/id/6961707 6986538 "zfs holds -r" doesn't work http://wesunsolve.net/bugid/id/6986538 6992148 zfs diff shouldn't insist on finding a .zfs/shares http://wesunsolve.net/bugid/id/6992148 7009010 unable to delete ZFS file on NFSv4 share once refquota has been reached http://wesunsolve.net/bugid/id/7009010 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Sep 28 08:59:14 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BE2C106564A; Wed, 28 Sep 2011 08:59:14 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8A30A8FC12; Wed, 28 Sep 2011 08:59:10 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id ACEE21A5FFF; Wed, 28 Sep 2011 10:59:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id td0TImwpZpSF; Wed, 28 Sep 2011 10:59:07 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 6024C1A5FF6; Wed, 28 Sep 2011 10:59:07 +0200 (CEST) Message-ID: <4E82E1DA.1000009@FreeBSD.org> Date: Wed, 28 Sep 2011 10:59:06 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: bug-followup@FreeBSD.org X-Enigmail-Version: 1.3.2 Content-Type: multipart/mixed; boundary="------------020904070709010904030807" Cc: zfs-devel@FreeBSD.org Subject: [PATCH] Re: bin/160400: zfs(1): zfs rename dumps core X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2011 08:59:14 -0000 This is a multi-part message in MIME format. --------------020904070709010904030807 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The reproduction of this issue can be simplified and similiar case has already been reported on freebsd-fs@: http://lists.freebsd.org/pipermail/freebsd-fs/2010-September/009379.html http://lists.freebsd.org/pipermail/freebsd-fs/2011-August/012291.html It is important that the mountpoint of the dataset to be renamed is set to none or legacy, otherwise this assertion is not triggered. # zfs create -o mountpoint=none tank/a # zfs create tank/a/b # zfs rename tank/a tank/c Assertion failed: (!clp->cl_alldependents), file /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c, line 470. Abort (core dumped) As to my code examination, the assertion in libzfs_changelist.c: change_one() is inproper or invalid. The code path is called with clp->cl_sorted = B_FALSE and cl_alldependents = B_TRUE from changelist_gather() if mountpoint is legacy or none. This misbehaviour was introduced in this OpenSolaris commit: changeset: 10196:210962933dfd user: William Gorrell date: Wed Jul 29 08:49:33 2009 -0600 summary: 6612218 inherited zfs set mountpoint mounts children before parent https://github.com/illumos/illumos-gate/commit/3cc4a7920cf40de22a5c8c465a4676b2b7f620dd#usr/src/lib/libzfs/common/libzfs_changelist.c I am generally putting in question why that assertion was in this place at it concerns exclusively the case with mountpoint=none or mountpoint=legacy and it manages only the order of the datasets being processed. I suggest removing the assertion as we can safely try to unmount the dependent filesystems. Please review the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020904070709010904030807 Content-Type: text/plain; name="libzfs_changelist.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="libzfs_changelist.c.patch" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6 ZnNfY2hhbmdlbGlzdC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGNkZGwvY29udHJpYi9vcGVuc29s YXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6ZnNfY2hhbmdlbGlzdC5jCShyZXZpc2lvbiAy MjU2ODkpCisrKyBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvbGliL2xpYnpmcy9jb21tb24v bGliemZzX2NoYW5nZWxpc3QuYwkod29ya2luZyBjb3B5KQpAQCAtNDY3LDcgKzQ2Nyw2IEBA IGNoYW5nZV9vbmUoemZzX2hhbmRsZV90ICp6aHAsIHZvaWQgKmRhdGEpCiAJCQkgKiBUaGlz IGlzIG5lY2Vzc2FyeSB3aGVuIHRoZSBvcmlnaW5hbCBtb3VudHBvaW50CiAJCQkgKiBpcyBs ZWdhY3kgb3Igbm9uZS4KIAkJCSAqLwotCQkJQVNTRVJUKCFjbHAtPmNsX2FsbGRlcGVuZGVu dHMpOwogCQkJdmVyaWZ5KHV1X2xpc3RfaW5zZXJ0X2JlZm9yZShjbHAtPmNsX2xpc3QsCiAJ CQkgICAgdXVfbGlzdF9maXJzdChjbHAtPmNsX2xpc3QpLCBjbikgPT0gMCk7CiAJCX0K --------------020904070709010904030807-- From owner-zfs-devel@FreeBSD.ORG Wed Sep 28 10:42:47 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63FB91065673; Wed, 28 Sep 2011 10:42:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 135A48FC1E; Wed, 28 Sep 2011 10:42:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 6A3D9150617; Wed, 28 Sep 2011 12:42:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id gl27T9Qr6xW1; Wed, 28 Sep 2011 12:42:44 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 23CFD150608; Wed, 28 Sep 2011 12:42:44 +0200 (CEST) Message-ID: <4E82FA22.5020406@FreeBSD.org> Date: Wed, 28 Sep 2011 12:42:42 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.3.2 Content-Type: multipart/mixed; boundary="------------020405000101050706040408" Cc: Subject: [PATCH] Illumos changeset #13469 (dmu_tx.c) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2011 10:42:47 -0000 This is a multi-part message in MIME format. --------------020405000101050706040408 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Illumos has now provided a fix for the Issue #1475 regarding invalid spill blkptr: https://www.illumos.org/issues/1475 changeset: 13469:b8e89e5c4167 tag: tip user: Albert Lee date: Sun Sep 25 03:07:35 2011 -0400 summary: 1475 zfs spill block hold can access invalid spill blkptr https://github.com/illumos/illumos-gate/commit/9dccfd2a04cd1645e2616b7307b3a88041aba991 A partial fix by pjd was already in our code: http://p4db.freebsd.org/changeView.cgi?CH=185940 http://p4db.freebsd.org/changeView.cgi?CH=185942 I suggest importing the full fix to match the Illumos version. Please review and/or comment the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020405000101050706040408 Content-Type: text/plain; name="dmu_tx.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmu_tx.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZG11X3R4LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy91dHMvY29tbW9uL2ZzL3pmcy9kbXVfdHguYwkocmV2aXNpb24gMjI1Njg5KQorKysgc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9kbXVfdHguYwko d29ya2luZyBjb3B5KQpAQCAtMjEsNiArMjEsOSBAQAogLyoKICAqIENvcHlyaWdodCAoYykg MjAwNSwgMjAxMCwgT3JhY2xlIGFuZC9vciBpdHMgYWZmaWxpYXRlcy4gQWxsIHJpZ2h0cyBy ZXNlcnZlZC4KICAqLworLyoKKyAqIENvcHlyaWdodCAyMDExIE5leGVudGEgU3lzdGVtcywg SW5jLiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KKyAqLwogCiAjaW5jbHVkZSA8c3lzL2RtdS5o PgogI2luY2x1ZGUgPHN5cy9kbXVfaW1wbC5oPgpAQCAtNjc2LDYgKzY3OSw4IEBACiAJQVNT RVJUM1AoZG11X290W2RuLT5kbl90eXBlXS5vdF9ieXRlc3dhcCwgPT0sIHphcF9ieXRlc3dh cCk7CiAKIAlpZiAoZG4tPmRuX21heGJsa2lkID09IDAgJiYgIWFkZCkgeworCQlibGtwdHJf dCAqYnA7CisKIAkJLyoKIAkJICogSWYgdGhlcmUgaXMgb25seSBvbmUgYmxvY2sgIChpLmUu IHRoaXMgaXMgYSBtaWNyby16YXApCiAJCSAqIGFuZCB3ZSBhcmUgbm90IGFkZGluZyBhbnl0 aGluZywgdGhlIGFjY291bnRpbmcgaXMgc2ltcGxlLgpAQCAtNjkwLDE0ICs2OTUsMTMgQEAK IAkJICogVXNlIG1heCBibG9jayBzaXplIGhlcmUsIHNpbmNlIHdlIGRvbid0IGtub3cgaG93 IG11Y2gKIAkJICogdGhlIHNpemUgd2lsbCBjaGFuZ2UgYmV0d2VlbiBub3cgYW5kIHRoZSBk YnVmIGRpcnR5IGNhbGwuCiAJCSAqLworCQlicCA9ICZkbi0+ZG5fcGh5cy0+ZG5fYmxrcHRy WzBdOwogCQlpZiAoZHNsX2RhdGFzZXRfYmxvY2tfZnJlZWFibGUoZG4tPmRuX29ianNldC0+ b3NfZHNsX2RhdGFzZXQsCi0JCSAgICAmZG4tPmRuX3BoeXMtPmRuX2Jsa3B0clswXSwKLQkJ ICAgIGRuLT5kbl9waHlzLT5kbl9ibGtwdHJbMF0uYmxrX2JpcnRoKSkgeworCQkgICAgYnAs IGJwLT5ibGtfYmlydGgpKQogCQkJdHhoLT50eGhfc3BhY2VfdG9vdmVyd3JpdGUgKz0gU1BB X01BWEJMT0NLU0laRTsKLQkJfSBlbHNlIHsKKwkJZWxzZQogCQkJdHhoLT50eGhfc3BhY2Vf dG93cml0ZSArPSBTUEFfTUFYQkxPQ0tTSVpFOwotCQl9Ci0JCWlmIChkbi0+ZG5fcGh5cy0+ ZG5fYmxrcHRyWzBdLmJsa19iaXJ0aCkKKwkJaWYgKCFCUF9JU19IT0xFKGJwKSkKIAkJCXR4 aC0+dHhoX3NwYWNlX3RvdW5yZWYgKz0gU1BBX01BWEJMT0NLU0laRTsKIAkJcmV0dXJuOwog CX0KQEAgLTEyNzMsNyArMTI3Nyw2IEBACiB7CiAJZG5vZGVfdCAqZG47CiAJZG11X3R4X2hv bGRfdCAqdHhoOwotCWJsa3B0cl90ICpicDsKIAogCXR4aCA9IGRtdV90eF9ob2xkX29iamVj dF9pbXBsKHR4LCB0eC0+dHhfb2Jqc2V0LCBvYmplY3QsCiAJICAgIFRIVF9TUElMTCwgMCwg MCk7CkBAIC0xMjg2LDE1ICsxMjg5LDE2IEBACiAJLyogSWYgYmxrcHRyIGRvZXNuJ3QgZXhp c3QgdGhlbiBhZGQgc3BhY2UgdG8gdG93cml0ZSAqLwogCWlmICghKGRuLT5kbl9waHlzLT5k bl9mbGFncyAmIEROT0RFX0ZMQUdfU1BJTExfQkxLUFRSKSkgewogCQl0eGgtPnR4aF9zcGFj ZV90b3dyaXRlICs9IFNQQV9NQVhCTE9DS1NJWkU7Ci0JCXR4aC0+dHhoX3NwYWNlX3RvdW5y ZWYgPSAwOwogCX0gZWxzZSB7CisJCWJsa3B0cl90ICpicDsKKwogCQlicCA9ICZkbi0+ZG5f cGh5cy0+ZG5fc3BpbGw7CiAJCWlmIChkc2xfZGF0YXNldF9ibG9ja19mcmVlYWJsZShkbi0+ ZG5fb2Jqc2V0LT5vc19kc2xfZGF0YXNldCwKIAkJICAgIGJwLCBicC0+YmxrX2JpcnRoKSkK IAkJCXR4aC0+dHhoX3NwYWNlX3Rvb3ZlcndyaXRlICs9IFNQQV9NQVhCTE9DS1NJWkU7CiAJ CWVsc2UKIAkJCXR4aC0+dHhoX3NwYWNlX3Rvd3JpdGUgKz0gU1BBX01BWEJMT0NLU0laRTsK LQkJaWYgKGJwLT5ibGtfYmlydGgpCisJCWlmICghQlBfSVNfSE9MRShicCkpCiAJCQl0eGgt PnR4aF9zcGFjZV90b3VucmVmICs9IFNQQV9NQVhCTE9DS1NJWkU7CiAJfQogfQo= --------------020405000101050706040408-- From owner-zfs-devel@FreeBSD.ORG Thu Sep 29 12:48:04 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 577A11065670 for ; Thu, 29 Sep 2011 12:48:04 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 08F268FC17 for ; Thu, 29 Sep 2011 12:48:03 +0000 (UTC) Received: by vcbf13 with SMTP id f13so558918vcb.13 for ; Thu, 29 Sep 2011 05:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=Ckv/P5PBv7EhZzDy26txSA/Ha/47RhaVVy7MjgipsAw=; b=uqwc4HvCA5ebc2lAP4qAlcmj2NNAqbwCF+y09UEV6sxQKoY7FnYp/FnuoFxT2i1Fdt WFuwEjeJExqg+vxQmSySw0Hihd9kBhr1nlzSVmhbWBW6oJEg/MqPRquljnto5ZekMgEQ ji8g3zYk0UcP/I+zBcfR6yT/iFXojVp2fcuM4= MIME-Version: 1.0 Received: by 10.52.35.180 with SMTP id i20mr6033992vdj.198.1317299023421; Thu, 29 Sep 2011 05:23:43 -0700 (PDT) Received: by 10.52.182.1 with HTTP; Thu, 29 Sep 2011 05:23:43 -0700 (PDT) Date: Thu, 29 Sep 2011 17:53:43 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2011 12:48:04 -0000 hi We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a race condition between a thread loading spa and another trying to access it. The two thread stacks trace these paths 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open ..... -> vdev_geom_open ......-> biowait 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... ->dsl_dir_open_spa -> _sx_slock the two threads should be serialized by spa_namespace_lock, but apparently the first thread, which is initializing spa, drops the lock in vdev_geom_open(). This lets the second thread come in and see and uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second thread tries to take lock in dsl_dir_open_spa() dp = spa_get_dsl(spa); rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is NULL Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What serializes the spa operations ? Some searching around the net showed up this thread, which has very similar problem http://freebsd.1045724.n5.nabble.com/ZFS-RAID-Z-panic-on-vdev-failure-subsequent-panics-and-hangs-td4032243.html Note that one of the devices was in UNAVAIL state because we had removed it on purpose Here are the stacktraces of the two threads -------------------------- thread 1 -------------------------------------------------------------- Thread 537 (Thread 100398): #0 sched_switch (td=0xffffff0002d118c0, newtd=0xffffff00023c38c0, flags=Variable "flags" is not available. ) at ../../../kern/sched_ule.c:1858 #1 0xffffffff805cb166 in mi_switch (flags=260, newtd=0x0) at ../../../kern/kern_synch.c:449 #2 0xffffffff805fe7b2 in sleepq_timedwait (wchan=0xffffff00027bb0e8, pri=76) at ../../../kern/subr_sleepqueue.c:644 #3 0xffffffff805cb6c1 in _sleep (ident=0xffffff00027bb0e8, lock=0xffffff800021a2b0, priority=Variable "priority" is not av ailable. ) at ../../../kern/kern_synch.c:230 #4 0xffffffff80638680 in biowait (bp=0xffffff00027bb0e8, wchan=0xffffffff80f09406 "vdev_geom_io") at ../../../kern/vfs_bio .c:3189 #5 0xffffffff80ea77d3 in vdev_geom_open_by_path (vd=0x1e07c0000, check_guid=262144) at /usr/src/sys/modules/zfs/../../cddl /contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:388 #6 0xffffffff80ea81f2 in vdev_geom_open (vd=0xffffff00027bce00, psize=0xffffff804d00a670, ashift=0xffffff804d00a668) at /u sr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:362 #7 0xffffffff80e5e877 in vdev_open (vd=0xffffff0002d86800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/ common/fs/zfs/vdev.c:1248 #8 0xffffffff80e5f411 in vdev_compact_children (pvd=0xffffff0002d86800) at /usr/src/sys/modules/zfs/../../cddl/contrib/ope nsolaris/uts/common/fs/zfs/vdev.c:269 #9 0xffffffff80e681cc in zap_idx_to_blk (zap=0xffffff0002747000, idx=18446743525245626136, valp=0xffffff804d00a720) at /us r/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:297 #10 0xffffffff80e5e877 in vdev_open (vd=0xffffffff80e681cc) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/ common/fs/zfs/vdev.c:1248 #11 0xffffffff80e4fef1 in spa_load (spa=0xffffff0002747000, state=SPA_LOAD_OPEN, type=SPA_IMPORT_EXISTING, mosconfig=411852 80) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1922 #12 0xffffffff80e513c2 in spa_open_common (pool=0x1
, spapp=0x0, tag=0x1, nvpolicy=0xffffff00027 47000, config=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:2376 #13 0xffffffff80e516ba in spa_open_common (pool=0xffffff8000b0b000 "sp3", spapp=0xffffff804d00a920, tag=0xffffffff80ef7800, nvpolicy=Variable "nvpolicy" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:2439 #14 0xffffffff80e8e459 in zfs_unmount_snap (name=0xffffff8000b0b000 "sp3", arg=Variable "arg" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:3112 #15 0xffffffff80e8e6b3 in zfs_ioc_snapshot (zc=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/f s/zfs/zfs_ioctl.c:2397 #16 0xffffffff8054cedb in devfs_ioctl_f (fp=0xffffff0023b20c30, com=3583531538, data=Variable "data" is not available. ) at ../../../fs/devfs/devfs_vnops.c:678 #17 0xffffffff806043b2 in kern_ioctl (td=Variable "td" is not available. ) at file.h:262 #18 0xffffffff806045ed in ioctl (td=0xffffff0002d118c0, uap=0xffffff804d00abb0) at ../../../kern/sys_generic.c:679 #19 0xffffffff80600dc5 in syscallenter (td=0xffffff0002d118c0, sa=0xffffff804d00aba0) at ../../../kern/subr_trap.c:315 #20 0xffffffff808acb7b in syscall (frame=0xffffff804d00ac40) at ../../../amd64/amd64/trap.c:888 #21 0xffffffff808953c2 in Xfast_syscall () at ../../../amd64/amd64/exception.S:377 #22 0x0000000801104f8c in ?? () Previous frame inner to this frame (corrupt stack?) ------------------------------------------------------------------------------------------------------------------- ----------------------------------- thread 2 ------------------------ panic thread ---------------------------- Thread 548 (Thread 100432): #0 doadump () at pcpu.h:224 #1 0xffffffff805c28ae in boot (howto=260) at ../../../kern/kern_shutdown.c:419 #2 0xffffffff805c2ce1 in panic (fmt=Variable "fmt" is not available. ) at ../../../kern/kern_shutdown.c:592 #3 0xffffffff808ac720 in trap_fatal (frame=0xc, eva=Variable "eva" is not available. ) at ../../../amd64/amd64/trap.c:783 #4 0xffffffff808acaff in trap_pfault (frame=0xffffff804d0b4600, usermode=0) at ../../../amd64/amd64/trap.c:699 #5 0xffffffff808acfdf in trap (frame=0xffffff804d0b4600) at ../../../amd64/amd64/trap.c:449 #6 0xffffffff808950e4 in calltrap () at ../../../amd64/amd64/exception.S:224 #7 0xffffffff805cae61 in _sx_slock (sx=0x368, opts=0, file=0xffffffff80ef9ee0 "/home/chs/freebsd/8.2/src/sys/modules/zfs/. ./../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c", line=334) at ../../../kern/kern_sx.c:239 #8 0xffffffff80e30a7b in dsl_dir_open_spa (spa=0xffffff0002747000, name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c:396 #9 0xffffffff80e3498b in dsl_dataset_hold (name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:642 #10 0xffffffff80e26973 in dmu_objset_snapshot (fsname=0xffffff0002d1d460 "\200}\xb6\200\xff\xff\xff\xff", snapname=0x0, tag =0xffffffff80ef9ee0 "/home/chs/freebsd/8.2/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c", props=0x14e, recursive=47305824, temporary=1, cleanup_fd=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts /common/fs/zfs/dmu_objset.c:946 #11 0xffffffff80e8d3ab in zfs_ioc_vdev_set_state (zc=0xffffff804d0b4918) at /usr/src/sys/modules/zfs/../../cddl/contrib/ope nsolaris/uts/common/fs/zfs/zfs_ioctl.c:1629 #12 0xffffffff80e8e576 in zfs_secpolicy_write_perms (name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:360 #13 0xffffffff8054cedb in devfs_ioctl_f (fp=0xffffffff80f0e820, com=18446742974794065712, data=Variable "data" is not avail able. ) at ../../../fs/devfs/devfs_vnops.c:678 #14 0xffffffff806043b2 in kern_ioctl (td=Variable "td" is not available. ) at file.h:262 #15 0xffffffff806045ed in ioctl (td=0xffffff0002d1d460, uap=0xffffff804d0b4bb0) at ../../../kern/sys_generic.c:679 #16 0xffffffff80600dc5 in syscallenter (td=0xffffff0002d1d460, sa=0xffffff804d0b4ba0) at ../../../kern/subr_trap.c:315 #17 0xffffffff808acb7b in syscall (frame=0xffffff804d0b4c40) at ../../../amd64/amd64/trap.c:888 #18 0xffffffff808953c2 in Xfast_syscall () at ../../../amd64/amd64/exception.S:377 #19 0x0000000801104f8c in ?? () ---------------------------------------------------------------------------------------------------- here is the spa structure --------------------------------- (kgdb) print {spa_t}0xffffff0002747000 $1 = {spa_name = "sp3", '\0' , spa_avl = {avl_child = {0x0, 0x0}, avl_pcb = 18446742974239113477}, spa_config = 0xffffff0002b73820, spa_config_syncing = 0x0, spa_config_splitting = 0x0, spa_load_info = 0xffffff0002715c60, spa_config_txg = 10, spa_sync_pass = 0, spa_state = POOL_STATE_ACTIVE, spa_inject_ref = 0, spa_sync_on = 0 '\0', spa_load_state = SPA_LOAD_OPEN, spa_import_flags = 0, spa_zio_taskq = {{ 0xffffff0023c09790, 0x0, 0xffffff0023c09330, 0x0}, {0xffffff0023c09800, 0x0, 0xffffff00239b1050, 0x0}, { 0xffffff0023c093b0, 0xffffff0023c091e0, 0xffffff0023a1f900, 0xffffff0023c09260}, {0xffffff0023a1f570, 0x0, 0xffffff0023c09620, 0x0}, {0xffffff0002b9a830, 0x0, 0xffffff0023a1f530, 0x0}, {0xffffff0023a1f7d0, 0x0, 0xffffff0023a1f960, 0x0}}, spa_dsl_pool = 0x0, spa_normal_class = 0xffffff0002feca80, spa_log_class = 0xffffff0002bf8b40, spa_first_txg = 0, spa_final_txg = 18446744073709551615, spa_freeze_txg = 18446744073709551615, spa_load_max_txg = 18446744073709551615, spa_claim_max_txg = 0, spa_loaded_ts = { tv_sec = 1317106605, tv_nsec = 601195533}, spa_meta_objset = 0x0, spa_vdev_txg_list = {tl_lock = {lock_object = { lo_name = 0xffffffff80f072bb "tl->tl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, tl_offset = 1040, tl_head = {0x0, 0x0, 0x0, 0x0}}, spa_root_vdev = 0xffffff0002d89800, spa_load_guid = 4958651374771581097, spa_config_dirty_list = {list_size = 1800, list_offset = 1096, list_head = { list_next = 0xffffff00027472e0, list_prev = 0xffffff00027472e0}}, spa_state_dirty_list = {list_size = 1800, list_offset = 1112, list_head = {list_next = 0xffffff0002747300, list_prev = 0xffffff0002747300}}, spa_spares = { sav_object = 0, sav_config = 0x0, sav_vdevs = 0x0, sav_count = 0, sav_sync = 0, sav_pending = 0x0, sav_npending = 0}, spa_l2cache = {sav_object = 0, sav_config = 0x0, sav_vdevs = 0x0, sav_count = 0, sav_sync = 0, sav_pending = 0x0, sav_npending = 0}, spa_config_object = 0, spa_config_generation = 0, spa_syncing_txg = 0, spa_deferred_bpobj = { bpo_lock = {lock_object = {lo_name = 0x0, lo_flags = 0, lo_data = 0, lo_witness = 0x0}, sx_lock = 0}, bpo_os = 0x0, bpo_object = 0, bpo_epb = 0, bpo_havecomp = 0 '\0', bpo_havesubobj = 0 '\0', bpo_phys = 0x0, bpo_dbuf = 0x0, bpo_cached_dbuf = 0x0}, spa_free_bplist = {{bpl_lock = {lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747408, list_prev = 0xffffff0002747408}}}, {bpl_lock = { lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747448, list_prev = 0xffffff0002747448}}}, {bpl_lock = {lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747488, list_prev = 0xffffff0002747488}}}, {bpl_lock = { lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff00027474c8, list_prev = 0xffffff00027474c8}}}}, spa_ubsync = {ub_magic = 0, ub_version = 28, ub_txg = 0, ub_guid_sum = 0, ub_timestamp = 0, ub_rootbp = {blk_dva = {{dva_word = {0, 0}}, {dva_word = {0, 0}}, {dva_word = {0, 0}}}, blk_prop = 0, blk_pad = {0, 0}, blk_phys_birth = 0, blk_birth = 0, blk_fill = 0, blk_cksum = {zc_word = {0, 0, 0, 0}}}, ---Type to continue, or q to quit--- ub_software_version = 0}, spa_uberblock = {ub_magic = 0, ub_version = 0, ub_txg = 0, ub_guid_sum = 0, ub_timestamp = 0, ub_rootbp = {blk_dva = {{dva_word = {0, 0}}, {dva_word = {0, 0}}, {dva_word = {0, 0}}}, blk_prop = 0, blk_pad = {0, 0}, blk_phys_birth = 0, blk_birth = 0, blk_fill = 0, blk_cksum = {zc_word = {0, 0, 0, 0}}}, ub_software_version = 0}, spa_extreme_rewind = 0, spa_last_io = 0, spa_scrub_lock = {lock_object = { lo_name = 0xffffffff80f07134 "spa->spa_scrub_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_scrub_inflight = 0, spa_scrub_io_cv = {cv_description = 0xffffffff80f071a2 "spa->spa_scrub_io_cv)", cv_waiters = 0}, spa_scrub_active = 0 '\0', spa_scrub_type = 0 '\0', spa_scrub_finished = 0 '\0', spa_scrub_started = 0 '\0', spa_scrub_reopen = 0 '\0', spa_scan_pass_start = 0, spa_scan_pass_exam = 0, spa_async_lock = { lock_object = {lo_name = 0xffffffff80f070b2 "spa->spa_async_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_async_thread = 0x0, spa_async_suspended = 0, spa_async_cv = { cv_description = 0xffffffff80f07179 "spa->spa_async_cv)", cv_waiters = 0}, spa_async_tasks = 0, spa_root = 0x0, spa_ena = 0, spa_last_open_failed = 0, spa_last_ubsync_txg = 0, spa_last_ubsync_txg_ts = 0, spa_load_txg = 0, spa_load_txg_ts = 0, spa_load_meta_errors = 0, spa_load_data_errors = 0, spa_verify_min_txg = 0, spa_errlog_lock = { lock_object = {lo_name = 0xffffffff80f070de "spa->spa_errlog_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_errlog_last = 0, spa_errlog_scrub = 0, spa_errlist_lock = {lock_object = { lo_name = 0xffffffff80f070c7 "spa->spa_errlist_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_errlist_last = {avl_root = 0x0, avl_compar = 0xffffffff80e4d180 , avl_offset = 40, avl_numnodes = 0, avl_size = 64}, spa_errlist_scrub = {avl_root = 0x0, avl_compar = 0xffffffff80e4d180 , avl_offset = 40, avl_numnodes = 0, avl_size = 64}, spa_deflate = 0, spa_history = 0, spa_history_lock = {lock_object = { lo_name = 0xffffffff80f070f4 "spa->spa_history_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_pending_vdev = 0x0, spa_props_lock = {lock_object = { lo_name = 0xffffffff80f0711f "spa->spa_props_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_pool_props_object = 0, spa_bootfs = 0, spa_failmode = 0, spa_delegation = 0, spa_config_list = { list_size = 24, list_offset = 0, list_head = {list_next = 0xffffff0002735000, list_prev = 0xffffff0002735000}}, spa_async_zio_root = 0xffffff00028b7760, spa_suspend_zio_root = 0x0, spa_suspend_lock = {lock_object = { lo_name = 0xffffffff80f07149 "spa->spa_suspend_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_suspend_cv = {cv_description = 0xffffffff80f071ba "spa->spa_suspend_cv)", cv_waiters = 0}, spa_suspended = 0 '\0', spa_claiming = 0 '\0', spa_is_root = 0, spa_minref = 0, spa_mode = 1, spa_log_state = SPA_LOG_UNKNOWN, spa_autoexpand = 0, spa_ddt = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, spa_ddt_stat_object = 0, spa_dedup_ditto = 0, spa_dedup_checksum = 0, spa_dspace = 0, spa_vdev_top_lock = {lock_object = { lo_name = 0xffffffff80f07160 "spa->spa_vdev_top_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_proc_lock = {lock_object = {lo_name = 0xffffffff80f0710b "spa->spa_proc_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_proc_cv = {cv_description = 0xffffffff80f0718e "spa->spa_proc_cv)", ---Type to continue, or q to quit--- cv_waiters = 0}, spa_proc_state = SPA_PROC_NONE, spa_proc = 0xffffffff80b5db60, spa_did = 0, spa_autoreplace = 0, spa_vdev_locks = 0, spa_creation_version = 0, spa_prev_software_version = 0, spa_config_lock = {{scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}}, spa_refcount = { rc_count = 1}, spa_splitting_newspa = 0} ------------------------------------------------------------------ Here is the panic message ------------------------------------ Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x380 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff805cae61 stack pointer = 0x28:0xffffff804d0b46b0 frame pointer = 0x28:0xffffff804d0b4860 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1698 (initial thread) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff805f4dfe at kdb_backtrace+0x5e #1 0xffffffff805c2cf7 at panic+0x187 #2 0xffffffff808ac720 at trap_fatal+0x290 #3 0xffffffff808acaff at trap_pfault+0x28f #4 0xffffffff808acfdf at trap+0x3df #5 0xffffffff808950e4 at calltrap+0x8 #6 0xffffffff80e3498b at dsl_dataset_hold+0x3b #7 0xffffffff80e26973 at dmu_objset_hold+0x23 #8 0xffffffff80e8d3ab at zfs_ioc_objset_stats+0x2b #9 0xffffffff80e8e576 at zfsdev_ioctl+0xe6 #10 0xffffffff8054cedb at devfs_ioctl_f+0x7b #11 0xffffffff806043b2 at kern_ioctl+0x102 #12 0xffffffff806045ed at ioctl+0xfd #13 0xffffffff80600dc5 at syscallenter+0x1e5 #14 0xffffffff808acb7b at syscall+0x4b #15 0xffffffff808953c2 at Xfast_syscall+0xe2 Uptime: 1m12s Dumping 1023 MB (2 chunks) ------------------------------------------------------------------ regards Samir From owner-zfs-devel@FreeBSD.ORG Thu Sep 29 13:21:11 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42BA6106566B for ; Thu, 29 Sep 2011 13:21:11 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id EE1668FC17 for ; Thu, 29 Sep 2011 13:21:10 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id A809F80B; Thu, 29 Sep 2011 15:21:08 +0200 (CEST) Date: Thu, 29 Sep 2011 15:20:34 +0200 From: Pawel Jakub Dawidek To: Samir Desai Message-ID: <20110929132034.GD1660@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KlAEzMkarCnErv5Q" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2011 13:21:11 -0000 --KlAEzMkarCnErv5Q Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 29, 2011 at 05:53:43PM +0530, Samir Desai wrote: > hi >=20 > We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a race > condition between a thread loading spa and another trying to access it. T= he > two thread stacks trace these paths >=20 > 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open ...= =2E. > -> vdev_geom_open ......-> biowait > 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... > ->dsl_dir_open_spa -> _sx_slock >=20 > the two threads should be serialized by spa_namespace_lock, but apparently > the first thread, which is initializing spa, drops the lock in > vdev_geom_open(). This lets the second thread come in and see and > uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second thre= ad > tries to take lock in dsl_dir_open_spa() dp =3D spa_get_dsl(spa); > rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is N= ULL >=20 >=20 > Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What > serializes the spa operations ? Could you try this patch: http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch It eliminates dropping spa_namespace_lock in vdev_geom.c. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --KlAEzMkarCnErv5Q Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk6EcKIACgkQForvXbEpPzT1tgCePbbHaKQKab/Qg747yyblpiBL VkgAnA7z2Zi8rKrnM5V940GNO/22KrDm =RFsL -----END PGP SIGNATURE----- --KlAEzMkarCnErv5Q-- From owner-zfs-devel@FreeBSD.ORG Tue Oct 4 09:12:44 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 141F31065670; Tue, 4 Oct 2011 09:12:44 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 756F78FC12; Tue, 4 Oct 2011 09:12:43 +0000 (UTC) Received: by vws11 with SMTP id 11so242251vws.13 for ; Tue, 04 Oct 2011 02:12:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Kwgf5WwF4qHUM7KxwPcxy6XEqD+wyDjCi6szpR2WOSc=; b=Ro6rSB0uSfHYGh+DznDIN/7XNx9EqUYuItMPSz4ufFC1JHaefzrZvZW3IELzp42MdJ Wioju2TgVJG0wZSglx4BRA9Nubtpg8FiT8ODgjKssKT9CW1lOabjGOq5gmlJTAO6wXqY +BQlaqRfeLQ78onvpwIWtnpvSbAGXHcUuIY6I= MIME-Version: 1.0 Received: by 10.52.23.176 with SMTP id n16mr924699vdf.268.1317719561090; Tue, 04 Oct 2011 02:12:41 -0700 (PDT) Received: by 10.52.182.1 with HTTP; Tue, 4 Oct 2011 02:12:41 -0700 (PDT) In-Reply-To: <20110929132034.GD1660@garage.freebsd.pl> References: <20110929132034.GD1660@garage.freebsd.pl> Date: Tue, 4 Oct 2011 14:42:41 +0530 Message-ID: From: Samir Desai To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 09:12:44 -0000 hi Pawel thanks a lot for this patch. It seems to be holding up so far for us. In case we still see issues, I will get back to you. regards Samir On Thu, Sep 29, 2011 at 6:50 PM, Pawel Jakub Dawidek wrote: > On Thu, Sep 29, 2011 at 05:53:43PM +0530, Samir Desai wrote: > > hi > > > > We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a > race > > condition between a thread loading spa and another trying to access it. > The > > two thread stacks trace these paths > > > > 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open > ..... > > -> vdev_geom_open ......-> biowait > > 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... > > ->dsl_dir_open_spa -> _sx_slock > > > > the two threads should be serialized by spa_namespace_lock, but > apparently > > the first thread, which is initializing spa, drops the lock in > > vdev_geom_open(). This lets the second thread come in and see and > > uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second > thread > > tries to take lock in dsl_dir_open_spa() dp = spa_get_dsl(spa); > > rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is > NULL > > > > > > Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What > > serializes the spa operations ? > > Could you try this patch: > > http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch > > It eliminates dropping spa_namespace_lock in vdev_geom.c. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > FreeBSD committer http://www.FreeBSD.org > Am I Evil? Yes, I Am! http://yomoli.com > From owner-zfs-devel@FreeBSD.ORG Fri Oct 14 06:51:10 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9FD1106564A for ; Fri, 14 Oct 2011 06:51:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 218D58FC15 for ; Fri, 14 Oct 2011 06:51:09 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA16728; Fri, 14 Oct 2011 09:34:53 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1REbM5-000PXI-Gc; Fri, 14 Oct 2011 09:34:53 +0300 Message-ID: <4E97D80B.90204@FreeBSD.org> Date: Fri, 14 Oct 2011 09:34:51 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111002 Thunderbird/7.0.1 MIME-Version: 1.0 To: Dave Lin References: In-Reply-To: X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 14 Oct 2011 16:07:30 +0000 Cc: "freebsd-fs@freebsd.org" Subject: Re: cannot run ztest (zfs test suite) without seeing core dump on 8.2 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 06:51:10 -0000 on 14/10/2011 01:33 Dave Lin said the following: > Hello all, > > It seems that I cannot run ztest (zfs test suite) without seeing core dump on latest 8.2 release. Has anyone seen this before and is there way to resolve this issue? Thanks. > > Here's the output capture: > > dave-free82# ztest -V > 5 vdevs, 7 datasets, 23 threads, 300 seconds... > child died with signal 11 > > Here's the uname -a info: > > 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Please try the following changes: https://gitorious.org/~avg/freebsd/avgbsd/commit/5f0bbe6ff83f463f583358b76cfe2e179449091b https://gitorious.org/~avg/freebsd/avgbsd/commit/b430b23e6cd579c577f8ff1dae445a8ee2603ffa https://gitorious.org/~avg/freebsd/avgbsd/commit/96e8886589b0c6bb91e1019efb204e6aac87f4ef https://gitorious.org/~avg/freebsd/avgbsd/commit/5479bc325beb8fa85f50e50f3dc18069489a2119 -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Oct 14 22:21:44 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AA4F1065680 for ; Fri, 14 Oct 2011 22:21:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 309A08FC0A for ; Fri, 14 Oct 2011 22:21:41 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 7203B18CD9; Fri, 14 Oct 2011 15:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1318630900; bh=FTBfjbiZoXAdZmSNeRSkYoOFcVrevY7EEAhr/B4bxa4=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:Subject: Content-Type; b=ef1RyuKsJb6F/po5be6AGYDFd5sk6utkORxVLQRa0f0OcA9+XInF5v714vQSoiyoW 1/kkf0hrBsufJBorw4itMWMKSn1ByhM3Un5g3uYdGTOOaI6g2a4NWApWgO2slijCw9 qyQfrfFqSIq20Gxtn4MkHUeJWNoxu7sn9HlLo+VU= Message-ID: <4E98B5F3.7020103@delphij.net> Date: Fri, 14 Oct 2011 15:21:39 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------020607010000090202070304" Cc: Subject: [PATCH FOR REVIEW] Extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 22:21:44 -0000 This is a multi-part message in MIME format. --------------020607010000090202070304 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I think we hit a bug in sa_find_sizes: - hdrsize grows by 4 bytes each time; - On return, hdrsize is rounded up to nearest 8 byte boundary; - On the other hand, when testing for whether spill is needed, the code uses sum of (*total + hdrsize) and rounds that number up to nearest 8 byte boundary. This causes a problem when *total==306 and hdrsize==12. We have: (306 + 12) = 318 ROUNDUP(318, 8) = 320 On return, we have: total = 306 hdrsize = 16 And spill is unset. This would lead to a panic. Comments? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOmLXyAAoJEATO+BI/yjfBlikH+wQCt9BFDyFtH7CAaCui8W4b MnGo1Ocpbgzx0UBVcJkb5pglqscEihDMfQXLQHzbf+v0Bu3nfH2MudNsdr4/Ig2X XUD6cBD1cAN1x5jWzBIKrS7nigxICfd+cOsZ2J1vRj0Qu6TJ9t3kAETuZsU6l4Ee 3KfvoTu48E/2CzeCD4aAbqJH6bbIJ26dKRQ+KTp+FTLLk8goqDCQ7RgbEfO1QYZm vV+Ee+olvHDddC0Oxe08/X8seWaZg/N2ZZNT1Umo8VeiRvbKlUEs1hj8KN2ddDtD 7ufTbEuSgQVMRvZT/9aCpEl4/g7o5yIKUFWyJuXViSxT/n9hk1a1rdRaWun6Ges= =nprr -----END PGP SIGNATURE----- --------------020607010000090202070304 Content-Type: text/plain; name="sa.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sa.diff" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv c2EuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvemZzL3NhLmMJKHJldmlzaW9uIDIyNjM2NikKKysrIHN5cy9jZGRsL2Nv bnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc2EuYwkod29ya2luZyBjb3B5 KQpAQCAtNjA1LDE0ICs2MDUsMTQgQEAgc2FfZmluZF9zaXplcyhzYV9vc190ICpzYSwgc2Ff YnVsa19hdHRyX3QgKmF0dHJfZGUKIAkJICogYW5kIHNwaWxsIGJ1ZmZlci4KIAkJICovCiAJ CWlmIChidWZ0eXBlID09IFNBX0JPTlVTICYmICppbmRleCA9PSAtMSAmJgotCQkgICAgUDJS T1VORFVQKCp0b3RhbCArIGhkcnNpemUsIDgpID4KKwkJICAgICgqdG90YWwgKyBQMlJPVU5E VVAoaGRyc2l6ZSwgOCkpID4KIAkJICAgIChmdWxsX3NwYWNlIC0gc2l6ZW9mIChibGtwdHJf dCkpKSB7CiAJCQkqaW5kZXggPSBpOwogCQkJZG9uZSA9IEJfVFJVRTsKIAkJfQogCiBuZXh0 OgotCQlpZiAoUDJST1VORFVQKCp0b3RhbCArIGhkcnNpemUsIDgpID4gZnVsbF9zcGFjZSAm JgorCQlpZiAoKCp0b3RhbCArIFAyUk9VTkRVUChoZHJzaXplLCA4KSkgPiBmdWxsX3NwYWNl ICYmCiAJCSAgICBidWZ0eXBlID09IFNBX0JPTlVTKQogCQkJKndpbGxfc3BpbGwgPSBCX1RS VUU7CiAJfQo= --------------020607010000090202070304-- From owner-zfs-devel@FreeBSD.ORG Mon Oct 17 22:14:56 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8F1C106566B for ; Mon, 17 Oct 2011 22:14:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8AD218FC17 for ; Mon, 17 Oct 2011 22:14:56 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 45FCA147764; Tue, 18 Oct 2011 00:14:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id nt6sA8FNJhst; Tue, 18 Oct 2011 00:14:53 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 0C515147759; Tue, 18 Oct 2011 00:14:52 +0200 (CEST) Message-ID: <4E9CA8DB.4070404@FreeBSD.org> Date: Tue, 18 Oct 2011 00:14:51 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: d@delphij.net References: <4E98B5F3.7020103@delphij.net> In-Reply-To: <4E98B5F3.7020103@delphij.net> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH FOR REVIEW] Extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2011 22:14:56 -0000 On 15. 10. 2011 0:21, Xin LI wrote: > Hi, > > I think we hit a bug in sa_find_sizes: > > - hdrsize grows by 4 bytes each time; > - On return, hdrsize is rounded up to nearest 8 byte boundary; > - On the other hand, when testing for whether spill is needed, the > code uses sum of (*total + hdrsize) and rounds that number up to > nearest 8 byte boundary. > > This causes a problem when *total==306 and hdrsize==12. We have: > > (306 + 12) = 318 > ROUNDUP(318, 8) = 320 > > On return, we have: > > total = 306 > hdrsize = 16 > > And spill is unset. This would lead to a panic. > > Comments? > > Cheers, In my opinion your computation is correct, the bugfix should go to HEAD and be reported to OpenSolaris, too. If you want I can report it upstream. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Mon Oct 17 22:44:19 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76DDD106564A; Mon, 17 Oct 2011 22:44:19 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5D3468FC0A; Mon, 17 Oct 2011 22:44:19 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 148CE182E3; Mon, 17 Oct 2011 15:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1318891459; bh=/pV+JfWV2oGh5e+obPFqyhYPI1tpDj8mtJt6iWrA4IA=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Yyt3ElTPcVlpNJM2FLA/KArqqs6abubvokknNTGZ7YjeI3vLz4rTdDaiHc48MEU9B 9XpltUswhmODZuvx+XPGjHzdBQeTGT50xbiH26RUevWswIMR2XV8UkShl85U9p6I9d S3kOnr5Pf34Gk1xftUXqSs0lHvOZsFxX4hpLfuJc= Message-ID: <4E9CAFC2.9020200@delphij.net> Date: Mon, 17 Oct 2011 15:44:18 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska References: <4E82FA22.5020406@FreeBSD.org> In-Reply-To: <4E82FA22.5020406@FreeBSD.org> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek Subject: Re: [PATCH] Illumos changeset #13469 (dmu_tx.c) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2011 22:44:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 09/28/11 03:42, Martin Matuska wrote: > Illumos has now provided a fix for the Issue #1475 regarding > invalid spill blkptr: https://www.illumos.org/issues/1475 > > changeset: 13469:b8e89e5c4167 tag: tip user: > Albert Lee date: Sun Sep 25 03:07:35 > 2011 -0400 summary: 1475 zfs spill block hold can access > invalid spill blkptr > https://github.com/illumos/illumos-gate/commit/9dccfd2a04cd1645e2616b7307b3a88041aba991 > > A partial fix by pjd was already in our code: > http://p4db.freebsd.org/changeView.cgi?CH=185940 > http://p4db.freebsd.org/changeView.cgi?CH=185942 > > I suggest importing the full fix to match the Illumos version. > Please review and/or comment the attached patch. I think these changes are safe because they did not change the logic internals and should be incorporated since they reduced diff against upstream. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOnK/BAAoJEATO+BI/yjfBnFsH+gMc+Hj5867phimJ6eJNACHQ xo5TK8eTVRs5dpOkpiqAtZuSEFQ5y8he1DYjTPrQOe3lPWJDoGJwAsHZtgQsgtAT ls8HmScri1RYkAlpEtCxvGKImFfB84agwWf27coEH54oDBQ4DVhPXhUlZ6xzgU1u w44lydOUjIWyO1RVydOckDnr7i6CjNm6lSHXBVVM40He0ScDdcG6O5MQChVuJrSD ibBJmXqIpWvNF97n2uvT6wSFV7Kk+npCdKIysG1fVqMEFiIQixlvXBk4DmNKcwuj eWPeKuL32wpDk04bJp0ca1heNavQwojkQn2IYdGFrdAYFFXT7kPOrb01ju+YTLg= =f4Z7 -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sun Oct 30 09:16:38 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9ED8F106566B for ; Sun, 30 Oct 2011 09:16:38 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (unknown [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 24FD18FC0C for ; Sun, 30 Oct 2011 09:16:38 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 83FA5BD77 for ; Sun, 30 Oct 2011 10:16:37 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id 0RFvJ5qKAces for ; Sun, 30 Oct 2011 10:16:35 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id D6877BD61 for ; Sun, 30 Oct 2011 10:16:34 +0100 (CET) Message-ID: <4EAD15F2.7070807@FreeBSD.org> Date: Sun, 30 Oct 2011 10:16:34 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Preview: upcoming Illumos ZFS changes in code review phase X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Oct 2011 09:16:38 -0000 Hi everyone, ZFS development in Illumos seems to be going on. I am in touch with the developers and closely following the activity on the mailing lists. Here is a preview of the patches and enhancements that are in the code review phase: 1.) show zfs ioctl args in truss (might be of interest for us) https://www.illumos.org/issues/1688 WEBREV: http://yalms.org/cr/truss-zfs/ 2.) 1644 add ZFS "clones" property https://www.illumos.org/issues/1644 1645 add ZFS "written" and "written@..." properties https://www.illumos.org/issues/1645 1646 "zfs send" should estimate size of stream https://www.illumos.org/issues/1646 1647 "zfs destroy" should determine space reclaimed by destroying multiple snapshots https://www.illumos.org/issues/1647 WEBREV: http://code.delphix.com/webrev-props/ 3.) persistent 'comment' field for a zpool https://www.illumos.org/issues/1693 WEBREV: http://www.kebe.com/~danmcd/webrevs/hackathon/zfs-comment/ 4.) "zpool reguid" command (change zpool's guid number) WEBREV: http://dev1.illumos.org/~gdamore/reguid/ Some of the patches have already received lots of comments so the final code added to Illumos is going to be different. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sat Nov 5 18:40:38 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B1CC106566B; Sat, 5 Nov 2011 18:40:38 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 153F78FC19; Sat, 5 Nov 2011 18:40:38 +0000 (UTC) Received: by iabz21 with SMTP id z21so6140959iab.13 for ; Sat, 05 Nov 2011 11:40:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:content-type:subject:date:message-id:cc:to:mime-version :x-mailer; bh=VyBC9/xc+RK58PcD+CYs+yatwGiKrv1SrvBg2Exnw94=; b=ostZgN57vieakv/T9ogETgo1FN7oXavAN3t6YMroBasjIYbzHLHqWYpjIXmm4zHvlN 7pG9S0jCSY5zMJMlkLXQVxhQvxKWuPPyXzMFgm+rkgqLgLhu69lTr6FN4N6ve9lGwY1x 5/oUt6cXMvahRl7Hi0d/cSQib/bmlKgcW2Apk= Received: by 10.42.147.72 with SMTP id m8mr28044670icv.56.1320516873555; Sat, 05 Nov 2011 11:14:33 -0700 (PDT) Received: from [192.168.20.5] (c-24-6-49-154.hsd1.ca.comcast.net. [24.6.49.154]) by mx.google.com with ESMTPS id d8sm8125683pbb.6.2011.11.05.11.14.31 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 05 Nov 2011 11:14:32 -0700 (PDT) From: Garrett Cooper Content-Type: multipart/mixed; boundary=Apple-Mail-3-759769076 Date: Sat, 5 Nov 2011 11:14:28 -0700 Message-Id: To: zfs-devel@FreeBSD.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Cc: rdivacky@freebsd.org, Dimitry Andric Subject: [PATCH] Fix some coding issues in zpool(8) uncovered by clang X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Nov 2011 18:40:38 -0000 --Apple-Mail-3-759769076 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Long story short, I was compiling rescue with clang this past = week with -g and -O0, which unrooted a few issues with zpool (more than = the standard optimization, BTW..). The following patch fixes some = formatting string and assign/test issues in zpool(8), and shouldn't = introduce any noticeable functional changes. If someone could review and possibly commit it, that would be = great. Thanks, -Garrett --Apple-Mail-3-759769076 Content-Disposition: attachment; filename=zpool-fix-clang-compilation-warnings.patch Content-Type: application/octet-stream; name="zpool-fix-clang-compilation-warnings.patch" Content-Transfer-Encoding: 7bit Index: cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c (revision 227072) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c (working copy) @@ -132,7 +132,7 @@ for (i = 0; i < argc; i++) { zpool_handle_t *zhp; - if (zhp = zpool_open_canfail(g_zfs, argv[i])) { + if ((zhp = zpool_open_canfail(g_zfs, argv[i]))) { if (add_pool(zhp, zlp) != 0) *err = B_TRUE; } else { Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 227072) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (working copy) @@ -2566,9 +2566,9 @@ if (pl->pl_next == NULL && !right_justify) (void) printf("%s", header); else if (right_justify) - (void) printf("%*s", pl->pl_width, header); + (void) printf("%*s", (int)pl->pl_width, header); else - (void) printf("%-*s", pl->pl_width, header); + (void) printf("%-*s", (int)pl->pl_width, header); } (void) printf("\n"); @@ -4066,7 +4066,7 @@ if (cur_version > cbp->cb_version) { (void) printf(gettext("Pool '%s' is already formatted " "using more current version '%llu'.\n"), - zpool_get_name(zhp), cur_version); + zpool_get_name(zhp), (u_longlong_t)cur_version); return (0); } if (cur_version == cbp->cb_version) { @@ -4321,8 +4321,9 @@ continue; (void) snprintf(internalstr, sizeof (internalstr), - "[internal %s txg:%lld] %s", - zfs_history_event_names[ievent], txg, + "[internal %s txg:%llu] %s", + zfs_history_event_names[ievent], + (u_longlong_t)txg, pathstr); cmdstr = internalstr; } --Apple-Mail-3-759769076-- From owner-zfs-devel@FreeBSD.ORG Fri Nov 18 13:04:53 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D71101065672; Fri, 18 Nov 2011 13:04:53 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 99C478FC1A; Fri, 18 Nov 2011 13:04:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 0B4FC13C69; Fri, 18 Nov 2011 14:04:53 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id m0lXdgdl6Z6R; Fri, 18 Nov 2011 14:04:48 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id EFD0F13C57; Fri, 18 Nov 2011 14:04:47 +0100 (CET) Message-ID: <4EC657EF.2070602@FreeBSD.org> Date: Fri, 18 Nov 2011 14:04:47 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Garrett Cooper References: In-Reply-To: X-Enigmail-Version: 1.3.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Dimitry Andric , rdivacky@freebsd.org Subject: Re: [PATCH] Fix some coding issues in zpool(8) uncovered by clang X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 13:04:53 -0000 On 5.11.2011 19:14, Garrett Cooper wrote: > Long story short, I was compiling rescue with clang this past week with -g and -O0, which unrooted a few issues with zpool (more than the standard optimization, BTW..). The following patch fixes some formatting string and assign/test issues in zpool(8), and shouldn't introduce any noticeable functional changes. > If someone could review and possibly commit it, that would be great. > Thanks, > -Garrett Looks good. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Fri Nov 18 13:17:59 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEA1B106566B; Fri, 18 Nov 2011 13:17:59 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 945088FC12; Fri, 18 Nov 2011 13:17:59 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 0836E103A9; Fri, 18 Nov 2011 14:17:59 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id mG2xcVfDXOtC; Fri, 18 Nov 2011 14:17:54 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id E508010394; Fri, 18 Nov 2011 14:17:53 +0100 (CET) Message-ID: <4EC65B01.4000906@FreeBSD.org> Date: Fri, 18 Nov 2011 14:17:53 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Pawel Jakub Dawidek Subject: [PATCH] New ZFS features from Illumos (13514, 13524, 13525) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 13:17:59 -0000 I have ported the recent changes from Illumos for our tree. The patch can be downloaded here: http://people.freebsd.org/~mm/patches/zfs/head-illumos-13514-13524-13525.patch What is it all about? It is three revisions covering 7 features. There are no changes to the on-disk format. There three new IOCTL's and one has been changed. I have written some compatibility code for the changed IOCTL (ZFS_IOC_DESTROY_SNAPS) so that it detects if the given argument is a nvlist. This way the recursive destroy of ZFS snapshots works with the older binaries on the new kernel. The other way (new binaries, old kernel) is little more problematic. If using the new flags to zfs send there may be (some) crambled output and it is not possible to destroy snapshots. The problem is I cannot properly detect (in another way than osversion) what utiliy versions are we running. But if this isn't such a big problem (new binary, old kernel) we could go with this version. Here is a list of the features, for detailed descriptions see links underneath: 1644 add ZFS "clones" property https://www.illumos.org/issues/1644 1645 add ZFS "written" and "written@..." properties https://www.illumos.org/issues/1645 1646 "zfs send" should estimate size of stream https://www.illumos.org/issues/1646 1647 "zfs destroy" should determine space reclaimed by destroying multiple snapshots https://www.illumos.org/issues/1647 1693 persistent 'comment' field for a zpool https://www.illumos.org/issues/1693 1708 adjust size of zpool history data https://www.illumos.org/issues/1708 1748 desire support for reguid in zfs https://www.illumos.org/issues/1748 Illumos changesets:: 13514: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/417c34452f03 (1748) 13524: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/417c34452f03 (1644, 1645, 1646, 1647, 1708) 13525: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/7059b67f1bc2 (1693) Please review and/or test my patch, comments and suggestions are welcome. http://people.freebsd.org/~mm/patches/zfs/head-illumos-13514-13524-13525.patch -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Dec 7 18:34:05 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A88411065672 for ; Wed, 7 Dec 2011 18:34:05 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BBD458FC15 for ; Wed, 7 Dec 2011 18:34:04 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA01893 for ; Wed, 07 Dec 2011 20:14:37 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EDFAD0D.2080100@FreeBSD.org> Date: Wed, 07 Dec 2011 20:14:37 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org References: <4EDFABD7.20301@FreeBSD.org> In-Reply-To: <4EDFABD7.20301@FreeBSD.org> X-Enigmail-Version: undefined X-Forwarded-Message-Id: <4EDFABD7.20301@FreeBSD.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: Fwd: Re: zfs-related(?) panic in cache_enter: wrong vnode type X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2011 18:34:05 -0000 Trying to find ZFS experts here too :-) -------- Original Message -------- Message-ID: <4EDFABD7.20301@FreeBSD.org> Date: Wed, 07 Dec 2011 20:09:27 +0200 From: Andriy Gapon To: freebsd-current@FreeBSD.ORG, freebsd-fs@FreeBSD.ORG Subject: Re: zfs-related(?) panic in cache_enter: wrong vnode type References: <4EDF995B.9000407@FreeBSD.org> <4EDFA354.60909@FreeBSD.org> In-Reply-To: <4EDFA354.60909@FreeBSD.org> on 07/12/2011 19:33 Andriy Gapon said the following: > > A detail that may or may not be useful. > It seems that the panic happened when tried to resume a vim process. The process > was suspended, its current directory and a file being edited/viewed may have been > already removed. And another data point: (kgdb) p ((znode_t *)dvp->v_data)->z_unlinked Perhaps lookup in unlinked directories should just fail? I am not sufficiently familiar with VFS, so just guessing. > on 07/12/2011 18:50 Andriy Gapon said the following: >> >> (kgdb) bt >> #0 doadump (textdump=1) at pcpu.h:224 >> #1 0xffffffff804f6d3b in kern_reboot (howto=260) at >> /usr/src/sys/kern/kern_shutdown.c:447 >> #2 0xffffffff804f63e9 in panic (fmt=0x104
) at >> /usr/src/sys/kern/kern_shutdown.c:635 >> #3 0xffffffff80585f46 in cache_enter (dvp=0xfffffe003d4763c0, >> vp=0xfffffe0142517000, cnp=0xffffff82393b3708) at /usr/src/sys/kern/vfs_cache.c:726 >> #4 0xffffffff81a90900 in zfs_lookup (dvp=0xfffffe003d4763c0, >> nm=0xffffff82393b3140 "..", vpp=0xffffff82393b36e0, cnp=0xffffff82393b3708, >> nameiop=0, cr=0xfffffe0042e88100, td=0xfffffe000fdfa480, >> flags=0) at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1470 >> #5 0xffffffff81a91570 in zfs_freebsd_lookup (ap=0xffffff82393b32c0) at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5858 >> #6 0xffffffff8073f054 in VOP_CACHEDLOOKUP_APV (vop=0xffffffff81b05a20, >> a=0xffffff82393b32c0) at vnode_if.c:187 >> #7 0xffffffff80586bf4 in vfs_cache_lookup (ap=Variable "ap" is not available. >> ) at vnode_if.h:80 >> #8 0xffffffff80740a5c in VOP_LOOKUP_APV (vop=0xffffffff81b05a20, >> a=0xffffff82393b33a0) at vnode_if.c:123 >> #9 0xffffffff8058e42c in lookup (ndp=0xffffff82393b36a0) at vnode_if.h:54 >> #10 0xffffffff8058f17e in namei (ndp=0xffffff82393b36a0) at >> /usr/src/sys/kern/vfs_lookup.c:312 >> #11 0xffffffff805a890d in vn_open_cred (ndp=0xffffff82393b36a0, >> flagp=0xffffff82393b3918, cmode=0, vn_open_flags=Variable "vn_open_flags" is not >> available. >> ) at /usr/src/sys/kern/vfs_vnops.c:195 >> #12 0xffffffff80589e7e in vop_stdvptocnp (ap=Variable "ap" is not available. >> ) at /usr/src/sys/kern/vfs_default.c:774 >> #13 0xffffffff8073b012 in VOP_VPTOCNP_APV (vop=0xffffffff80a99140, >> a=0xffffff82393b39b0) at vnode_if.c:3479 >> #14 0xffffffff80584665 in vn_vptocnp_locked (vp=0xffffff82393b3a50, >> cred=0xfffffe0042e88100, >> buf=0xfffffe000ca06000 >> "������������������������������������������������������������������������������������������������������������������������������������������������������"..., >> buflen=0xffffff82393b3a4c) at vnode_if.h:1564 >> #15 0xffffffff80584bab in vn_fullpath1 (td=0xfffffe000fdfa480, >> vp=0xfffffe003d4763c0, rdir=0xfffffe000cd4d000, >> buf=0xfffffe000ca06000 >> "������������������������������������������������������������������������������������������������������������������������������������������������������"..., >> retbuf=0xffffff82393b3ab0, buflen=1023) at /usr/src/sys/kern/vfs_cache.c:1218 >> #16 0xffffffff8058526a in kern___getcwd (td=0xfffffe000fdfa480, buf=0x80880a000 >>
, bufseg=UIO_USERSPACE, buflen=1024) at >> /usr/src/sys/kern/vfs_cache.c:960 >> #17 0xffffffff805853f4 in sys___getcwd (td=Variable "td" is not available. >> ) at /usr/src/sys/kern/vfs_cache.c:934 >> #18 0xffffffff806d2069 in amd64_syscall (td=0xfffffe000fdfa480, traced=0) at >> subr_syscall.c:131 >> #19 0xffffffff806bb4e7 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 >> #20 0x00000008031adb2c in ?? () >> Previous frame inner to this frame (corrupt stack?) >> (kgdb) fr 3 >> #3 0xffffffff80585f46 in cache_enter (dvp=0xfffffe003d4763c0, >> vp=0xfffffe0142517000, cnp=0xffffff82393b3708) at /usr/src/sys/kern/vfs_cache.c:726 >> 726 KASSERT(vp == NULL || vp->v_type == VDIR, >> (kgdb) list >> 721 if (dvp->v_cache_dd != NULL) { >> 722 CACHE_WUNLOCK(); >> 723 cache_free(ncp); >> 724 return; >> 725 } >> 726 KASSERT(vp == NULL || vp->v_type == VDIR, >> 727 ("wrong vnode type %p", vp)); >> 728 dvp->v_cache_dd = ncp; >> 729 } >> 730 >> (kgdb) p *vp >> $1 = {v_type = VREG, v_tag = 0xffffffff81afe449 "zfs", v_op = 0xffffffff81b05a20, >> v_data = 0xfffffe020d9a8320, v_mount = 0xfffffe001a283600, v_nmntvnodes = >> {tqe_next = 0xfffffe00347c6d20, >> tqe_prev = 0xfffffe013b0575c8}, v_un = {vu_mount = 0x0, vu_socket = 0x0, >> vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, >> v_hash = 0, v_cache_src = { >> lh_first = 0x0}, v_cache_dst = {tqh_first = 0xfffffe003dfcf690, tqh_last = >> 0xfffffe003dfcf6b0}, v_cache_dd = 0x0, v_cstart = 0, v_lasta = 0, v_lastw = 0, >> v_clen = 0, v_lock = {lock_object = { >> lo_name = 0xffffffff81afe449 "zfs", lo_flags = 91947008, lo_data = 0, >> lo_witness = 0xffffff800066e380}, lk_lock = 18446741874952610944, lk_exslpfail = >> 0, lk_timo = 51, lk_pri = 96}, >> v_interlock = {lock_object = {lo_name = 0xffffffff807e610a "vnode interlock", >> lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff8000665600}, mtx_lock = 4}, >> v_vnlock = 0xfffffe0142517098, >> v_holdcnt = 2, v_usecount = 1, v_iflag = 0, v_vflag = 0, v_writecount = 0, >> v_freelist = {tqe_next = 0xfffffe00347c6d20, tqe_prev = 0xfffffe02110d3e30}, >> v_bufobj = {bo_mtx = {lock_object = { >> lo_name = 0xffffffff807efdfa "bufobj interlock", lo_flags = 16973824, >> lo_data = 0, lo_witness = 0xffffff800066c300}, mtx_lock = 4}, bo_clean = {bv_hd = >> {tqh_first = 0x0, >> tqh_last = 0xfffffe0142517140}, bv_root = 0x0, bv_cnt = 0}, bo_dirty = >> {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe0142517160}, bv_root = 0x0, bv_cnt = >> 0}, bo_numoutput = 0, bo_flag = 0, >> bo_ops = 0xffffffff80a95ec0, bo_bsize = 131072, bo_object = >> 0xfffffe01fccb5620, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = >> 0xfffffe0142517000, __bo_vnode = 0xfffffe0142517000}, >> v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} >> (kgdb) >> > > -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Dec 17 13:44:45 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AD4B106566B; Sat, 17 Dec 2011 13:44:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2A94D8FC12; Sat, 17 Dec 2011 13:44:43 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA04645; Sat, 17 Dec 2011 15:44:42 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbuZ8-0009ct-II; Sat, 17 Dec 2011 15:44:42 +0200 Message-ID: <4EEC9CC9.9030001@FreeBSD.org> Date: Sat, 17 Dec 2011 15:44:41 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek X-Enigmail-Version: undefined Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: vcmn_err: incorrect panic message X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 13:44:45 -0000 Guys, what do you think about the following change or a variation of it? With the current code, in the unfortunate event of panic-ing via vcmn_err() a panic message would actually be a format string (placeholders are not substituted). commit f6cd1dc812b4cfdb3a123052ce0cb49a5afa941d Author: Andriy Gapon Date: Thu Dec 15 00:04:31 2011 +0200 opensolaris compat: fix vcmn_err so that panic produces a proper message ... instead of just a verbatim format string diff --git a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c index 12e1854..abde30d 100644 --- a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c @@ -28,29 +28,35 @@ void vcmn_err(int ce, const char *fmt, va_list adx) { char buf[256]; + const char *prefix; + prefix = NULL; /* silence unwitty compilers */ switch (ce) { case CE_CONT: - snprintf(buf, sizeof(buf), "Solaris(cont): %s\n", fmt); + prefix = "Solaris(cont): "; break; case CE_NOTE: - snprintf(buf, sizeof(buf), "Solaris: NOTICE: %s\n", fmt); + prefix = "Solaris: NOTICE: "; break; case CE_WARN: - snprintf(buf, sizeof(buf), "Solaris: WARNING: %s\n", fmt); + prefix = "Solaris: WARNING: "; break; case CE_PANIC: - snprintf(buf, sizeof(buf), "Solaris(panic): %s\n", fmt); + prefix = "Solaris(panic): "; break; case CE_IGNORE: break; default: panic("Solaris: unknown severity level"); } - if (ce == CE_PANIC) - panic("%s", buf); - if (ce != CE_IGNORE) - vprintf(buf, adx); + if (ce == CE_PANIC) { + vsnprintf(buf, sizeof(buf), fmt, adx); + panic("%s%s", prefix, buf); + } + if (ce != CE_IGNORE) { + printf("%s", prefix); + vprintf(fmt, adx); + } } void -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Dec 18 12:50:05 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8706106566B for ; Sun, 18 Dec 2011 12:50:05 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 8476F8FC12 for ; Sun, 18 Dec 2011 12:50:05 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id B349AE9A; Sun, 18 Dec 2011 13:32:44 +0100 (CET) Date: Sun, 18 Dec 2011 13:31:38 +0100 From: Pawel Jakub Dawidek To: Andriy Gapon Message-ID: <20111218123137.GD1685@garage.freebsd.pl> References: <4EEC9CC9.9030001@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TybLhxa8M7aNoW+V" Content-Disposition: inline In-Reply-To: <4EEC9CC9.9030001@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: vcmn_err: incorrect panic message X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 12:50:05 -0000 --TybLhxa8M7aNoW+V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 17, 2011 at 03:44:41PM +0200, Andriy Gapon wrote: > Guys, >=20 > what do you think about the following change or a variation of it? > With the current code, in the unfortunate event of panic-ing via vcmn_err= () a > panic message would actually be a format string (placeholders are not sub= stituted). The patch looks good. > commit f6cd1dc812b4cfdb3a123052ce0cb49a5afa941d > Author: Andriy Gapon > Date: Thu Dec 15 00:04:31 2011 +0200 >=20 > opensolaris compat: fix vcmn_err so that panic produces a proper mess= age >=20 > ... instead of just a verbatim format string >=20 > diff --git a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > index 12e1854..abde30d 100644 > --- a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > @@ -28,29 +28,35 @@ void > vcmn_err(int ce, const char *fmt, va_list adx) > { > char buf[256]; > + const char *prefix; >=20 > + prefix =3D NULL; /* silence unwitty compilers */ > switch (ce) { > case CE_CONT: > - snprintf(buf, sizeof(buf), "Solaris(cont): %s\n", fmt); > + prefix =3D "Solaris(cont): "; > break; > case CE_NOTE: > - snprintf(buf, sizeof(buf), "Solaris: NOTICE: %s\n", fmt); > + prefix =3D "Solaris: NOTICE: "; > break; > case CE_WARN: > - snprintf(buf, sizeof(buf), "Solaris: WARNING: %s\n", fmt); > + prefix =3D "Solaris: WARNING: "; > break; > case CE_PANIC: > - snprintf(buf, sizeof(buf), "Solaris(panic): %s\n", fmt); > + prefix =3D "Solaris(panic): "; > break; > case CE_IGNORE: > break; > default: > panic("Solaris: unknown severity level"); > } > - if (ce =3D=3D CE_PANIC) > - panic("%s", buf); > - if (ce !=3D CE_IGNORE) > - vprintf(buf, adx); > + if (ce =3D=3D CE_PANIC) { > + vsnprintf(buf, sizeof(buf), fmt, adx); > + panic("%s%s", prefix, buf); > + } > + if (ce !=3D CE_IGNORE) { > + printf("%s", prefix); > + vprintf(fmt, adx); > + } > } >=20 > void --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --TybLhxa8M7aNoW+V Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk7t3SgACgkQForvXbEpPzS/UQCgzG+1NjbxNM6eaqiVWoOGHK4I bLkAnisMJYS93VN3R6kithVmrYK4cPWN =u/vl -----END PGP SIGNATURE----- --TybLhxa8M7aNoW+V-- From owner-zfs-devel@FreeBSD.ORG Sat Feb 25 09:56:26 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B60E8106564A; Sat, 25 Feb 2012 09:56:26 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 2EA948FC12; Sat, 25 Feb 2012 09:56:23 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 5B65B2510A; Sat, 25 Feb 2012 10:56:22 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id PFje1-WLoNTh; Sat, 25 Feb 2012 10:56:20 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 258A7250FD; Sat, 25 Feb 2012 10:56:20 +0100 (CET) Message-ID: <4F48B043.6020006@FreeBSD.org> Date: Sat, 25 Feb 2012 10:56:19 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <201201052216.q05MGfLY058717@svn.freebsd.org> In-Reply-To: <201201052216.q05MGfLY058717@svn.freebsd.org> X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: Re: svn commit: r229663 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 09:56:26 -0000 As of our current code arc_meta_limit is not enforced at all, because dnlc_reduce_cache() does nothing. This way the meta cache usually outgrows its limit which should be 1/4 of arc by default. Users in mailinglists report performance penalties due to this problem. We could learn from Brian Behlendorf's zfsonlinux and implement something similiar. In his first attempt: https://github.com/behlendorf/zfs/commit/6a8f9b6bf0de3e3d09fcfa32e129c978e7641a8f He added a arc_kmem_reap_now() if metadata is outside the arc_meta_limit. Then he introduced arc_adjust_meta() which looks promising, we might be interested in doing something like that: https://github.com/behlendorf/zfs/commit/ab26409db753bb087842ab6f1af943f3386c764f#L53R2004 On 5.1.2012 23:16, Pawel Jakub Dawidek wrote: > Author: pjd > Date: Thu Jan 5 22:16:41 2012 > New Revision: 229663 > URL: http://svn.freebsd.org/changeset/base/229663 > > Log: > - Allow to change vfs.zfs.arc_meta_limit at runtime. > - Change vfs.zfs.arc_meta_used from CTLFLAG_RDTUN to CTLFLAG_RD, as it is > not a tunable. > > MFC after: 3 days > > Modified: > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:14:49 2012 (r229662) > +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:16:41 2012 (r229663) > @@ -464,10 +464,10 @@ static uint64_t arc_loaned_bytes; > static uint64_t arc_meta_used; > static uint64_t arc_meta_limit; > static uint64_t arc_meta_max = 0; > -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RDTUN, > - &arc_meta_used, 0, "ARC metadata used"); > -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RDTUN, > - &arc_meta_limit, 0, "ARC metadata limit"); > +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RD, &arc_meta_used, 0, > + "ARC metadata used"); > +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RW, &arc_meta_limit, 0, > + "ARC metadata limit"); > > typedef struct l2arc_buf_hdr l2arc_buf_hdr_t; > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sat Feb 25 10:15:04 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DBE7106566B; Sat, 25 Feb 2012 10:15:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 091EA8FC12; Sat, 25 Feb 2012 10:15:04 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id F10A325890; Sat, 25 Feb 2012 11:15:01 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id nwA9sXoe1lX9; Sat, 25 Feb 2012 11:15:00 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id CA02325886; Sat, 25 Feb 2012 11:14:59 +0100 (CET) Message-ID: <4F48B4A4.7060800@FreeBSD.org> Date: Sat, 25 Feb 2012 11:15:00 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <201201052216.q05MGfLY058717@svn.freebsd.org> <4F48B043.6020006@FreeBSD.org> In-Reply-To: <4F48B043.6020006@FreeBSD.org> X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: Enforcing arc_meta_limit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 10:15:04 -0000 I am correcting myself: The wording "not at all" is incorrect, but it is possible to outgrow the limit and I can confirm that on several of my servers with higher ZFS utilization. On 25.2.2012 10:56, Martin Matuska wrote: > As of our current code arc_meta_limit is not enforced at all, because > dnlc_reduce_cache() does nothing. > This way the meta cache usually outgrows its limit which should be 1/4 > of arc by default. > Users in mailinglists report performance penalties due to this problem. > > We could learn from Brian Behlendorf's zfsonlinux and implement > something similiar. > > In his first attempt: > https://github.com/behlendorf/zfs/commit/6a8f9b6bf0de3e3d09fcfa32e129c978e7641a8f > He added a arc_kmem_reap_now() if metadata is outside the arc_meta_limit. > > Then he introduced arc_adjust_meta() which looks promising, we might be > interested in doing something like that: > https://github.com/behlendorf/zfs/commit/ab26409db753bb087842ab6f1af943f3386c764f#L53R2004 > > On 5.1.2012 23:16, Pawel Jakub Dawidek wrote: >> Author: pjd >> Date: Thu Jan 5 22:16:41 2012 >> New Revision: 229663 >> URL: http://svn.freebsd.org/changeset/base/229663 >> >> Log: >> - Allow to change vfs.zfs.arc_meta_limit at runtime. >> - Change vfs.zfs.arc_meta_used from CTLFLAG_RDTUN to CTLFLAG_RD, as it is >> not a tunable. >> >> MFC after: 3 days >> >> Modified: >> head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c >> >> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c >> ============================================================================== >> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:14:49 2012 (r229662) >> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:16:41 2012 (r229663) >> @@ -464,10 +464,10 @@ static uint64_t arc_loaned_bytes; >> static uint64_t arc_meta_used; >> static uint64_t arc_meta_limit; >> static uint64_t arc_meta_max = 0; >> -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RDTUN, >> - &arc_meta_used, 0, "ARC metadata used"); >> -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RDTUN, >> - &arc_meta_limit, 0, "ARC metadata limit"); >> +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RD, &arc_meta_used, 0, >> + "ARC metadata used"); >> +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RW, &arc_meta_limit, 0, >> + "ARC metadata limit"); >> >> typedef struct l2arc_buf_hdr l2arc_buf_hdr_t; >> > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Mon May 14 22:03:44 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 078A11065786; Mon, 14 May 2012 22:03:44 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA038FC1B; Mon, 14 May 2012 22:03:40 +0000 (UTC) Received: by dadv36 with SMTP id v36so7371632dad.13 for ; Mon, 14 May 2012 15:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=t3otcqBdzxMEfbZH8bw76fc4FPYr5NoxBTPGx+ZBPeM=; b=JPiAyyK+/JsCiENc/kFw+pblFoaN8kN9ajRooNxdUEB2wXOcElmmRmANKW0JRB0XnH UpCnmrEN0DgxULswbO77rDfhTbZHA/lhvAYrekchKiqCReKfs/23RhhIf01JK8dq792u kiPomGzPU9MTuwOpYurg7Gp/qPBrwcmk+jZAwy8xnWoMKcKK69YngKxhABXb7GwK23DB O51mtIx4Am/IE2LG5TSCUlOnI+rVU0Xtc3nHsKdlQFFYgrYYWR1MGLIKbyEyZRzprkhP ghazve40SzcI0tSooKNgGURFLYOUNG+kTlPJwb+wihhZRTJNrHiZ41gvXeRipLsClQ65 ibDw== MIME-Version: 1.0 Received: by 10.68.218.72 with SMTP id pe8mr26849356pbc.45.1337033019956; Mon, 14 May 2012 15:03:39 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.143.154.25 with HTTP; Mon, 14 May 2012 15:03:39 -0700 (PDT) Date: Mon, 14 May 2012 15:03:39 -0700 X-Google-Sender-Auth: FQ3HDNLtfWSh7VTZg9xzc4Sdn3U Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org, Martin Matuska , pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2012 22:03:44 -0000 Hi pawel / xin / martin / all Hope you are all doing well. It has been quite a while since we exchanged mail. At HighCloud ( http://www.highcloudsecurity.com) , we released our first VM Centric Security product (based on freebsd and ZFS) back in October 2011. We are gaining traction with customers and are continuing to enhance the features, performance and stability of our product (just released 1.1.5 a couple of weeks ago). I'd like to introduce Chuck Silvers to this group. He is a former colleague from Veritas / Symantec's file systems team and now works with me at High Cloud Security. Chuck is also a long time netbsd developer (but don't hold that against him :-)) Chuck has been looking at ZFS performance and stability issues for us and wanted to share a couple of interesting results with this group. I'll let him speak about them himself. Regards, -- Tushar | 408.203.9736 | tushar.tambay@gmail.com From owner-zfs-devel@FreeBSD.ORG Mon May 14 22:05:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D68D1106566C; Mon, 14 May 2012 22:05:35 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9991F8FC0C; Mon, 14 May 2012 22:05:35 +0000 (UTC) Received: by dadv36 with SMTP id v36so7373501dad.13 for ; Mon, 14 May 2012 15:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=px/A3z0tSLjs6cxfpbnR6A8omly+4Y9HLFTQj4jKjJc=; b=THM3w4Igf0M4zMbmd6tejO38oAS/uCpOzxRncOvVOoyX2BJGf4GwFcN0rmx98TlMye yaL5AZ8eHLXd2bOclfqESYV2glIdI5X2GoUCpkVUVx7x7ULj/QyVpDYyJh/wZZRNz7Mm hcrByWMGjMcDOrpY/Ms1KeJy15zUsqpDz2PZgw6d5LzS2KK0OGLMlRG17MTsXubiSbGh Tlet5uDk8QXs5EyhV4lwR0NrDDgKlibUJrj+zm9oMK/Ts/Jqm+BAD/wjexOGO8qYuOoQ EEDFXGBJJ81FR2VTV6i6agdBbStYva5Jk8WdaJ4jw/fqI68Y2+YiEFRuVtc7lx0Oq3Hl aCfQ== MIME-Version: 1.0 Received: by 10.68.202.8 with SMTP id ke8mr26786141pbc.94.1337033135306; Mon, 14 May 2012 15:05:35 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.143.154.25 with HTTP; Mon, 14 May 2012 15:05:35 -0700 (PDT) In-Reply-To: References: Date: Mon, 14 May 2012 15:05:35 -0700 X-Google-Sender-Auth: IfNMsMHr2E8zyRQJ3xiDnWP9-xI Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org, Martin Matuska , pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Chuck Silvers Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2012 22:05:36 -0000 forgot to CC Chuck himself :-) On Mon, May 14, 2012 at 3:03 PM, Tushar Tambay wrote: > Hi pawel / xin / martin / all > > Hope you are all doing well. > > It has been quite a while since we exchanged mail. At HighCloud ( > http://www.highcloudsecurity.com) , we released our first VM Centric > Security product (based on freebsd and ZFS) back in October 2011. We are > gaining traction with customers and are continuing to enhance the features, > performance and stability of our product (just released 1.1.5 a couple of > weeks ago). > > I'd like to introduce Chuck Silvers to this group. He is a former > colleague from Veritas / Symantec's file systems team and now works with > me at High Cloud Security. Chuck is also a long time netbsd developer (but > don't hold that against him :-)) > > Chuck has been looking at ZFS performance and stability issues for us and > wanted to share a couple of interesting results with this group. I'll let > him speak about them himself. > > Regards, > > -- > Tushar | 408.203.9736 | tushar.tambay@gmail.com > > -- Tushar | 408.203.9736 | tushar.tambay@gmail.com From owner-zfs-devel@FreeBSD.ORG Wed May 23 20:07:04 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 23451106566C; Wed, 23 May 2012 20:07:04 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id CDBBB8FC12; Wed, 23 May 2012 20:07:03 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 6601012DD6; Wed, 23 May 2012 13:07:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1337803623; bh=Jf+tq2q4zX8iou6SWzq64fc7wJlbWSzHGUOkPZ9AnvA=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=aDbKnUbYq056RT7C9SZIUmdCjyJG+Xm88Au5mNpNdCh+8y1bMcNWrIu7BlX+Q5UPI qpEF5C9kn4ZLXv6tiLv6a1EG8FyBtHNx2l7E6yZwP8CQkm8j8nk8fkvkp8s9kdcbv8 +VXAXbYyVE2fsXcdxaT15PvvHpBw8pSPEavVl+VQ= Message-ID: <4FBD4365.6030201@delphij.net> Date: Wed, 23 May 2012 13:07:01 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Tushar Tambay References: In-Reply-To: X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net, zfs-devel@freebsd.org, Chuck Silvers Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:07:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/14/12 15:05, Tushar Tambay wrote: > forgot to CC Chuck himself :-) > > On Mon, May 14, 2012 at 3:03 PM, Tushar Tambay > wrote: > >> Hi pawel / xin / martin / all >> >> Hope you are all doing well. >> >> It has been quite a while since we exchanged mail. At HighCloud >> ( http://www.highcloudsecurity.com) , we released our first VM >> Centric Security product (based on freebsd and ZFS) back in >> October 2011. We are gaining traction with customers and are >> continuing to enhance the features, performance and stability of >> our product (just released 1.1.5 a couple of weeks ago). >> >> I'd like to introduce Chuck Silvers to this group. He is a >> former colleague from Veritas / Symantec's file systems team and >> now works with me at High Cloud Security. Chuck is also a long >> time netbsd developer (but don't hold that against him :-)) >> >> Chuck has been looking at ZFS performance and stability issues >> for us and wanted to share a couple of interesting results with >> this group. I'll let him speak about them himself. Welcome! Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBCAAGBQJPvUNlAAoJEG80Jeu8UPuz6sAH/1PX+BoZuxG5ExCMVBCijee5 KN+x3xOXllf50kOgj3ig3PVqa/loFYC/tXQjRsI+F/zqmBIbt06T7kTIaewgdDG6 0Z1Hl0qGz9xZDjrVuPuuBRsxCoF0gmjYB5bFAuhXsGFbwElA2Dyoqom0/Fjl0HwA SToZ0QNJwnDyZ0z43wuf3M5qMvB0gdzT6cKXsItMDePh1JURi6tPJFCOcBg8BQE6 LXNw9o7CjeTd3YT62X9RFSnW7w5oz6V0gIGlRhWKRZtawu0ObO6+aIqlfvjywQ9W rMxUmYEkb/q/CT/JKyOI7zhsb5mgiS59UHTdb3kKQqBxuAUnPQBBAJ+qyefcr6s= =eQkM -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed May 23 20:15:40 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A8B7106564A for ; Wed, 23 May 2012 20:15:40 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id EECD58FC0A for ; Wed, 23 May 2012 20:15:39 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 6C6D535B; Wed, 23 May 2012 22:15:38 +0200 (CEST) Date: Wed, 23 May 2012 22:13:53 +0200 From: Pawel Jakub Dawidek To: zfs-devel@freebsd.org Message-ID: <20120523201353.GC1399@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DIOMP1UsTsWJauNi" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:15:40 -0000 --DIOMP1UsTsWJauNi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > Hi pawel / xin / martin / all >=20 > Hope you are all doing well. >=20 > It has been quite a while since we exchanged mail. At HighCloud ( > http://www.highcloudsecurity.com) , we released our first VM Centric > Security product (based on freebsd and ZFS) back in October 2011. We are > gaining traction with customers and are continuing to enhance the feature= s, > performance and stability of our product (just released 1.1.5 a couple of > weeks ago). >=20 > I'd like to introduce Chuck Silvers to this group. He is a former colleag= ue > from Veritas / Symantec's file systems team and now works with me at High > Cloud Security. Chuck is also a long time netbsd developer (but don't hold > that against him :-)) >=20 > Chuck has been looking at ZFS performance and stability issues for us and > wanted to share a couple of interesting results with this group. I'll let > him speak about them himself. Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! I added your e-mail to the list. Let me know if you received it. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --DIOMP1UsTsWJauNi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk+9RQEACgkQForvXbEpPzSOoQCg3Q9/ihJiGLPn8ZwWMl2MT6iF xJIAoNl15jaOCIdIDYNTDA1THUMGyyys =rmA+ -----END PGP SIGNATURE----- --DIOMP1UsTsWJauNi-- From owner-zfs-devel@FreeBSD.ORG Thu May 24 23:30:30 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40B9B106564A for ; Thu, 24 May 2012 23:30:30 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id B36B18FC15 for ; Thu, 24 May 2012 23:30:29 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 5773A217EC for ; Fri, 25 May 2012 01:30:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id Ih3AFwTdpEC1 for ; Fri, 25 May 2012 01:30:21 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id EF6EF217E4 for ; Fri, 25 May 2012 01:30:20 +0200 (CEST) Message-ID: <4FBEC48D.5080600@FreeBSD.org> Date: Fri, 25 May 2012 01:30:21 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS features (SPA_VERSION successor) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 23:30:30 -0000 Hi folks, last week Illumos has introduced a new concept called "ZFS features", a version-independent successor to SPA_VERSION, together with the first "feature" - async destroy. It is a huge commit and will take some time to implement (there is quite detailed documentation and manpages inside). As to the text in the issue 2619, the feature flags are based on a proposal to the ZFS working group: http://lists.freebsd.org/pipermail/freebsd-fs/2011-May/011568.html Maybe some of FreeBSD-related ZFS development could take advantage of this, when ported. References: https://hg.openindiana.org/upstream/illumos/illumos-gate/rev/2889e2596bd6 https://hg.openindiana.org/upstream/illumos/illumos-gate/rev/1949b688d5fb https://www.illumos.org/issues/2619 https://www.illumos.org/issues/2747 Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue May 29 18:50:30 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2683C106564A; Tue, 29 May 2012 18:50:30 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 05C228FC15; Tue, 29 May 2012 18:50:29 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id E5F23321081; Tue, 29 May 2012 19:42:09 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yfm+NVYjwckl; Tue, 29 May 2012 19:42:08 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id C164A32107E; Tue, 29 May 2012 19:42:08 +0100 (BST) Date: Tue, 29 May 2012 11:42:07 -0700 From: Chuck Silvers To: Pawel Jakub Dawidek Message-ID: <20120529184207.GB11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120523201353.GC1399@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 18:50:30 -0000 On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: > On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > > Hi pawel / xin / martin / all > > > > Hope you are all doing well. > > > > It has been quite a while since we exchanged mail. At HighCloud ( > > http://www.highcloudsecurity.com) , we released our first VM Centric > > Security product (based on freebsd and ZFS) back in October 2011. We are > > gaining traction with customers and are continuing to enhance the features, > > performance and stability of our product (just released 1.1.5 a couple of > > weeks ago). > > > > I'd like to introduce Chuck Silvers to this group. He is a former colleague > > from Veritas / Symantec's file systems team and now works with me at High > > Cloud Security. Chuck is also a long time netbsd developer (but don't hold > > that against him :-)) > > > > Chuck has been looking at ZFS performance and stability issues for us and > > wanted to share a couple of interesting results with this group. I'll let > > him speak about them himself. > > Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! > > I added your e-mail to the list. Let me know if you received it. hi pawel, I got it, thanks. we have a number of patches to freebsd that we're using, most of which modify parts of the system other than ZFS, should I just submit those as PRs? or post them to whichever freebsd-* mailing list seems the most appropriate? the few that are ZFS-specific I'll send here shortly. -Chuck From owner-zfs-devel@FreeBSD.ORG Tue May 29 20:38:31 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 706D9106566C; Tue, 29 May 2012 20:38:31 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 237BC8FC17; Tue, 29 May 2012 20:38:31 +0000 (UTC) Received: from stevebender.sldomain.com (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q4TKcU5J020230 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 29 May 2012 14:38:30 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: "Justin T. Gibbs" In-Reply-To: <20120529184207.GB11065@yoshi.hcs.int> Date: Tue, 29 May 2012 14:38:24 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> To: Chuck Silvers X-Mailer: Apple Mail (2.1278) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Tue, 29 May 2012 14:38:30 -0600 (MDT) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 20:38:31 -0000 On May 29, 2012, at 12:42 PM, Chuck Silvers wrote: > On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: >> On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: >>> Hi pawel / xin / martin / all >>>=20 >>> Hope you are all doing well. >>>=20 >>> It has been quite a while since we exchanged mail. At HighCloud ( >>> http://www.highcloudsecurity.com) , we released our first VM Centric >>> Security product (based on freebsd and ZFS) back in October 2011. We = are >>> gaining traction with customers and are continuing to enhance the = features, >>> performance and stability of our product (just released 1.1.5 a = couple of >>> weeks ago). >>>=20 >>> I'd like to introduce Chuck Silvers to this group. He is a former = colleague >>> from Veritas / Symantec's file systems team and now works with me = at High >>> Cloud Security. Chuck is also a long time netbsd developer (but = don't hold >>> that against him :-)) >>>=20 >>> Chuck has been looking at ZFS performance and stability issues for = us and >>> wanted to share a couple of interesting results with this group. = I'll let >>> him speak about them himself. >>=20 >> Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! >>=20 >> I added your e-mail to the list. Let me know if you received it. >=20 > hi pawel, >=20 > I got it, thanks. >=20 > we have a number of patches to freebsd that we're using, most of which > modify parts of the system other than ZFS, should I just submit those = as PRs? > or post them to whichever freebsd-* mailing list seems the most = appropriate? > the few that are ZFS-specific I'll send here shortly. >=20 PRs often (unfortunately) get lost. If you can provide an enumeration = of your changes, I can introduce you to a committer that works in that area to = help get the changes into the tree. Longer term, we should figure out how to = make you a committer. :-) -- Justin= From owner-zfs-devel@FreeBSD.ORG Wed May 30 01:00:06 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E4113106566C; Wed, 30 May 2012 01:00:05 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-gh0-f172.google.com (mail-gh0-f172.google.com [209.85.160.172]) by mx1.freebsd.org (Postfix) with ESMTP id 7B2AC8FC0C; Wed, 30 May 2012 01:00:05 +0000 (UTC) Received: by ghbg16 with SMTP id g16so3027477ghb.17 for ; Tue, 29 May 2012 18:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=sD69/mHguj9WkfEcmCD9rzpVWALJeBuZaw4Ah7WXwVU=; b=xWn5NBm1RJqgDRqXp4Vk9o5bXMaeKz6sM7V0qObW5pXW1UV7pmaBU/YPsIX/s36aHO CwHZTX1yMXbpJM8nTDsHrXR9FJ6osDa/KZwuwLjdrDC6CGNPaQARbfliLn6Gs+O0S3pO kP3mHcAZRu5mc06Pz1fbK0GYgff/s8eXxViiKQuYcNIqhjh4ALw4MN7xuef979Bcb0+8 TFfIGNs8HNVyK6nxYdXVW9YPKBZ/TYVxXp5NEFiTqamKW9wje26rWydPwKqt55Y3prCd /DxlqjKTsPmr684Xxbo7iApkyUt6lx5vFQtlNAP+zMkRNGNJk+8+xh6jzsJhWXtU9319 CZzQ== MIME-Version: 1.0 Received: by 10.43.49.3 with SMTP id uy3mr853165icb.2.1338339604274; Tue, 29 May 2012 18:00:04 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Tue, 29 May 2012 18:00:04 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Tue, 29 May 2012 18:00:04 -0700 (PDT) In-Reply-To: <20120529184207.GB11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> Date: Wed, 30 May 2012 09:00:04 +0800 Message-ID: From: Marcelo Araujo To: Chuck Silvers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 01:00:06 -0000 Hey dear Chuck, I'm wondering if you could share all changes made over ZFS. I'd like to give a try in your improvements. BR, - Araujo On May 30, 2012 2:50 AM, "Chuck Silvers" wrote: > On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: > > On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > > > Hi pawel / xin / martin / all > > > > > > Hope you are all doing well. > > > > > > It has been quite a while since we exchanged mail. At HighCloud ( > > > http://www.highcloudsecurity.com) , we released our first VM Centric > > > Security product (based on freebsd and ZFS) back in October 2011. We > are > > > gaining traction with customers and are continuing to enhance the > features, > > > performance and stability of our product (just released 1.1.5 a couple > of > > > weeks ago). > > > > > > I'd like to introduce Chuck Silvers to this group. He is a former > colleague > > > from Veritas / Symantec's file systems team and now works with me at > High > > > Cloud Security. Chuck is also a long time netbsd developer (but don't > hold > > > that against him :-)) > > > > > > Chuck has been looking at ZFS performance and stability issues for us > and > > > wanted to share a couple of interesting results with this group. I'll > let > > > him speak about them himself. > > > > Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! > > > > I added your e-mail to the list. Let me know if you received it. > > hi pawel, > > I got it, thanks. > > we have a number of patches to freebsd that we're using, most of which > modify parts of the system other than ZFS, should I just submit those as > PRs? > or post them to whichever freebsd-* mailing list seems the most > appropriate? > the few that are ZFS-specific I'll send here shortly. > > -Chuck > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Wed May 30 07:06:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 201091065670 for ; Wed, 30 May 2012 07:06:35 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C297D8FC0A for ; Wed, 30 May 2012 07:06:34 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 8B4E7D35; Wed, 30 May 2012 09:06:33 +0200 (CEST) Date: Wed, 30 May 2012 09:04:44 +0200 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20120530070444.GA1372@garage.freebsd.pl> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 07:06:35 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > > we have a number of patches to freebsd that we're using, most of which > > modify parts of the system other than ZFS, should I just submit those a= s PRs? > > or post them to whichever freebsd-* mailing list seems the most appropr= iate? > > the few that are ZFS-specific I'll send here shortly. > >=20 >=20 > PRs often (unfortunately) get lost. If you can provide an enumeration of= your > changes, I can introduce you to a committer that works in that area to he= lp get > the changes into the tree. Longer term, we should figure out how to make= you > a committer. :-) I agree, the best chances to get your changes in is to find active committer in your area and approach him directly or get a commit bit. This mailing list is a good place to start when it comes to ZFS changes, but changes to other parts of the system that are needed to improve ZFS are welcome here too, of course. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --0F1p//8PRICkK4MW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/FxowACgkQForvXbEpPzSJaACff+iqloDFmyLdn3qmrFRCnmdZ tiQAnj8kpbNQtWN07Oq9JyLyKE1A/vuf =gYbD -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 00:11:39 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9AD05106564A; Thu, 31 May 2012 00:11:39 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 728FE8FC08; Thu, 31 May 2012 00:11:39 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 1F345321080; Thu, 31 May 2012 01:11:33 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c7CCLs1WnvZ3; Thu, 31 May 2012 01:11:32 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 3B1CB32107F; Thu, 31 May 2012 01:11:32 +0100 (BST) Date: Wed, 30 May 2012 17:11:30 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120531001130.GC11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:11:39 -0000 On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > PRs often (unfortunately) get lost. If you can provide an enumeration of your > changes, I can introduce you to a committer that works in that area to help get > the changes into the tree. thanks, that will be great. here's our list of patches: - fixes and enhancements to the aesni driver - replace the asm version of aesni_decrypt_cbc() with a C wrapper around aesni_dec() (like aesni_encrypt_cbc() was already a wrapper around aesni_enc()) since using the asm one often crashed mysteriously for us. - allow using the same aesni session from multiple CPUs simultaneously by moving the saved FPU context from the session to the stack. - mark aesni as CRYPTOCAP_F_SYNC since it is synchronous. - enhance the NFS server to support multiple writes to the same file in parallel if the underlying fs supports it. this builds on top of the addition of a "flags" parameter to VFS_FHTOVP() that I backported from -current to our 8.2-based tree. - allow adjusting NKPT via the kernel config file on amd64 too. - have uuidgen ignore ipfw0 when looking for a MAC address. (this only shows up when ipfw is built into the kernel and all the interfaces with real MAC addresses are from modules.) there are a couple minor ZFS ones, I'll send those here separately. some of these are really more workarounds than fixes, but you get the idea. we also have a fix for the open-vm-tools vmxnet driver to support more than 4GB of RAM. I suppose I should push that back to the sourceforge project as well as the freebsd ports tree. we also built several other ports (php5-* and py26-django) without gettext or libiconv (to avoid GPL3). I don't know if these are interesting to anyone else. I also ported the openbsd "vmt" driver to freebsd since that was easier than getting the open-vm-tools vmtoolsd to build without libiconv. I think that's all so far. > Longer term, we should figure out how to make you a committer. :-) I'm happy to feed our changes back through other folks. :-) -Chuck From owner-zfs-devel@FreeBSD.ORG Thu May 31 00:32:36 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B9E1106564A for ; Thu, 31 May 2012 00:32:36 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 52B818FC12 for ; Thu, 31 May 2012 00:32:36 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 00F31110002 for ; Thu, 31 May 2012 01:32:35 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9+GZv9gHI5I6 for ; Thu, 31 May 2012 01:32:35 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 4A68132107F for ; Thu, 31 May 2012 01:32:35 +0100 (BST) Date: Wed, 30 May 2012 17:32:34 -0700 From: Chuck Silvers To: zfs-devel@freebsd.org Message-ID: <20120531003233.GD11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:32:36 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline we only have a few changes to ZFS itself, and now that I look I see that you've found one of them independently (r230256). the other ones are: - improve performance of booting from a ZFS root under ESXi. previously this would sit there for about 5 minutes before even starting to load the kernel. the problem is that the ZFS pool-discovery code opens every possible GPT partition looking for pools, and it rereads the GPT each time, one sector at a time. we changed the GPT code to read the whole GPT in one shot, which reduced the delay to almost nothing. I remember seeing some discussion about a PR on this topic some time back but I don't know if any fix was ever applied and I don't see the PR now. as I recall, the proposal in that discussion was to improve the boot code caching so that it wouldn't reread the GPT at all, which I imagine would work just as well as what we did. (hmm, this isn't actually a change to ZFS either.) - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. patches for both of these are attached. -Chuck --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="patch.biosdisk-gpt" Index: sys/boot/i386/libi386/biosdisk.c =================================================================== RCS file: /home/chs/freebsd/cvs/src/sys/boot/i386/libi386/biosdisk.c,v retrieving revision 1.62.2.2.4.1 diff -u -p -r1.62.2.2.4.1 biosdisk.c --- sys/boot/i386/libi386/biosdisk.c 21 Dec 2010 17:09:25 -0000 1.62.2.2.4.1 +++ sys/boot/i386/libi386/biosdisk.c 6 Jul 2011 20:44:31 -0000 @@ -853,9 +853,10 @@ bd_open_gpt(struct open_disk *od, struct struct gpt_hdr *hdr; struct gpt_ent *ent; struct gpt_part *gp; - int entries_per_sec, error, i, part; + int entries_per_sec, error, i, part, nsec; daddr_t lba, elba; char gpt[BIOSDISK_SECSIZE], tbl[BIOSDISK_SECSIZE]; + char *buf, *tblp; /* * Following calculations attempt to determine the correct value @@ -900,22 +901,31 @@ bd_open_gpt(struct open_disk *od, struct return (EINVAL); } + entries_per_sec = BIOSDISK_SECSIZE / hdr->hdr_entsz; + nsec = hdr->hdr_entries / entries_per_sec; + if (nsec > 128) { + DEBUG("too many GPT table sectors %d", nsec); + return (EINVAL); + } + buf = alloca(nsec * BIOSDISK_SECSIZE); + if (bd_read(od, hdr->hdr_lba_table, nsec, buf)) { + DEBUG("error reading GPT table"); + return (EIO); + } + /* Now walk the partition table to count the number of valid partitions. */ part = 0; - entries_per_sec = BIOSDISK_SECSIZE / hdr->hdr_entsz; + tblp = buf; elba = hdr->hdr_lba_table + hdr->hdr_entries / entries_per_sec; for (lba = hdr->hdr_lba_table; lba < elba; lba++) { - if (bd_read(od, lba, 1, tbl)) { - DEBUG("error reading GPT table"); - return (EIO); - } for (i = 0; i < entries_per_sec; i++) { - ent = (struct gpt_ent *)(tbl + i * hdr->hdr_entsz); + ent = (struct gpt_ent *)(tblp + i * hdr->hdr_entsz); if (uuid_is_nil(&ent->ent_type, NULL) || ent->ent_lba_start == 0 || ent->ent_lba_end < ent->ent_lba_start) continue; part++; } + tblp += BIOSDISK_SECSIZE; } /* Save the important information about all the valid partitions. */ @@ -923,14 +933,10 @@ bd_open_gpt(struct open_disk *od, struct if (part != 0) { od->od_partitions = malloc(part * sizeof(struct gpt_part)); part = 0; + tblp = buf; for (lba = hdr->hdr_lba_table; lba < elba; lba++) { - if (bd_read(od, lba, 1, tbl)) { - DEBUG("error reading GPT table"); - error = EIO; - goto out; - } for (i = 0; i < entries_per_sec; i++) { - ent = (struct gpt_ent *)(tbl + i * hdr->hdr_entsz); + ent = (struct gpt_ent *)(tblp + i * hdr->hdr_entsz); if (uuid_is_nil(&ent->ent_type, NULL) || ent->ent_lba_start == 0 || ent->ent_lba_end < ent->ent_lba_start) @@ -942,6 +948,7 @@ bd_open_gpt(struct open_disk *od, struct od->od_partitions[part].gp_end = ent->ent_lba_end; part++; } + tblp += BIOSDISK_SECSIZE; } } od->od_flags |= BD_GPTOK; --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="patch.zfs-resilver-sysctl" --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c.ORG 2011-10-12 14:27:36.000000000 +0530 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c 2011-10-12 14:38:44.000000000 +0530 @@ -69,6 +69,14 @@ enum ddt_class zfs_scrub_ddt_class_max = DDT_CLASS_DUPLICATE; int dsl_scan_delay_completion = B_FALSE; /* set to delay scan completion */ +SYSCTL_DECL(_vfs_zfs); +TUNABLE_INT("vfs.zfs.zfs_resilver_delay", &zfs_resilver_delay); +SYSCTL_INT(_vfs_zfs, OID_AUTO, zfs_resilver_delay, CTLFLAG_RW, + &zfs_resilver_delay, 0, "Resilver delay"); +TUNABLE_INT("vfs.zfs.zfs_resilver_min_time_ms", &zfs_resilver_min_time_ms); +SYSCTL_INT(_vfs_zfs, OID_AUTO, zfs_resilver_min_time_ms, CTLFLAG_RW, + &zfs_resilver_min_time_ms, 0, "Resilver min time"); + #define DSL_SCAN_IS_SCRUB_RESILVER(scn) \ ((scn)->scn_phys.scn_func == POOL_SCAN_SCRUB || \ (scn)->scn_phys.scn_func == POOL_SCAN_RESILVER) --9jxsPFA5p3P2qPhR-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 06:20:16 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D7BF106564A for ; Thu, 31 May 2012 06:20:16 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2715F8FC17 for ; Thu, 31 May 2012 06:20:15 +0000 (UTC) Received: by yhgm50 with SMTP id m50so485360yhg.13 for ; Wed, 30 May 2012 23:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=gCkKF9+C6sREsyIXgRqnwUEYw+udnfXy1lnNb/JYXcU=; b=Dh5wLEcGvZ+80ZmXBXjAhE03z7qcTVcBXxDs26WecYE6Y/nEnqmjSk3v+SuxCX6vpy 9d7ZkbBpWF2JzZGwbmXP/EaH5ujUgKBdZB7Jwl9GvGkYdVKslXOnyAXBXSJLNUIqqiAa HlLSmyX2oZNmjmzBoda96ImVNoJApBkycWA80ZPG9HITiaperlX4XRJgAzJJIQ8aPWch a9pR1B/uWbKV6XaR2tF9c6lzl7aCM0BIFEXrgGzdY0XtCG/izw4MkZkjTS6rxMeQKUWs P+1wwxzHpQDq6nFPwXKcTytfRN1ZTQ/NkmyRkUX4Y5g8ttAsRSvWVyP0hMcC6lBRjNHL 8IEw== MIME-Version: 1.0 Received: by 10.50.181.232 with SMTP id dz8mr12888881igc.72.1338445215097; Wed, 30 May 2012 23:20:15 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Wed, 30 May 2012 23:20:15 -0700 (PDT) In-Reply-To: <20120531003233.GD11065@yoshi.hcs.int> References: <20120531003233.GD11065@yoshi.hcs.int> Date: Thu, 31 May 2012 14:20:15 +0800 Message-ID: From: Marcelo Araujo To: Chuck Silvers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 06:20:16 -0000 2012/5/31 Chuck Silvers > > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. > > Dear Chuck, The patch above looks very good and useful. Thanks to share it with us, I believe you shall open a PR and let's try to forward to someone with write access on zfs. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu May 31 07:48:04 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AAF41065672 for ; Thu, 31 May 2012 07:48:04 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id B6D6D8FC08 for ; Thu, 31 May 2012 07:48:03 +0000 (UTC) Received: by yenl8 with SMTP id l8so674437yen.13 for ; Thu, 31 May 2012 00:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=gGCHUOT7LIyYWi5fftWw85TsTG0sLrY55JFL0Z0X7ig=; b=q/Wm1oUIU+oo6ofJ6kUF1hOVo5UzpMAD8x0Wbogw1CSlnExv/RJjW+ll9bm3dqUgA0 4cfeRxzGzIk2yIYgYBWkNfCd23xgqzMf6v4sotsf+HnBtthIU1EioNuQWTJD31lnEfR6 Y86344yiy0SPv3c/Ol/e4q7/ZR0wi0Sz5hb4t0w+bG3kXTZoGlPeKEbr5JDzFwtJoSJs 7Hir7+qLw4qIDuZBG6RUblHPYO5ciul+U2FMZGpTduzVrEyMqsm90b2+ILi4QW7GOVJ5 8ZNpBq3kNZm7UbmivgAxofRHE55yuSFnydmGO+tFGb4RxwNpQ7PCM3mrF030in0adHGO xa0A== MIME-Version: 1.0 Received: by 10.50.47.135 with SMTP id d7mr671666ign.66.1338450482439; Thu, 31 May 2012 00:48:02 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Thu, 31 May 2012 00:48:02 -0700 (PDT) Date: Thu, 31 May 2012 15:48:02 +0800 Message-ID: From: Marcelo Araujo To: zfs-devel@freebsd.org Content-Type: multipart/mixed; boundary=14dae93403ad8e26d204c15049cd X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: SPA version 5000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 07:48:04 -0000 --14dae93403ad8e26d204c15049cd Content-Type: text/plain; charset=ISO-8859-1 Hello Martin and all, I'm testing the new patch and it works smoothly, I haven't done a benchmark yet about how long to destroy the pool with async_destroy enabled against not enabled. However, I felt some missing information, as an example on the man page, also the "zpool list -v" wasn't documented. Here attached a very small patch that should be applied after apply the Martin's patch. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org --14dae93403ad8e26d204c15049cd Content-Type: application/octet-stream; name="zfs-features-vs-head-20120531.patch" Content-Disposition: attachment; filename="zfs-features-vs-head-20120531.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h2viuc0s0 LS0tIHpwb29sX21haW4uYwkyMDEyLTA1LTMxIDE1OjMwOjA5LjAwMDAwMDAwMCArMDgwMAorKysg bmV3L3pwb29sX21haW4uYy1uZXcJMjAxMi0wNS0zMSAxNToyODoxNy4wMDAwMDAwMDAgKzA4MDAK QEAgLTIzNSw3ICsyMzUsNyBAQAogCWNhc2UgSEVMUF9MQUJFTENMRUFSOgogCQlyZXR1cm4gKGdl dHRleHQoIlx0bGFiZWxjbGVhciBbLWZdIDx2ZGV2PlxuIikpOwogCWNhc2UgSEVMUF9MSVNUOgot CQlyZXR1cm4gKGdldHRleHQoIlx0bGlzdCBbLUhdIFstbyBwcm9wZXJ0eVssLi4uXV0gIgorCQly ZXR1cm4gKGdldHRleHQoIlx0bGlzdCBbLUh2XSBbLW8gcHJvcGVydHlbLC4uLl1dICIKIAkJICAg ICJbLVQgZHx1XSBbcG9vbF0gLi4uIFtpbnRlcnZhbCBbY291bnRdXVxuIikpOwogCWNhc2UgSEVM UF9PRkZMSU5FOgogCQlyZXR1cm4gKGdldHRleHQoIlx0b2ZmbGluZSBbLXRdIDxwb29sPiA8ZGV2 aWNlPiAuLi5cbiIpKTsKLS0tIHpwb29sLjgJMjAxMi0wNS0yOCAyMDowMjo1MC4wMDAwMDAwMDAg KzA4MDAKKysrIG5ldy96cG9vbC44LW5ldwkyMDEyLTA1LTMxIDE1OjI4OjE3LjAwMDAwMDAwMCAr MDgwMApAQCAtMTM0OCw2ICsxMzQ4LDggQEAKIC5JdCBGbCBICiBTY3JpcHRlZCBtb2RlLiBEbyBu b3QgZGlzcGxheSBoZWFkZXJzLCBhbmQgc2VwYXJhdGUgZmllbGRzIGJ5IGEgc2luZ2xlIHRhYgog aW5zdGVhZCBvZiBhcmJpdHJhcnkgc3BhY2UuCisuSXQgRmwgdgorU2hvdyBtb3JlIGRldGFpbGVk IHBvb2wgaW5mb3JtYXRpb24uIAogLkl0IEZsIG8gQXIgcHJvcGVydHkgTnMgT3AgLCBOcyBBciAu Li4KIENvbW1hLXNlcGFyYXRlZCBsaXN0IG9mIHByb3BlcnRpZXMgdG8gZGlzcGxheS4gU2VlIHRo ZQogLlFxIFN4IFByb3BlcnRpZXMKLS0tIHpwb29sLWZlYXR1cmVzLjUJMjAxMi0wNS0zMSAxNToy OToxMC4wMDAwMDAwMDAgKzA4MDAKKysrIG5ldy96cG9vbC1mZWF0dXJlcy41LW5ldwkyMDEyLTA1 LTMxIDE1OjI4OjE3LjAwMDAwMDAwMCArMDgwMApAQCAtMTU0LDYgKzE1NCwxNiBAQAogYXZhaWxh YmxlIHRocm91Z2ggdGhlCiAuU3kgZnJlZWluZwogcHJvcGVydHkuCisuRWwKKy5FbAorLlNoIEVY QU1QTEVTCisuQmwgLXRhZyAtd2lkdGggMG4KKy5JdCBTeSBFeGFtcGxlIDEgTm8gQWN0aXZhdGlu ZyB0aGUgYXN5bmNfZGVzdHJveSBmZWF0dXJlCisuUHAKK1RoZSBmb2xsb3dpbmcgY29tbWFuZCBh Y3RpdmF0aW5nIHRoZSBhc3luY19kZXN0cm95IGZlYXR1cmUgb24gcG9vbCBjYWxsZWQgdGFuawor LkJkIC1saXRlcmFsIC1vZmZzZXQgMm4KKy5MaSAjIEljIHpwb29sIHNldCBmZWF0dXJlQGFzeW5j X2Rlc3Ryb3k9ZW5hYmxlZCB0YW5rCisuRWQKIC5TaCBTRUUgQUxTTwogLlhyIHpwb29sIDgKIC5T aCBBVVRIT1JTCg== --14dae93403ad8e26d204c15049cd-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 09:13:49 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14083106564A for ; Thu, 31 May 2012 09:13:49 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id B6FF78FC08 for ; Thu, 31 May 2012 09:13:48 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 8EAD9450; Thu, 31 May 2012 11:13:47 +0200 (CEST) Date: Thu, 31 May 2012 11:11:58 +0200 From: Pawel Jakub Dawidek To: Chuck Silvers Message-ID: <20120531091158.GA1401@garage.freebsd.pl> References: <20120531003233.GD11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gKMricLos+KVdGMg" Content-Disposition: inline In-Reply-To: <20120531003233.GD11065@yoshi.hcs.int> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 09:13:49 -0000 --gKMricLos+KVdGMg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 30, 2012 at 05:32:34PM -0700, Chuck Silvers wrote: > we only have a few changes to ZFS itself, and now that I look I see that > you've found one of them independently (r230256). >=20 > the other ones are: >=20 > - improve performance of booting from a ZFS root under ESXi. > previously this would sit there for about 5 minutes before even > starting to load the kernel. the problem is that the ZFS pool-discove= ry > code opens every possible GPT partition looking for pools, and it rere= ads > the GPT each time, one sector at a time. we changed the GPT code to > read the whole GPT in one shot, which reduced the delay to almost noth= ing. > I remember seeing some discussion about a PR on this topic some time b= ack > but I don't know if any fix was ever applied and I don't see the PR no= w. > as I recall, the proposal in that discussion was to improve the boot c= ode > caching so that it wouldn't reread the GPT at all, which I imagine wou= ld > work just as well as what we did. > (hmm, this isn't actually a change to ZFS either.) The problem I remember with this code was that it checked 128 partitions for every single disk in a brute-force fasion in hope that maybe there are no p3-p110 partitions, but maybe there is p111 one. Easy (but not ideal) solution to this was to stop scanning partitions when 3 in a row don't exist. Ideal solution was to teach the code to actually look into partition table and trying only those partitions that really exist. I haven't looked closely at your patch yet, though. > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysct= l. I have a bigger patch for this too: http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch The reason I didn't commit this patch yet is that some of those values can be safely modified after boot can could be made CTLFLAG_RW like maybe the two you convered. If you would like to analyse them and see which are safe to be made RW that would be great. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --gKMricLos+KVdGMg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/HNd4ACgkQForvXbEpPzSiLQCfZDRqw/DbcBItRmu3bAun+sjm tg0AoKAncAc6pftmUA5m+ktFBppNXrF0 =d0e3 -----END PGP SIGNATURE----- --gKMricLos+KVdGMg-- From owner-zfs-devel@FreeBSD.ORG Thu Jun 7 15:27:01 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6C04106566B for ; Thu, 7 Jun 2012 15:27:01 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id A3E7D8FC12 for ; Thu, 7 Jun 2012 15:27:01 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 57F661D97D; Thu, 7 Jun 2012 17:26:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id A8svxPvWXuzc; Thu, 7 Jun 2012 17:26:50 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id E5FAF1D96D; Thu, 7 Jun 2012 17:26:49 +0200 (CEST) Message-ID: <4FD0C839.9020203@FreeBSD.org> Date: Thu, 07 Jun 2012 17:26:49 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 15:27:02 -0000 I have now completed boot support for ZFS features. Please review and comment my patch: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch Boot-only patch: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch I have also an alternate boot patch that adds functions like nvlist_name(), nvlist_value(), but the previous one has a smaller footprint: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch The boot code can be tested with the "zhack" tool, e.g. with: zhack feature add tank org.freebsd:test zhack feature ref -m tank org.freebsd:test Now you are unable to boot. zhack feature ref -d tank org.freebsd:test Now you are able to boot. If there are no quick objections, I will commit this to -HEAD with a 1-month MFC. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Thu Jun 7 19:32:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5B86106564A for ; Thu, 7 Jun 2012 19:32:35 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-gh0-f182.google.com (mail-gh0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 84B768FC17 for ; Thu, 7 Jun 2012 19:32:35 +0000 (UTC) Received: by ghbz22 with SMTP id z22so822723ghb.13 for ; Thu, 07 Jun 2012 12:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=G38sg0dD7dGFUnu2BCKuI3Abg7KFYvGHo5cfUnEReYg=; b=N0axwq5QVYmtApv9JsrZnPv1eoIqzQUWFhBDr54h+oyahnV7TWIFwv23FclKLhu18D Ri9lbQg62iMyXIkj+cm5JFmeri6jQjDlMkVsLgn6CstjR8lxOeplPtplkg9yvEis7OGh dwmzHHmG5lhg6qI9NY3jff51UkBqGNyAU5T4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=G38sg0dD7dGFUnu2BCKuI3Abg7KFYvGHo5cfUnEReYg=; b=Rs97n8W3vd8iSrWrVzu5omkHG4Q3Yv2uq8upSvLHeiitgf0VYh6p4sD+X8Y/p09brs KQ7gevcB+63ns2ATiJLJTxETrfDsjgym7lRL4kB/8lW+066ARyfHqQUiD4FciDZxee/Z UatyMfXSYU1Mlh6ij02oGnRbAIJoS4ckBOIvhOkqMBkKzesv2tsmLCgFN4P2i6Nw0/SM ejP9FZPqSEArKVjFdu2gGGvR7L+7Rhb98K7T0E/RrCSkNlZB7ZglIlnKKdhtqtpuWO6A iZ5d6183ejwfMVfFn7ue1Xjobt9r5cpReT6xtf0Th/nEB4b3+UJm3grihtFkRzxHsztM DsTA== MIME-Version: 1.0 Received: by 10.50.196.232 with SMTP id ip8mr2004947igc.50.1339097554614; Thu, 07 Jun 2012 12:32:34 -0700 (PDT) Received: by 10.231.237.78 with HTTP; Thu, 7 Jun 2012 12:32:34 -0700 (PDT) In-Reply-To: <4FD0C839.9020203@FreeBSD.org> References: <4FD0C839.9020203@FreeBSD.org> Date: Thu, 7 Jun 2012 12:32:34 -0700 Message-ID: From: Matthew Ahrens To: Martin Matuska X-Gm-Message-State: ALoCoQnI6iGOH6YCdl8AxxCbwflnXhcm7QrUNgSMWQe/TrwrhYBQxgd4IppPynWGXZ/rcla4K3U9 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Christopher Siden Subject: Re: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 19:32:35 -0000 Martin, I haven't reviewed the code but I think it's great you're doing this. Thanks! I hope that feature flags will be useful in FreeBSD. Let me or Chris know if you have any issues using it to develop new features. --matt On Thu, Jun 7, 2012 at 8:26 AM, Martin Matuska wrote: > I have now completed boot support for ZFS features. > > Please review and comment my patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch > > Boot-only patch: > > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch > > I have also an alternate boot patch that adds functions like > nvlist_name(), nvlist_value(), but the previous one has a smaller > footprint: > > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch > > The boot code can be tested with the "zhack" tool, e.g. with: > zhack feature add tank org.freebsd:test > zhack feature ref -m tank org.freebsd:test > Now you are unable to boot. > zhack feature ref -d tank org.freebsd:test > Now you are able to boot. > > If there are no quick objections, I will commit this to -HEAD with a > 1-month MFC. > > Cheers, > mm > > -- > Martin Matuska > FreeBSD committer > http://blog.vx.sk > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Sun Jun 10 13:20:41 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30D19106564A; Sun, 10 Jun 2012 13:20:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4D4028FC08; Sun, 10 Jun 2012 13:20:40 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA19349; Sun, 10 Jun 2012 16:20:38 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Sdi4M-000097-Jz; Sun, 10 Jun 2012 16:20:38 +0300 Message-ID: <4FD49F25.7070909@FreeBSD.org> Date: Sun, 10 Jun 2012 16:20:37 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Martin Matuska References: <4FD0C839.9020203@FreeBSD.org> In-Reply-To: <4FD0C839.9020203@FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jun 2012 13:20:41 -0000 on 07/06/2012 18:26 Martin Matuska said the following: > I have now completed boot support for ZFS features. > > Please review and comment my patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch > > Boot-only patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch The boot part looks OK to me. Thank you for working on this! > I have also an alternate boot patch that adds functions like > nvlist_name(), nvlist_value(), but the previous one has a smaller footprint: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch > > The boot code can be tested with the "zhack" tool, e.g. with: > zhack feature add tank org.freebsd:test > zhack feature ref -m tank org.freebsd:test > Now you are unable to boot. > zhack feature ref -d tank org.freebsd:test > Now you are able to boot. > > If there are no quick objections, I will commit this to -HEAD with a > 1-month MFC. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Jun 11 23:30:31 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AAB941065674; Mon, 11 Jun 2012 23:30:31 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 8553D8FC1B; Mon, 11 Jun 2012 23:30:31 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id C0ACC321081; Tue, 12 Jun 2012 00:22:16 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vuB-HrZPu1QK; Tue, 12 Jun 2012 00:22:16 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 2E2F332107E; Tue, 12 Jun 2012 00:22:16 +0100 (BST) Date: Mon, 11 Jun 2012 16:22:14 -0700 From: Chuck Silvers To: Pawel Jakub Dawidek Message-ID: <20120611232214.GM11065@yoshi.hcs.int> References: <20120531003233.GD11065@yoshi.hcs.int> <20120531091158.GA1401@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120531091158.GA1401@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 23:30:31 -0000 On Thu, May 31, 2012 at 11:11:58AM +0200, Pawel Jakub Dawidek wrote: > On Wed, May 30, 2012 at 05:32:34PM -0700, Chuck Silvers wrote: > > we only have a few changes to ZFS itself, and now that I look I see that > > you've found one of them independently (r230256). > > > > the other ones are: > > > > - improve performance of booting from a ZFS root under ESXi. > > previously this would sit there for about 5 minutes before even > > starting to load the kernel. the problem is that the ZFS pool-discovery > > code opens every possible GPT partition looking for pools, and it rereads > > the GPT each time, one sector at a time. we changed the GPT code to > > read the whole GPT in one shot, which reduced the delay to almost nothing. > > I remember seeing some discussion about a PR on this topic some time back > > but I don't know if any fix was ever applied and I don't see the PR now. > > as I recall, the proposal in that discussion was to improve the boot code > > caching so that it wouldn't reread the GPT at all, which I imagine would > > work just as well as what we did. > > (hmm, this isn't actually a change to ZFS either.) > > The problem I remember with this code was that it checked 128 partitions > for every single disk in a brute-force fasion in hope that maybe there > are no p3-p110 partitions, but maybe there is p111 one. > > Easy (but not ideal) solution to this was to stop scanning partitions > when 3 in a row don't exist. Ideal solution was to teach the code to > actually look into partition table and trying only those partitions that > really exist. yea, reading the GPT once and iterating over the valid partitions would be good too. but if there are multiple valid partitions, there's no need to go back and re-read the GPT again for each one. and even when you read the GPT the first time, it would best to read it all at once instead of 1 sector at a time. so really all of these are addressing different aspects of this issue. in practice, fixing any one of these aspects is probably enough to make this not be a problem. > I haven't looked closely at your patch yet, though. did you get a chance to look at it yet? or do you think people are going to want to hold out for a different subset of the complete set of potential changes above? > > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. > > I have a bigger patch for this too: > > http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch > > The reason I didn't commit this patch yet is that some of those values > can be safely modified after boot can could be made CTLFLAG_RW like > maybe the two you convered. > > If you would like to analyse them and see which are safe to be made RW > that would be great. I looked through these and all of them looked safe to me to be changed on the fly. the values just don't seem to used in any kind of persistent fashion. -Chuck From owner-zfs-devel@FreeBSD.ORG Tue Jun 12 03:32:16 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93270106566B; Tue, 12 Jun 2012 03:32:16 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 707C88FC0C; Tue, 12 Jun 2012 03:32:16 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id E0D89110002; Tue, 12 Jun 2012 04:32:15 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8dNKXvA-H-oG; Tue, 12 Jun 2012 04:32:14 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id DB40032107E; Tue, 12 Jun 2012 04:32:14 +0100 (BST) Date: Mon, 11 Jun 2012 20:32:12 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120612033212.GO11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2012 03:32:16 -0000 On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > PRs often (unfortunately) get lost. If you can provide an enumeration of your > changes, I can introduce you to a committer that works in that area to help get > the changes into the tree. ... and some more changes from the last couple of weeks: - support for larger kernel RPC sizes (ie. for the NFS server). this probably needs more work but it would be useful to discuss it with someone. - improve kernel RPC worker thread efficiency by only waking up a thread when there's at least 1 complete RPC waiting in the socket buffer. - enhance the open-vm-tools vmxnet driver to use more rx buffers if the hypervisor supports it, so that it's much less likely to drop packets. it looks like there's still a lot more that could be done to improve this driver, eg. TSO/LRO, checksum offloading, using the large-packet rx ring, etc. it might be better to skip all this and just port the vmxnet3 driver from linux, but that seems likely to be a larger undertaking. I'll guess that rick macklem would be the person to talk to about the NFS/RPC stuff, is that right? -Chuck From owner-zfs-devel@FreeBSD.ORG Tue Jun 12 04:19:30 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2798106566C for ; Tue, 12 Jun 2012 04:19:30 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 9F8078FC0C for ; Tue, 12 Jun 2012 04:19:30 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 7DD20321081; Tue, 12 Jun 2012 05:19:30 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tifha0MtQZPw; Tue, 12 Jun 2012 05:19:29 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id C972532107E; Tue, 12 Jun 2012 05:19:29 +0100 (BST) Date: Mon, 11 Jun 2012 21:19:28 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120612041928.GP11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> <20120612033212.GO11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120612033212.GO11065@yoshi.hcs.int> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2012 04:19:30 -0000 On Mon, Jun 11, 2012 at 08:32:12PM -0700, Chuck Silvers wrote: > On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > > PRs often (unfortunately) get lost. If you can provide an enumeration of your > > changes, I can introduce you to a committer that works in that area to help get > > the changes into the tree. oh, and a couple more: - the dtrace profile and lockstat providers both strip off too many stack frames from the stack() data on amd64, which makes it harder to see what's going on. I'm not sure if this is always the case or if there are other interrupt paths with different stack behaviour. - sys/lockstat.h only defines the lockstat hooks as non-empty if KDTRACE_HOOKS is enabled, but nothing ensures that opt_kdtrace.h was previously included, and most of the .c files that include the various lock headers that include sys/lockstat.h don't previously include opt_kdtrace.h. thus, the inline versions of the various locks that are used in drivers are built into the base kernel mostly miss out on the lockstat hooks. -Chuck From owner-zfs-devel@FreeBSD.ORG Sun Jul 1 15:48:20 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 157F8106564A; Sun, 1 Jul 2012 15:48:20 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 88A978FC0C; Sun, 1 Jul 2012 15:48:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id B7AA827BF7; Sun, 1 Jul 2012 17:48:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id yO_5TzjTbpnx; Sun, 1 Jul 2012 17:48:16 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 4E87627BE4; Sun, 1 Jul 2012 17:48:16 +0200 (CEST) Message-ID: <4FF07140.9020401@FreeBSD.org> Date: Sun, 01 Jul 2012 17:48:16 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.4.2 Content-Type: multipart/mixed; boundary="------------090509050607070809000505" Cc: zfs-devel@FreeBSD.org Subject: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 15:48:20 -0000 This is a multi-part message in MIME format. --------------090509050607070809000505 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi, I would like to hear your opinion on the attached patch to add scrub and resilver tunables. This way users can add more priority to scrub and resilver (make it faster) at cost of other I/O etc. On-line version of the patch: http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch The patch adds tuning for all of the dsl_scan.c tunables, as available in illumos. zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to be non-negative, otherwise we trigger a kernel assert in pause(). Other values are used for timer comparsions and should be safe even if negative (resulting behavior equals a value of zero). Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090509050607070809000505 Content-Type: text/plain; charset=windows-1250; name="dsl_scan.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dsl_scan.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZHNsX3NjYW4uYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xh cmlzL3V0cy9jb21tb24vZnMvemZzL2RzbF9zY2FuLmMJKHJldmlzaW9uIDIzNzc0NSkKKysr IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvZHNsX3Nj YW4uYwkod29ya2luZyBjb3B5KQpAQCAtNjgsNiArNjgsMzYgQEAKIGludCB6ZnNfcmVzaWx2 ZXJfbWluX3RpbWVfbXMgPSAzMDAwOyAvKiBtaW4gbWlsbGlzZWNzIHRvIHJlc2lsdmVyIHBl ciB0eGcgKi8KIGJvb2xlYW5fdCB6ZnNfbm9fc2NydWJfaW8gPSBCX0ZBTFNFOyAvKiBzZXQg dG8gZGlzYWJsZSBzY3J1YiBpL28gKi8KIGJvb2xlYW5fdCB6ZnNfbm9fc2NydWJfcHJlZmV0 Y2ggPSBCX0ZBTFNFOyAvKiBzZXQgdG8gZGlzYWJsZSBzcnViIHByZWZldGNoaW5nICovCisK K1NZU0NUTF9ERUNMKF92ZnNfemZzKTsKK1RVTkFCTEVfSU5UKCJ2ZnMuemZzLnRvcF9tYXhp bmZsaWdodCIsICZ6ZnNfdG9wX21heGluZmxpZ2h0KTsKK1NZU0NUTF9JTlQoX3Zmc196ZnMs IE9JRF9BVVRPLCB0b3BfbWF4aW5mbGlnaHQsIENUTEZMQUdfUlcsCisJJnpmc190b3BfbWF4 aW5mbGlnaHQsIDAsICJNYXhpbXVtIEkvT3MgcGVyIHRvcC1sZXZlbCB2ZGV2Iik7CitUVU5B QkxFX0lOVCgidmZzLnpmcy5yZXNpbHZlcl9kZWxheSIsICZ6ZnNfcmVzaWx2ZXJfZGVsYXkp OworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHJlc2lsdmVyX2RlbGF5LCBDVExG TEFHX1JXLAorICAgICZ6ZnNfcmVzaWx2ZXJfZGVsYXksIDAsICJOdW1iZXIgb2YgdGlja3Mg dG8gZGVsYXkgcmVzaWx2ZXIiKTsKK1RVTkFCTEVfSU5UKCJ2ZnMuemZzLnNjcnViX2RlbGF5 IiwgJnpmc19zY3J1Yl9kZWxheSk7CitTWVNDVExfSU5UKF92ZnNfemZzLCBPSURfQVVUTywg c2NydWJfZGVsYXksIENUTEZMQUdfUlcsCisgICAgJnpmc19zY3J1Yl9kZWxheSwgMCwgIk51 bWJlciBvZiB0aWNrcyB0byBkZWxheSBzY3J1YiIpOworVFVOQUJMRV9JTlQoInZmcy56ZnMu c2Nhbl9pZGxlIiwgJnpmc19zY2FuX2lkbGUpOworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lE X0FVVE8sIHNjYW5faWRsZSwgQ1RMRkxBR19SVywKKyAgICAmemZzX3NjYW5faWRsZSwgMCwg IklkbGUgc2NhbiB3aW5kb3cgaW4gY2xvY2sgdGlja3MiKTsKK1RVTkFCTEVfSU5UKCJ2ZnMu emZzLnNjYW5fbWluX3RpbWVfbXMiLCAmemZzX3NjYW5fbWluX3RpbWVfbXMpOworU1lTQ1RM X0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHNjYW5fbWluX3RpbWVfbXMsIENUTEZMQUdfUlcs CisgICAgJnpmc19zY2FuX21pbl90aW1lX21zLCAwLCAiTWluIG1pbGxpc2VjcyB0byBzY3J1 YiBwZXIgdHhnIik7CitUVU5BQkxFX0lOVCgidmZzLnpmcy5mcmVlX21pbl90aW1lX21zIiwg Jnpmc19mcmVlX21pbl90aW1lX21zKTsKK1NZU0NUTF9JTlQoX3Zmc196ZnMsIE9JRF9BVVRP LCBmcmVlX21pbl90aW1lX21zLCBDVExGTEFHX1JXLAorICAgICZ6ZnNfZnJlZV9taW5fdGlt ZV9tcywgMCwgIk1pbiBtaWxsaXNlY3MgdG8gZnJlZSBwZXIgdHhnIik7CitUVU5BQkxFX0lO VCgidmZzLnpmcy5yZXNpbHZlcl9taW5fdGltZV9tcyIsICZ6ZnNfcmVzaWx2ZXJfbWluX3Rp bWVfbXMpOworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHJlc2lsdmVyX21pbl90 aW1lX21zLCBDVExGTEFHX1JXLAorICAgICZ6ZnNfcmVzaWx2ZXJfbWluX3RpbWVfbXMsIDAs ICJNaW4gbWlsbGlzZWNzIHRvIHJlc2lsdmVyIHBlciB0eGciKTsKK1RVTkFCTEVfSU5UKCJ2 ZnMuemZzLm5vX3NjcnViX2lvIiwgJnpmc19ub19zY3J1Yl9pbyk7CitTWVNDVExfSU5UKF92 ZnNfemZzLCBPSURfQVVUTywgbm9fc2NydWJfaW8sIENUTEZMQUdfUlcsCisgICAgJnpmc19u b19zY3J1Yl9pbywgMCwgIkRpc2FibGUgc2NydWIgSS9PIik7CitUVU5BQkxFX0lOVCgidmZz Lnpmcy5ub19zY3J1Yl9wcmVmZXRjaCIsICZ6ZnNfbm9fc2NydWJfcHJlZmV0Y2gpOworU1lT Q1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIG5vX3NjcnViX3ByZWZldGNoLCBDVExGTEFH X1JXLAorICAgICZ6ZnNfbm9fc2NydWJfcHJlZmV0Y2gsIDAsICJEaXNhYmxlIHNjcnViIHBy ZWZldGNoaW5nIik7CisKIGVudW0gZGR0X2NsYXNzIHpmc19zY3J1Yl9kZHRfY2xhc3NfbWF4 ID0gRERUX0NMQVNTX0RVUExJQ0FURTsKIAogI2RlZmluZQlEU0xfU0NBTl9JU19TQ1JVQl9S RVNJTFZFUihzY24pIFwKQEAgLTE3MDksNyArMTczOSw3IEBACiAJCSAqIHRoZW4gdGhyb3R0 bGUgb3VyIHdvcmtsb2FkIHRvIGxpbWl0IHRoZSBpbXBhY3Qgb2YgYSBzY2FuLgogCQkgKi8K IAkJaWYgKGRkaV9nZXRfbGJvbHQ2NCgpIC0gc3BhLT5zcGFfbGFzdF9pbyA8PSB6ZnNfc2Nh bl9pZGxlKQotCQkJZGVsYXkoc2Nhbl9kZWxheSk7CisJCQlkZWxheShNQVgoc2Nhbl9kZWxh eSwwKSk7CiAKIAkJemlvX25vd2FpdCh6aW9fcmVhZChOVUxMLCBzcGEsIGJwLCBkYXRhLCBz aXplLAogCQkgICAgZHNsX3NjYW5fc2NydWJfZG9uZSwgTlVMTCwgemlvX3ByaW9yaXR5LAo= --------------090509050607070809000505-- From owner-zfs-devel@FreeBSD.ORG Sun Jul 1 16:07:08 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D055106564A; Sun, 1 Jul 2012 16:07:08 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 2D7BD8FC0A; Sun, 1 Jul 2012 16:07:08 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BC6A0F42; Sun, 1 Jul 2012 18:07:06 +0200 (CEST) Date: Sun, 1 Jul 2012 18:04:59 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20120701160458.GQ1400@garage.freebsd.pl> References: <4FF07140.9020401@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KD3NH8oGZ7XN2Llp" Content-Disposition: inline In-Reply-To: <4FF07140.9020401@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 16:07:08 -0000 --KD3NH8oGZ7XN2Llp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 01, 2012 at 05:48:16PM +0200, Martin Matuska wrote: > Hi, >=20 > I would like to hear your opinion on the attached patch to add scrub and > resilver tunables. > This way users can add more priority to scrub and resilver (make it > faster) at cost of other I/O etc. >=20 > On-line version of the patch: > http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch >=20 > The patch adds tuning for all of the dsl_scan.c tunables, as available > in illumos. > zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to > be non-negative, otherwise we trigger a kernel assert in pause(). > Other values are used for timer comparsions and should be safe even if > negative (resulting behavior equals a value of zero). I had similar patch for some time now: http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch The only reason I haven't committed it was that I wasn't sure with variables can be safely modified at run-time. If you did the audit and you are sure we can make them RW, then I'm fine with the patch (except for style issues mentioned below). > +SYSCTL_INT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, > + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); Should be four spaces instead of tab. > - delay(scan_delay); > + delay(MAX(scan_delay,0)); Missing space before comma. Although maybe we should make it unsigned and use SYSCTL_UINT()? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --KD3NH8oGZ7XN2Llp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/wdSoACgkQForvXbEpPzQL3QCfdbWFMJ8RwNan+rPTJZpRSNWs O20AoMbaBuZXn5o3dSeJtm9moG/GMd44 =1wgh -----END PGP SIGNATURE----- --KD3NH8oGZ7XN2Llp-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 2 06:48:37 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAA2D1065786; Mon, 2 Jul 2012 06:48:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 4C5608FC0A; Mon, 2 Jul 2012 06:48:37 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id A2CC4287AD; Mon, 2 Jul 2012 08:48:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id GzCoSWtijn0u; Mon, 2 Jul 2012 08:48:34 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 335452879F; Mon, 2 Jul 2012 08:48:34 +0200 (CEST) Message-ID: <4FF14442.5080905@FreeBSD.org> Date: Mon, 02 Jul 2012 08:48:34 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4FF07140.9020401@FreeBSD.org> <20120701160458.GQ1400@garage.freebsd.pl> In-Reply-To: <20120701160458.GQ1400@garage.freebsd.pl> X-Enigmail-Version: 1.4.2 Content-Type: multipart/mixed; boundary="------------000308000506080402090900" Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 06:48:37 -0000 This is a multi-part message in MIME format. --------------000308000506080402090900 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 1.7.2012 18:04, Pawel Jakub Dawidek wrote: > On Sun, Jul 01, 2012 at 05:48:16PM +0200, Martin Matuska wrote: >> Hi, >> >> I would like to hear your opinion on the attached patch to add scrub and >> resilver tunables. >> This way users can add more priority to scrub and resilver (make it >> faster) at cost of other I/O etc. >> >> On-line version of the patch: >> http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch >> >> The patch adds tuning for all of the dsl_scan.c tunables, as available >> in illumos. >> zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to >> be non-negative, otherwise we trigger a kernel assert in pause(). >> Other values are used for timer comparsions and should be safe even if >> negative (resulting behavior equals a value of zero). > I had similar patch for some time now: > > http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch > > The only reason I haven't committed it was that I wasn't sure with > variables can be safely modified at run-time. If you did the audit and > you are sure we can make them RW, then I'm fine with the patch (except > for style issues mentioned below). > >> +SYSCTL_INT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, >> + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); > Should be four spaces instead of tab. > >> - delay(scan_delay); >> + delay(MAX(scan_delay,0)); > Missing space before comma. > > Although maybe we should make it unsigned and use SYSCTL_UINT()? > I updated my patch to use unsigned integers, as negative values make here really no sense. >From my practical tests, zfs_top_maxinflight should never be zero (or negative), as this makes the zpool command hang (scrub runs in terms of bytes/sec). Changing all other variables to any values on run-time is fine. Please check the updated patch: http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000308000506080402090900 Content-Type: text/plain; charset=windows-1250; name="dsl_scan.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dsl_scan.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (revision 237745) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (working copy) @@ -58,16 +58,47 @@ static dsl_syncfunc_t dsl_scan_cancel_sync; static void dsl_scan_sync_state(dsl_scan_t *, dmu_tx_t *tx); -int zfs_top_maxinflight = 32; /* maximum I/Os per top-level */ -int zfs_resilver_delay = 2; /* number of ticks to delay resilver */ -int zfs_scrub_delay = 4; /* number of ticks to delay scrub */ -int zfs_scan_idle = 50; /* idle window in clock ticks */ +unsigned int zfs_top_maxinflight = 32; /* maximum I/Os per top-level */ +unsigned int zfs_resilver_delay = 2; /* number of ticks to delay resilver */ +unsigned int zfs_scrub_delay = 4; /* number of ticks to delay scrub */ +unsigned int zfs_scan_idle = 50; /* idle window in clock ticks */ -int zfs_scan_min_time_ms = 1000; /* min millisecs to scrub per txg */ -int zfs_free_min_time_ms = 1000; /* min millisecs to free per txg */ -int zfs_resilver_min_time_ms = 3000; /* min millisecs to resilver per txg */ +unsigned int zfs_scan_min_time_ms = 1000; /* min millisecs to scrub per txg */ +unsigned int zfs_free_min_time_ms = 1000; /* min millisecs to free per txg */ +unsigned int zfs_resilver_min_time_ms = 3000; /* min millisecs to resilver + per txg */ boolean_t zfs_no_scrub_io = B_FALSE; /* set to disable scrub i/o */ boolean_t zfs_no_scrub_prefetch = B_FALSE; /* set to disable srub prefetching */ + +SYSCTL_DECL(_vfs_zfs); +TUNABLE_INT("vfs.zfs.top_maxinflight", &zfs_top_maxinflight); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); +TUNABLE_INT("vfs.zfs.resilver_delay", &zfs_resilver_delay); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, resilver_delay, CTLFLAG_RW, + &zfs_resilver_delay, 0, "Number of ticks to delay resilver"); +TUNABLE_INT("vfs.zfs.scrub_delay", &zfs_scrub_delay); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scrub_delay, CTLFLAG_RW, + &zfs_scrub_delay, 0, "Number of ticks to delay scrub"); +TUNABLE_INT("vfs.zfs.scan_idle", &zfs_scan_idle); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scan_idle, CTLFLAG_RW, + &zfs_scan_idle, 0, "Idle scan window in clock ticks"); +TUNABLE_INT("vfs.zfs.scan_min_time_ms", &zfs_scan_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scan_min_time_ms, CTLFLAG_RW, + &zfs_scan_min_time_ms, 0, "Min millisecs to scrub per txg"); +TUNABLE_INT("vfs.zfs.free_min_time_ms", &zfs_free_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, free_min_time_ms, CTLFLAG_RW, + &zfs_free_min_time_ms, 0, "Min millisecs to free per txg"); +TUNABLE_INT("vfs.zfs.resilver_min_time_ms", &zfs_resilver_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, resilver_min_time_ms, CTLFLAG_RW, + &zfs_resilver_min_time_ms, 0, "Min millisecs to resilver per txg"); +TUNABLE_INT("vfs.zfs.no_scrub_io", &zfs_no_scrub_io); +SYSCTL_INT(_vfs_zfs, OID_AUTO, no_scrub_io, CTLFLAG_RW, + &zfs_no_scrub_io, 0, "Disable scrub I/O"); +TUNABLE_INT("vfs.zfs.no_scrub_prefetch", &zfs_no_scrub_prefetch); +SYSCTL_INT(_vfs_zfs, OID_AUTO, no_scrub_prefetch, CTLFLAG_RW, + &zfs_no_scrub_prefetch, 0, "Disable scrub prefetching"); + enum ddt_class zfs_scrub_ddt_class_max = DDT_CLASS_DUPLICATE; #define DSL_SCAN_IS_SCRUB_RESILVER(scn) \ @@ -405,7 +436,7 @@ dsl_scan_check_pause(dsl_scan_t *scn, const zbookmark_t *zb) { uint64_t elapsed_nanosecs; - int mintime; + unsigned int mintime; /* we never skip user/group accounting objects */ if (zb && (int64_t)zb->zb_object < 0) @@ -1638,7 +1669,7 @@ boolean_t needs_io; int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; int zio_priority; - int scan_delay = 0; + unsigned int scan_delay = 0; if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) @@ -1695,7 +1726,8 @@ if (needs_io && !zfs_no_scrub_io) { vdev_t *rvd = spa->spa_root_vdev; - uint64_t maxinflight = rvd->vdev_children * zfs_top_maxinflight; + uint64_t maxinflight = rvd->vdev_children * + MAX(zfs_top_maxinflight, 1); void *data = zio_data_buf_alloc(size); mutex_enter(&spa->spa_scrub_lock); @@ -1709,7 +1741,7 @@ * then throttle our workload to limit the impact of a scan. */ if (ddi_get_lbolt64() - spa->spa_last_io <= zfs_scan_idle) - delay(scan_delay); + delay(MAX((int)scan_delay, 0)); zio_nowait(zio_read(NULL, spa, bp, data, size, dsl_scan_scrub_done, NULL, zio_priority, --------------000308000506080402090900-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 2 07:06:25 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E5B8106564A; Mon, 2 Jul 2012 07:06:25 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 43FCE8FC15; Mon, 2 Jul 2012 07:06:25 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 2270310C; Mon, 2 Jul 2012 09:06:24 +0200 (CEST) Date: Mon, 2 Jul 2012 09:04:18 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20120702070418.GA1372@garage.freebsd.pl> References: <4FF07140.9020401@FreeBSD.org> <20120701160458.GQ1400@garage.freebsd.pl> <4FF14442.5080905@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <4FF14442.5080905@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 07:06:25 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 02, 2012 at 08:48:34AM +0200, Martin Matuska wrote: > +SYSCTL_UINT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, > + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); This tab is still not fixed. Other than that the patch looks good. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/xR/IACgkQForvXbEpPzR6jgCfQHgI1ahFJMJ1fi0OzeHyQLkw I8gAn3wxj089Kbn3KDyc/YefrNaStqM4 =Dc6F -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 28 14:26:10 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC99E106566B; Tue, 28 Aug 2012 14:26:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 178FC8FC0C; Tue, 28 Aug 2012 14:26:06 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA22612; Tue, 28 Aug 2012 17:25:53 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <503CD4F1.6060001@FreeBSD.org> Date: Tue, 28 Aug 2012 17:25:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120730 Thunderbird/14.0 MIME-Version: 1.0 To: Trent Nelson References: <20120824011517.GJ42732@snakebite.org> In-Reply-To: <20120824011517.GJ42732@snakebite.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 28 Aug 2012 15:18:45 +0000 Cc: freebsd-fs@FreeBSD.org Subject: Re: chmod -h 000x against symlink has bizarre results on ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:26:11 -0000 on 24/08/2012 04:15 Trent Nelson said the following: > Hi folks, > > I recently set up a FreeBSD build slave for the Python project, > and noticed some symlink tests were failing in a very strange way > (http://bugs.python.org/issue15748). > > When chmod -h 000x is done against a file/link of length less than > 24, the target seems to get padded out to 24 with 0s. If it's > longer than 24, it'll get truncated. 'x' can be 7, 6, 5 or 4 and > the behaviour is the same. > > Here's the output from the attached test_readlink.sh, also available > at http://bugs.python.org/file26979/test_readlink.sh: > > % ./test_readlink.sh > > ****** TEST 1: link/target length less than 24 ****** > before chmod -h 0007: > -rw-r----- /tmp/lt24 > lrwxr-x--- /tmp/lt24.lnk->/tmp/lt24 > python os.readlink(/tmp/lt24.lnk): > '/tmp/lt24' > after chmod -h 0007: > -rw-r----- /tmp/lt24 > l------rwx /tmp/lt24.lnk->/tmp/lt24 > python os.readlink(/tmp/lt24.lnk): > '/tmp/lt24\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > target is padded out with NULLs to 24 > > > > > ****** TEST 2: link/target length longer than 24 ****** > before chmod -h 0007: > -rw-r----- /tmp/definitelywaylongerthantwentyfour > lrwxr-x--- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylongerthantwentyfour > python os.readlink(/tmp/definitelywaylongerthantwentyfour.lnk): > '/tmp/definitelywaylongerthantwentyfour' > after chmod -h 0007: > -rw-r----- /tmp/definitelywaylongerthantwentyfour > l------rwx > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > python os.readlink(/tmp/definitelywaylongerthantwentyfour.lnk): > '/tmp/definitelywaylonger' > ^^^^^^^^^^^^^^^^^^^^^^^^ > target gets truncated to 24 > > > > ****** Other modes... ****** > after chmod -h 0006: > l------rw- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0005: > l------r-x > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0004: > l------r-- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0000: > l--------- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylongerthantwentyfour > > > This only happens on ZFS. I'm on v28, don't have any v15s lying > around. > > I'm perplexed. Can others reproduce it? > I can reproduce this problem I can also provide some additional bits of information using a modified version of zdb: $ ln -fs definitelywaylongerthantwentyfour definitelywaylongerthantwentyfour.lnk $ stat -s definitelywaylongerthantwentyfour.lnk st_dev=3895460379 st_ino=27165 st_mode=0120755 st_nlink=1 st_uid=0 st_gid=0 st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 st_ctime=1346161009 st_birthtime=1346161009 st_blksize=131072 st_blocks=1 st_flags=0 $ zdb -ddddddd tank/tmp 27165 Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp DVA[0]=<0:5c5375e000:200> DVA[1]=<0:4c1a80ce00:200> [L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P birth=70882769L/70882769P fill=24910 cksum=1c72e8f065:89bbdf9d575:1732432c541ff:2d672d98b0ff66 Object lvl iblk dblk dsize lsize %full type 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) (Z=inherit) 209 bonus System attributes dnode flags: USERUSED_ACCOUNTED dnode maxblkid: 0 path /definitelywaylongerthantwentyfour.lnk uid 0 gid 0 atime Tue Aug 28 16:36:49 2012 mtime Tue Aug 28 16:36:49 2012 ctime Tue Aug 28 16:36:49 2012 crtime Tue Aug 28 16:36:49 2012 gen 70882769 mode 120755 size 33 parent 3 links 1 pflags 40800000104 symlink definitelywaylongerthantwentyfour symlink size 33 Indirect blocks: $ chmod -h 0007 definitelywaylongerthantwentyfour.lnk $ stat -s definitelywaylongerthantwentyfour.lnk st_dev=3895460379 st_ino=27165 st_mode=0120007 st_nlink=1 st_uid=0 st_gid=0 st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 st_ctime=1346161227 st_birthtime=1346161227 st_blksize=131072 st_blocks=1 st_flags=0 $ zdb -ddddddd tank/tmp 27165 Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp DVA[0]=<0:5c556b4400:200> DVA[1]=<0:4c1a989600:200> [L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P birth=70882812L/70882812P fill=24910 cksum=170e778d58:737e87307d3:140a45f4106a6:283187f7da9de7 Object lvl iblk dblk dsize lsize %full type 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) (Z=inherit) 216 bonus System attributes dnode flags: USERUSED_ACCOUNTED dnode maxblkid: 0 path /definitelywaylongerthantwentyfour.lnk uid 0 gid 0 atime Tue Aug 28 16:36:49 2012 mtime Tue Aug 28 16:36:49 2012 ctime Tue Aug 28 16:40:27 2012 crtime Tue Aug 28 16:36:49 2012 gen 70882769 mode 120007 size 33 parent 3 links 1 pflags 40800000004 symlink definitelywaylongerthant symlink size 24 Indirect blocks: Note how the file/object size remains 33, but size of ZPL_SYMLINK attribute is changed to 24. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Aug 28 15:01:41 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06FCB106566C; Tue, 28 Aug 2012 15:01:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1724A8FC14; Tue, 28 Aug 2012 15:01:39 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA23086; Tue, 28 Aug 2012 18:01:34 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <503CDD4E.6050902@FreeBSD.org> Date: Tue, 28 Aug 2012 18:01:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120730 Thunderbird/14.0 MIME-Version: 1.0 To: Trent Nelson References: <20120824011517.GJ42732@snakebite.org> <503CD4F1.6060001@FreeBSD.org> In-Reply-To: <503CD4F1.6060001@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 28 Aug 2012 15:52:37 +0000 Cc: freebsd-fs@FreeBSD.org Subject: Re: chmod -h 000x against symlink has bizarre results on ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 15:01:41 -0000 on 28/08/2012 17:25 Andriy Gapon said the following: [snip] > I can reproduce this problem > I can also provide some additional bits of information using a modified version of > zdb: > > $ ln -fs definitelywaylongerthantwentyfour definitelywaylongerthantwentyfour.lnk > $ stat -s definitelywaylongerthantwentyfour.lnk > st_dev=3895460379 st_ino=27165 st_mode=0120755 st_nlink=1 st_uid=0 st_gid=0 > st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 > st_ctime=1346161009 st_birthtime=1346161009 st_blksize=131072 st_blocks=1 st_flags=0 > $ zdb -ddddddd tank/tmp 27165 > Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp > DVA[0]=<0:5c5375e000:200> DVA[1]=<0:4c1a80ce00:200> [L0 DMU objset] fletcher4 lzjb > LE contiguous unique double size=800L/200P birth=70882769L/70882769P fill=24910 > cksum=1c72e8f065:89bbdf9d575:1732432c541ff:2d672d98b0ff66 > > Object lvl iblk dblk dsize lsize %full type > 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) > (Z=inherit) > 209 bonus System attributes > dnode flags: USERUSED_ACCOUNTED > dnode maxblkid: 0 > path /definitelywaylongerthantwentyfour.lnk > uid 0 > gid 0 > atime Tue Aug 28 16:36:49 2012 > mtime Tue Aug 28 16:36:49 2012 > ctime Tue Aug 28 16:36:49 2012 > crtime Tue Aug 28 16:36:49 2012 > gen 70882769 > mode 120755 > size 33 > parent 3 > links 1 > pflags 40800000104 > symlink definitelywaylongerthantwentyfour > symlink size 33 > Indirect blocks: > > $ chmod -h 0007 definitelywaylongerthantwentyfour.lnk > $ stat -s definitelywaylongerthantwentyfour.lnk > st_dev=3895460379 st_ino=27165 st_mode=0120007 st_nlink=1 st_uid=0 st_gid=0 > st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 > st_ctime=1346161227 st_birthtime=1346161227 st_blksize=131072 st_blocks=1 st_flags=0 > $ zdb -ddddddd tank/tmp 27165 > Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp > DVA[0]=<0:5c556b4400:200> DVA[1]=<0:4c1a989600:200> [L0 DMU objset] fletcher4 lzjb > LE contiguous unique double size=800L/200P birth=70882812L/70882812P fill=24910 > cksum=170e778d58:737e87307d3:140a45f4106a6:283187f7da9de7 > > Object lvl iblk dblk dsize lsize %full type > 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) > (Z=inherit) > 216 bonus System attributes > dnode flags: USERUSED_ACCOUNTED > dnode maxblkid: 0 > path /definitelywaylongerthantwentyfour.lnk > uid 0 > gid 0 > atime Tue Aug 28 16:36:49 2012 > mtime Tue Aug 28 16:36:49 2012 > ctime Tue Aug 28 16:40:27 2012 > crtime Tue Aug 28 16:36:49 2012 > gen 70882769 > mode 120007 > size 33 > parent 3 > links 1 > pflags 40800000004 > symlink definitelywaylongerthant > symlink size 24 > Indirect blocks: > > Note how the file/object size remains 33, but size of ZPL_SYMLINK attribute is > changed to 24. > Will you be able to test the following patch? Preferably on a temporary test pool - I don't want to risk your data. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c index 69374fb..7f61517 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c @@ -1695,6 +1695,7 @@ sa_modify_attrs(sa_handle_t *hdl, sa_attr_type_t newattr, ASSERT(action == SA_REPLACE); SA_ADD_BULK_ATTR(attr_desc, j, attr, locator, datastart, buflen); + length_idx++; } else { length = SA_REGISTERED_LEN(sa, attr); if (length == 0) { -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Sep 18 18:43:20 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 54F9D106566B for ; Tue, 18 Sep 2012 18:43:20 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 266928FC0A for ; Tue, 18 Sep 2012 18:43:19 +0000 (UTC) Received: from [192.168.6.100] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q8IIhIbp038342 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 18 Sep 2012 12:43:18 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) From: "Justin T. Gibbs" Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 18 Sep 2012 12:43:12 -0600 Message-Id: To: zfs-devel@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Tue, 18 Sep 2012 12:43:18 -0600 (MDT) Cc: Will Andrews Subject: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 18:43:20 -0000 Hi, Some time back, the folks at HighCloud Security ported a bunch of the ZFS tests from the Solaris test suite to FreeBSD. At Spectra Logic, we have continued the porting effort to enable more test cases, and now runs these tests in our nightly continuous integration system. Will and I are now starting the long process of extracting and pushing back the many changes Spectra has made to ZFS. This will be a lengthy process. However, the test suite is nicely contained and does not rely on any of our other kernel or userland changes. Is there anyone on the list who would be interested in taking the test suite and working with us to get it upstreamed into FreeBSD? Otherwise I fear it will be several months before Will or I can get to it on our own. Thanks, Justin From owner-zfs-devel@FreeBSD.ORG Wed Sep 19 03:01:44 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 893201065670; Wed, 19 Sep 2012 03:01:44 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id E91738FC14; Wed, 19 Sep 2012 03:01:40 +0000 (UTC) Received: by qcsg15 with SMTP id g15so544422qcs.13 for ; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=aE6FFwHp/KjIzMKDjkDkqab2ySVqYpEPZAmiVHit2vY=; b=ulUdh+QL4MJr5upacIGeMIjAEpSwAfCM/xOe8bpRLzW7Ho2za30NShYVr2N+ZaMlk9 UV29pIr12tL9LNyT+3/GBEA2RePvlZFNFq6NupGfPzSm/QQWkBOdAxKblYW3h5lHURt2 9ROVbQUp5EmeezSZCOvKtUMFUQmH+MDoFUr1l/GYG88padY1yZMHGTLh7vfqW4vzPW4K tZLKkOAZ98epBpNZg/GPubu91ebQNRSKhJa4sbgK9nzpR8xzPgonYn6arJNl+6CSycry swWl6yJbN5AjQfXcc2EFclajfoQXoJ01tdpiIpZuHX3fXa0X5ksoHhz5n7Uile7POg+u ZIjg== MIME-Version: 1.0 Received: by 10.224.60.17 with SMTP id n17mr4324381qah.20.1348023699742; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Sep 2012 11:01:39 +0800 Message-ID: From: Marcelo Araujo To: "Justin T. Gibbs" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2012 03:01:44 -0000 Hello Justin, Yes, I have interesting, because currently I'm working in a project that depend directly of ZFS as well as we need a good test suite. I have machines as well as I can dedicate some hours of my night to help uppstreamed the test suite into FreeBSD. Best Regards, - Araujo 2012/9/19 Justin T. Gibbs > Hi, > > Some time back, the folks at HighCloud Security ported a bunch of > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > Logic, we have continued the porting effort to enable more test > cases, and now runs these tests in our nightly continuous integration > system. > > Will and I are now starting the long process of extracting and > pushing back the many changes Spectra has made to ZFS. This will > be a lengthy process. However, the test suite is nicely contained > and does not rely on any of our other kernel or userland changes. > Is there anyone on the list who would be interested in taking the > test suite and working with us to get it upstreamed into FreeBSD? > Otherwise I fear it will be several months before Will or I can get > to it on our own. > > Thanks, > Justin > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu Sep 20 16:46:32 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 735A4106564A for ; Thu, 20 Sep 2012 16:46:32 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 41E698FC16 for ; Thu, 20 Sep 2012 16:46:32 +0000 (UTC) Received: from [192.168.6.120] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q8KGkVPr052277 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 20 Sep 2012 10:46:31 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) From: "Justin T. Gibbs" In-Reply-To: Date: Thu, 20 Sep 2012 10:46:26 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: zfs-devel@FreeBSD.org X-Mailer: Apple Mail (2.1498) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Thu, 20 Sep 2012 10:46:31 -0600 (MDT) Cc: Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 16:46:32 -0000 On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: > Hi, >=20 > Some time back, the folks at HighCloud Security ported a bunch of > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > Logic, we have continued the porting effort to enable more test > cases, and now runs these tests in our nightly continuous integration > system. I've published a snapshot of our work here: = http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz cddl/tools/regression/stf.README has instructions on how to build and = run the tests. cddl/tools/regression/stf.changes is a revision log (with diffs) of what HighCloud and Spectra havedone to the code. In briefly re-reviewing the changes, I believe that David Xu's work to implement process shared mutexes means change set 464984 should be redone. I haven't verified this though. -- Justin From owner-zfs-devel@FreeBSD.ORG Fri Sep 21 02:30:08 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96524106564A; Fri, 21 Sep 2012 02:30:08 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 347688FC12; Fri, 21 Sep 2012 02:30:08 +0000 (UTC) Received: by qady23 with SMTP id y23so1021759qad.13 for ; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=g/9rU9LnxFntc+MlNp1agMBytnCe04NGbNbBAPchKbU=; b=FcUbxvbFEFDkhdPPYDo5tux4Pc+vgor5ioQl3Zm7gm26S8/WzfzjEwUuaxRdVBzxSK IR/H80Lw+6WcX3D69ulS/mDsJE9A//XlUTE0zaV+lBbpPfpBNzjZ5H5iS0UU2xqwd79F nEgKxBxLS7U77BX3goF4rjnBiml0iSdcpZZKdekxFfy30Q6Cis1wlZscX9O/sfdRuueP x0t3rRrCvsvaRRvqt7uezgQB2kf68H2K1O+OBllQgQMpBtg8w+Fp5iJvf+pjsItGeWOp 5t6gZ+LQ7BVqJpAwCfKFmrDyhRkRzmIi6BpPuMuBjnPoaCSKU5jASnZIrNnIItPJQTJJ 9poA== MIME-Version: 1.0 Received: by 10.224.71.74 with SMTP id g10mr9322254qaj.34.1348194607387; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Sep 2012 10:30:07 +0800 Message-ID: From: Marcelo Araujo To: "Justin T. Gibbs" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 02:30:08 -0000 Justin, Thank you so much to share it, I'm gonna take a close look and make some plan to work on it. Also, would be interesting know, whom is working and in which part, so we could avoid dual effort on the same thing. I'm gonna build a machine today to test it. Best Regards, - Araujo 2012/9/21 Justin T. Gibbs > On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: > > > Hi, > > > > Some time back, the folks at HighCloud Security ported a bunch of > > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > > Logic, we have continued the porting effort to enable more test > > cases, and now runs these tests in our nightly continuous integration > > system. > > I've published a snapshot of our work here: > http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz > > cddl/tools/regression/stf.README has instructions on how to build and run > the tests. > > cddl/tools/regression/stf.changes is a revision log (with diffs) of what > HighCloud and Spectra havedone to the code. > > In briefly re-reviewing the changes, I believe that David Xu's work > to implement process shared mutexes means change set 464984 should > be redone. I haven't verified this though. > > -- > Justin > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Fri Sep 21 17:02:03 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 20D02106567E; Fri, 21 Sep 2012 17:02:03 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id A2DAF8FC0A; Fri, 21 Sep 2012 17:02:02 +0000 (UTC) Received: by qady23 with SMTP id y23so1584092qad.13 for ; Fri, 21 Sep 2012 10:01:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=KpUWCHYh/wRR4n/fuT8rFJcelbLwLaOSkDjq8jerD5g=; b=NCRH6RlTxgL1+UxNs4D36YnaXAiF836cqqNb6KD9BAFu/CBUehw565cHBggq4doaid RHQghqvZerxoX0F/PcjuG5G/RlicEGZt5H4hq3xPbbifY4YqSkd7ymmKDAWPuz6oYGYY +vDroCdjEr12vPdW/ItQ34eOWbUqTsmxCN2ReEfA6uI9jCuT45L+qXL4aN7pUmhczIxq owsOLfc2qQiEH5D0K6ycqOMssICm3aEPPW5IlGl8pIlLxEfsP7N/qUHeuLAPZjGdrNpD TY8+7xk3bY8I2lZAO/Musiq+vrli5br44eor/UoARNvW21tWwpM4dt1SSwLVR6SpzTBG qoxg== MIME-Version: 1.0 Received: by 10.224.39.195 with SMTP id h3mr13702631qae.39.1348246916018; Fri, 21 Sep 2012 10:01:56 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.49.81.193 with HTTP; Fri, 21 Sep 2012 10:01:55 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Sep 2012 10:01:55 -0700 X-Google-Sender-Auth: Kg9XiNjkI46wQqAzfldCX7j3NyU Message-ID: From: Matt Olander To: araujo@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org, Will Andrews , larry Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 17:02:03 -0000 On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo wrote: > Thank you so much to share it, I'm gonna take a close look and make some > plan to work on it. > Also, would be interesting know, whom is working and in which part, so we > could avoid dual effort on the same thing. > > I'm gonna build a machine today to test it. Nice job guys! Araujo, I've cc'd Larry at iXsystems. He's already setting up to take a look so maybe you guys can coordinate ;) Cheers, -matt > 2012/9/21 Justin T. Gibbs > >> On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: >> >> > Hi, >> > >> > Some time back, the folks at HighCloud Security ported a bunch of >> > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra >> > Logic, we have continued the porting effort to enable more test >> > cases, and now runs these tests in our nightly continuous integration >> > system. >> >> I've published a snapshot of our work here: >> http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz >> >> cddl/tools/regression/stf.README has instructions on how to build and run >> the tests. >> >> cddl/tools/regression/stf.changes is a revision log (with diffs) of what >> HighCloud and Spectra havedone to the code. >> >> In briefly re-reviewing the changes, I believe that David Xu's work >> to implement process shared mutexes means change set 464984 should >> be redone. I haven't verified this though. >> >> -- >> Justin >> >> >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> > > > > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 17:49:05 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 362611065688; Sat, 22 Sep 2012 17:49:05 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 856448FC08; Sat, 22 Sep 2012 17:49:04 +0000 (UTC) Received: by qcsg15 with SMTP id g15so411983qcs.13 for ; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=eTADDmFEtr/Q/A4g1mBoZIHmFwLhhNkAS+iB4Nl1XkU=; b=ElkWpK1QM9zCgUByLo5b0ShZ0c6LgkIanDrsFYvIBdlIlNF4P3UzcYGf4gt5sSmv/b 9db1Ncz0frWvyw3nKDYx67253KFnpaeo1f9Cu6zeu2AawOuU/PQ0B+SOjfDtKZ+XDa89 SA72gqjU2ukI3p+mw9wTQr9UCE3AsaWAxpYcDOcVGE9vySw31Vv5IyPaOi25XagA4caA zhQSlEjj3vfywC9WN8NUCDYKRBiJ6Vv085sX3C4PYKvf4MffjExMGuIbC7QHBcmie6tt onP7g1lwqntZHghGEG/UlHgWAxGZOtyjmsZLhuVBBbHOsYj3XeDwSN2eQWasKylZXwHq Vpsw== MIME-Version: 1.0 Received: by 10.224.177.77 with SMTP id bh13mr2569298qab.56.1348336138172; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) In-Reply-To: References: Date: Sun, 23 Sep 2012 01:48:58 +0800 Message-ID: From: Marcelo Araujo To: matt@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews , larry Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 17:49:05 -0000 2012/9/22 Matt Olander > On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo > wrote: > > Thank you so much to share it, I'm gonna take a close look and make some > > plan to work on it. > > Also, would be interesting know, whom is working and in which part, so we > > could avoid dual effort on the same thing. > > > > I'm gonna build a machine today to test it. > > Nice job guys! > > Araujo, I've cc'd Larry at iXsystems. He's already setting up to take > a look so maybe you guys can coordinate ;) > > Cheers, > -matt > > Dear Matt and Larry, Would be great if we could be in touch and have some coordination to work together and bring it up on FreeBSD. So, let me know your plan guys, and let's sync our tasks. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 16:28:09 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5194A106564A; Sat, 22 Sep 2012 16:28:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1B2998FC08; Sat, 22 Sep 2012 16:28:07 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07410; Sat, 22 Sep 2012 19:28:06 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFSYo-000NTB-7M; Sat, 22 Sep 2012 19:28:06 +0300 Message-ID: <505DE715.8020806@FreeBSD.org> Date: Sat, 22 Sep 2012 19:28:05 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 17:51:28 +0000 Cc: Subject: zfs: allow to mount root from a pool not in zpool.cache X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 16:28:09 -0000 Currently FreeBSD ZFS kernel code doesn't allow to mount root filesystem on a pool that is not listed in zpool.cache as only pools from the cache are known to ZFS at that time. This patch is an attempt to improve the behavior: http://people.freebsd.org/~avg/spa_import_rootpool.diff This could be useful when importing pools that were exported from other systems. There is a tunable vfs.zfs.rootpool.prefer_cached_config which is set to 1 by default. 1 means just use a cached pool config if it's found in the cache, 0 means to re-probe disks and read supposedly latest/actual config in any case. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 16:49:38 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C056106564A; Sat, 22 Sep 2012 16:49:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 587368FC08; Sat, 22 Sep 2012 16:49:36 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07478; Sat, 22 Sep 2012 19:49:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFStb-000NTq-6o; Sat, 22 Sep 2012 19:49:35 +0300 Message-ID: <505DEC1C.4000305@FreeBSD.org> Date: Sat, 22 Sep 2012 19:49:32 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 18:08:17 +0000 Cc: Subject: znextboot: nextboot-like tool for zfs at zfsboot level X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 16:49:38 -0000 Please find here a patchset that implement znextboot, a nextboot-like tool for zfs at zfsboot level: http://people.freebsd.org/~avg/znextboot.diff Theory of operation. zfsboot, through loader, exports to kernel environment the GUIDs of the very first pool it found ("primary pool") and the very first leaf vdev of that pool ("primary vdev"). Note that the primary pool is not necessarily a boot pool or a root pool, since a user can switch between pools and filesystems at various stages: zfsboot, zfsloader, rootfs specification. znextboot is a new tool that simply passes zfsboot/boot2 options to kernel ZFS via ioctl. Kernel ZFS writes the options as a NUL terminated ASCII string to the Pad2 area of the primary vdev of the primary pool. The Pad2 area has been known as "Boot Block Header" before. Its use was never formalized. Peviously it used to contain a special header (with zero useful information), now ZFS just zeroes it out. So, upon reboot zfsboot reads options from that area and zeros the area. The tool is intended for remote management of systems that use approaches similar to "Boot Environments". It is implemented at zfsboot level as opposed to loader level, because it was easier. My skills weren't sufficient to integrate the ZFS logic with loader's nextboot logic implemented in Forth. Some problematic areas in the current patchset: - I used just the next number for the nextboot ioctl. This will result in conflict when a new ioctl is added upstream. We need to think about reserving a range for OS-specific ioctls. - znextboot userland utility currently lacks any documentation. - znextboot lacks any sanity checking / validation for arguments that are passed to it. - probably more... -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 17:13:11 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 27BD4106566B; Sat, 22 Sep 2012 17:13:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E32BB8FC17; Sat, 22 Sep 2012 17:13:09 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA07567; Sat, 22 Sep 2012 20:13:08 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFTGN-000NUn-W4; Sat, 22 Sep 2012 20:13:08 +0300 Message-ID: <505DF1A3.1020809@FreeBSD.org> Date: Sat, 22 Sep 2012 20:13:07 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 18:08:27 +0000 Cc: freebsd-geom@FreeBSD.org Subject: zfs zvol: set geom mediasize right at creation time X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 17:13:11 -0000 Please review the following patch. In addition to what the description says I almost by accident sneaked another change into the patch. It's setting of stripesize to volblocksize. I think that the change should make sense, but it is really a different change. A side note: setting sectorsize to volblocksize seemed like an overkill and it would certainly mess the existing zvols in use. Maybe there should be another property like reportedblocksize or something. commit 1585e6cfb602c2a2647b9f802445bb174bc430a4 Author: Andriy Gapon Date: Wed Sep 19 20:49:28 2012 +0300 zvol: set mediasize in geom provider right upon its creation ... instead of deferring the action until first open. Unlike upstream this has no benefit on FreeBSD. We know that as soon as the provider is created it is going to be tasted and thus opened. Initial mediasize of zero causes tasting failure and subsequent retasting because of the size change. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c index d47d270..6e9e7a3 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c @@ -475,6 +475,7 @@ zvol_create_minor(const char *name) zvol_state_t *zv; objset_t *os; dmu_object_info_t doi; + uint64_t volblocksize, volsize; int error; ZFS_LOG(1, "Creating ZVOL %s...", name); @@ -535,9 +536,20 @@ zvol_create_minor(const char *name) zv = zs->zss_data = kmem_zalloc(sizeof (zvol_state_t), KM_SLEEP); #else /* !sun */ + error = zap_lookup(os, ZVOL_ZAP_OBJ, "size", 8, 1, &volsize); + if (error) { + ASSERT(error == 0); + dmu_objset_disown(os, zvol_tag); + mutex_exit(&spa_namespace_lock); + return (error); + } + DROP_GIANT(); g_topology_lock(); zv = zvol_geom_create(name); + zv->zv_volsize = volsize; + zv->zv_provider->mediasize = zv->zv_volsize; + #endif /* !sun */ (void) strlcpy(zv->zv_name, name, MAXPATHLEN); @@ -554,6 +566,7 @@ zvol_create_minor(const char *name) error = dmu_object_info(os, ZVOL_OBJ, &doi); ASSERT(error == 0); zv->zv_volblocksize = doi.doi_data_block_size; + zv->zv_provider->stripesize = zv->zv_volblocksize; if (spa_writeable(dmu_objset_spa(os))) { if (zil_replay_disable) -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 16:17:47 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58443106566C; Mon, 24 Sep 2012 16:17:47 +0000 (UTC) (envelope-from larry@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 202AC8FC20; Mon, 24 Sep 2012 16:17:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.iXsystems.com (Postfix) with ESMTP id 63227524; Mon, 24 Sep 2012 09:17:46 -0700 (PDT) Received: from mail.iXsystems.com ([127.0.0.1]) by localhost (mail.ixsystems.com [127.0.0.1]) (maiad, port 10024) with ESMTP id 37921-02; Mon, 24 Sep 2012 09:17:45 -0700 (PDT) Received: from [10.2.3.72] (kruse-72.3.ixsystems.com [10.2.3.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 10D0951E; Mon, 24 Sep 2012 09:17:45 -0700 (PDT) Message-ID: <506087A8.4000001@ixsystems.com> Date: Mon, 24 Sep 2012 09:17:44 -0700 From: Larry Maloney Organization: iX Systems User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: In-Reply-To: Content-Type: multipart/mixed; boundary="------------060605070709090805030809" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: larry@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 16:17:47 -0000 This is a multi-part message in MIME format. --------------060605070709090805030809 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sounds great! We should schedule a meeting, what is your schedule this week? Larry On 09/22/2012 10:48, Marcelo Araujo wrote: > > > 2012/9/22 Matt Olander > > > On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo > > wrote: > > Thank you so much to share it, I'm gonna take a close look and > make some > > plan to work on it. > > Also, would be interesting know, whom is working and in which > part, so we > > could avoid dual effort on the same thing. > > > > I'm gonna build a machine today to test it. > > Nice job guys! > > Araujo, I've cc'd Larry at iXsystems. He's already setting up to take > a look so maybe you guys can coordinate ;) > > Cheers, > -matt > > > Dear Matt and Larry, > > Would be great if we could be in touch and have some coordination to > work together and bring it up on FreeBSD. > So, let me know your plan guys, and let's sync our tasks. > > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org -- ========================== Larry P. Maloney Senior Systems Engineer Cell: 408-893-0294 --------------060605070709090805030809-- From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 16:37:10 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CBB781065672; Mon, 24 Sep 2012 16:37:10 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 61FDC8FC08; Mon, 24 Sep 2012 16:37:10 +0000 (UTC) Received: by qafi29 with SMTP id i29so2618583qaf.13 for ; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=Vzpn9+UP/ku1etxA4xWfF5k3zGR7hP9+siE401MBpcM=; b=iGZzIko4hOOCVT87s6eGq34LHxLcKAdxGlSSpZ4ong30pF65Vgegpxnq1HBiKYBns4 TYGM63Y9FQiKk77bKatSS0mYYihpLZpjToSW2HUIZLRM7xyonN+FgirZSwKQtH6WC8bP FfzBCsJ60ypPKfWjP+cp4Km7YW0vrtmn8OlOLKSYNA8boctUF+/o3r+IRLFN1QZ6qH8/ +P7KhsSey/5PvAToRhgeEENQvfgnuiOyrqKXh41wYMbiEUmXB6rZwory32n9PcBsoLuR hG5k7XqQ9Yxu3V+kHpbU5DiaZUbQSsIV01ypUVXUWryh6+kEuGKllx1itCHDCUrsdYgY ApIA== MIME-Version: 1.0 Received: by 10.224.18.138 with SMTP id w10mr9529005qaa.20.1348504629631; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) In-Reply-To: <506087A8.4000001@ixsystems.com> References: <506087A8.4000001@ixsystems.com> Date: Tue, 25 Sep 2012 00:37:09 +0800 Message-ID: From: Marcelo Araujo To: larry@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 16:37:10 -0000 Hello dear Larry, Right now I'm located in Taipei/Taiwan, I do believe we have a difference of 11hours of time-zone. Let me check this week the best day/time to have a meeting with you. Soon as I have the proper date, I'm gonna contact you back. Great know you, and definitely we must have a chat this week. Best Regards, - Araujo 2012/9/25 Larry Maloney > Sounds great! > > We should schedule a meeting, what is your schedule this week? > > Larry > > > > On 09/22/2012 10:48, Marcelo Araujo wrote: > > > > 2012/9/22 Matt Olander > >> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >> wrote: >> > Thank you so much to share it, I'm gonna take a close look and make some >> > plan to work on it. >> > Also, would be interesting know, whom is working and in which part, so >> we >> > could avoid dual effort on the same thing. >> > >> > I'm gonna build a machine today to test it. >> >> Nice job guys! >> >> Araujo, I've cc'd Larry at iXsystems. He's already setting up to take >> a look so maybe you guys can coordinate ;) >> >> Cheers, >> -matt >> >> > Dear Matt and Larry, > > Would be great if we could be in touch and have some coordination to work > together and bring it up on FreeBSD. > So, let me know your plan guys, and let's sync our tasks. > > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell: 408-893-0294 > > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 17:17:32 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 766F1106566B; Mon, 24 Sep 2012 17:17:32 +0000 (UTC) (envelope-from larry@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 26DA58FC0C; Mon, 24 Sep 2012 17:17:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.iXsystems.com (Postfix) with ESMTP id 6BEABC6B; Mon, 24 Sep 2012 10:17:31 -0700 (PDT) Received: from mail.iXsystems.com ([127.0.0.1]) by localhost (mail.ixsystems.com [127.0.0.1]) (maiad, port 10024) with ESMTP id 40745-01; Mon, 24 Sep 2012 10:17:30 -0700 (PDT) Received: from [10.2.3.72] (kruse-72.3.ixsystems.com [10.2.3.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id E867FC62; Mon, 24 Sep 2012 10:17:30 -0700 (PDT) Message-ID: <506095AA.2080503@ixsystems.com> Date: Mon, 24 Sep 2012 10:17:30 -0700 From: Larry Maloney Organization: iX Systems User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: <506087A8.4000001@ixsystems.com> In-Reply-To: Content-Type: multipart/mixed; boundary="------------080006090905010202080001" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: larry@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 17:17:32 -0000 This is a multi-part message in MIME format. --------------080006090905010202080001 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sounds great! Just let me know when, and I'll work my schedule to fit yours. Larry On 09/24/2012 09:37, Marcelo Araujo wrote: > Hello dear Larry, > > Right now I'm located in Taipei/Taiwan, I do believe we have a > difference of 11hours of time-zone. > Let me check this week the best day/time to have a meeting with you. > Soon as I have the proper date, I'm gonna contact you back. > > Great know you, and definitely we must have a chat this week. > > Best Regards, > - Araujo > > 2012/9/25 Larry Maloney > > > Sounds great! > > We should schedule a meeting, what is your schedule this week? > > Larry > > > > On 09/22/2012 10:48, Marcelo Araujo wrote: >> >> >> 2012/9/22 Matt Olander > > >> >> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >> > wrote: >> > Thank you so much to share it, I'm gonna take a close look >> and make some >> > plan to work on it. >> > Also, would be interesting know, whom is working and in >> which part, so we >> > could avoid dual effort on the same thing. >> > >> > I'm gonna build a machine today to test it. >> >> Nice job guys! >> >> Araujo, I've cc'd Larry at iXsystems. He's already setting up >> to take >> a look so maybe you guys can coordinate ;) >> >> Cheers, >> -matt >> >> >> Dear Matt and Larry, >> >> Would be great if we could be in touch and have some coordination >> to work together and bring it up on FreeBSD. >> So, let me know your plan guys, and let's sync our tasks. >> >> >> Best Regards, >> -- >> Marcelo Araujo >> araujo@FreeBSD.org > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell:408-893-0294 > > > > > -- > Marcelo Araujo > araujo@FreeBSD.org -- ========================== Larry P. Maloney Senior Systems Engineer Cell: 408-893-0294 --------------080006090905010202080001-- From owner-zfs-devel@FreeBSD.ORG Tue Sep 25 07:32:09 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 31412106564A for ; Tue, 25 Sep 2012 07:32:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 69A328FC1B for ; Tue, 25 Sep 2012 07:32:07 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA03422 for ; Tue, 25 Sep 2012 10:32:01 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TGPce-00086U-Ju for zfs-devel@FreeBSD.org; Tue, 25 Sep 2012 10:32:00 +0300 Message-ID: <50615DEE.6030609@FreeBSD.org> Date: Tue, 25 Sep 2012 10:31:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: ZFS_OBJ_HOLD_ENTER/z_hold_mtx recursion via getnewvnode X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Sep 2012 07:32:09 -0000 FYI: http://lists.freebsd.org/pipermail/freebsd-fs/2012-September/015188.html This panic seems to gain popularity lately. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 03:25:32 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99429106564A; Fri, 28 Sep 2012 03:25:32 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 317F78FC0C; Fri, 28 Sep 2012 03:25:32 +0000 (UTC) Received: by qcsl39 with SMTP id l39so2400791qcs.13 for ; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=z/wRAmQrHY0KGxkxJy2jFRZdIsVZBGCH5kEbSTvZO3U=; b=EiydS/SVNvG5M/lFWrsfow445SIDBh+rtB8grz0AmP5jssWkAcD1n2ZxFkX+Bg65XK wrFIDziRrIYtUyOQbk3PJEd+6ORrVOsiD/yme6SDtPh6ONNL3QRXQQZUnGCPI0ckRss1 yU7SKxeot+hP8UtaV7tNmSMj5XOT6VTuUrkodMVAODfLwMDsMR9DxZ2UVNIjLwvy/2MK 3QHdggbsXf6BZIR//VfMNzUQ+Pdk/PV03rwz6xTAx1Vf7QZcN+q+3oTdHB8A3BVs+nZ1 diZOK0J+zkO2z+Q/PkzOiCtIWLpLaS5qRz/fnEEXtqiJVbjSiTqa6u7YTie/3F2nj5oh avew== MIME-Version: 1.0 Received: by 10.224.18.138 with SMTP id w10mr14270496qaa.20.1348802731308; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) In-Reply-To: <506095AA.2080503@ixsystems.com> References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> Date: Fri, 28 Sep 2012 11:25:31 +0800 Message-ID: From: Marcelo Araujo To: larry@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 03:25:32 -0000 Hello dear Larry, I'm wondering if you could talk with me this Friday. If yes, let me know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, my time zone is UTC/GMT +8. Would be great if we can talk around 11PM(Taiwan) at my side, it will be 2PM(California) in your side, just let me know few hours in advance if we can talk or not. In case you can't, we can re-schedule to the next week. Here is my skype: araujobsd ** Best Regards, - Marcelo Araujo ** 2012/9/25 Larry Maloney > Sounds great! Just let me know when, and I'll work my schedule to fit > yours. > > Larry > > > On 09/24/2012 09:37, Marcelo Araujo wrote: > > Hello dear Larry, > > Right now I'm located in Taipei/Taiwan, I do believe we have a > difference of 11hours of time-zone. > Let me check this week the best day/time to have a meeting with you. Soon > as I have the proper date, I'm gonna contact you back. > > Great know you, and definitely we must have a chat this week. > > Best Regards, > - Araujo > > 2012/9/25 Larry Maloney > >> Sounds great! >> >> We should schedule a meeting, what is your schedule this week? >> >> Larry >> >> >> >> On 09/22/2012 10:48, Marcelo Araujo wrote: >> >> >> >> 2012/9/22 Matt Olander >> >>> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >>> wrote: >>> > Thank you so much to share it, I'm gonna take a close look and make >>> some >>> > plan to work on it. >>> > Also, would be interesting know, whom is working and in which part, so >>> we >>> > could avoid dual effort on the same thing. >>> > >>> > I'm gonna build a machine today to test it. >>> >>> Nice job guys! >>> >>> Araujo, I've cc'd Larry at iXsystems. He's already setting up to take >>> a look so maybe you guys can coordinate ;) >>> >>> Cheers, >>> -matt >>> >>> >> Dear Matt and Larry, >> >> Would be great if we could be in touch and have some coordination to work >> together and bring it up on FreeBSD. >> So, let me know your plan guys, and let's sync our tasks. >> >> >> Best Regards, >> -- >> Marcelo Araujo >> araujo@FreeBSD.org >> >> >> >> -- >> ========================== >> Larry P. Maloney >> Senior Systems Engineer >> Cell: 408-893-0294 >> >> > > > -- > Marcelo Araujo > araujo@FreeBSD.org > > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell: 408-893-0294 > > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 08:19:37 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0A9B1065670; Fri, 28 Sep 2012 08:19:37 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 9A1A38FC15; Fri, 28 Sep 2012 08:19:37 +0000 (UTC) Received: from Xins-MacBook-Pro.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 5E8C3FF35; Fri, 28 Sep 2012 01:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1348820377; bh=6dRh4y9x7/VmMUtIYK/dUnz4MXdb1Fceg7HaTeURQ9A=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=IJrcnep9LcFtzVAVDg6/y3nMSa2nNDE5u+wl8v9d7AoQA9x7ARMVHZLBJqlLAlCmM 21kSCbG/r2oiz7MyJpI9z7eA++in7Gxi+dagfyDTBzgd6Dbim4c+n7WLtJwC2v9zCO 7zsiw4Kc6p4woU3S1NCZOiW1/Ug/OrFmiwH0XFw0= Message-ID: <50655D99.8090703@delphij.net> Date: Fri, 28 Sep 2012 01:19:37 -0700 From: Xin Li Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> In-Reply-To: X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: 7bit Cc: Will Andrews , larry@ixsystems.com, zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 08:19:38 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Marcelo, On 9/27/12 8:25 PM, Marcelo Araujo wrote: > Hello dear Larry, > > I'm wondering if you could talk with me this Friday. If yes, let me > know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, > my time zone is UTC/GMT +8. > > Would be great if we can talk around 11PM(Taiwan) at my side, it > will be 2PM(California) in your side, just let me know few hours in > advance if we can talk or not. In case you can't, we can > re-schedule to the next week. I believe these time was wrong. 23:00 CST for you is 8:00 PDT for us. 14:00 PDT for us is about 5:00 CST in your morning. 17:00 PDT for us is about 8:00 CST in your morning. > Here is my skype: araujobsd ** Best Regards, - Marcelo Araujo ** > > 2012/9/25 Larry Maloney > >> Sounds great! Just let me know when, and I'll work my schedule >> to fit yours. >> >> Larry >> >> >> On 09/24/2012 09:37, Marcelo Araujo wrote: >> >> Hello dear Larry, >> >> Right now I'm located in Taipei/Taiwan, I do believe we have a >> difference of 11hours of time-zone. Let me check this week the >> best day/time to have a meeting with you. Soon as I have the >> proper date, I'm gonna contact you back. >> >> Great know you, and definitely we must have a chat this week. >> >> Best Regards, - Araujo >> >> 2012/9/25 Larry Maloney >> >>> Sounds great! >>> >>> We should schedule a meeting, what is your schedule this week? >>> >>> Larry >>> >>> >>> >>> On 09/22/2012 10:48, Marcelo Araujo wrote: >>> >>> >>> >>> 2012/9/22 Matt Olander >>> >>>> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >>>> wrote: >>>>> Thank you so much to share it, I'm gonna take a close look >>>>> and make >>>> some >>>>> plan to work on it. Also, would be interesting know, whom >>>>> is working and in which part, so >>>> we >>>>> could avoid dual effort on the same thing. >>>>> >>>>> I'm gonna build a machine today to test it. >>>> >>>> Nice job guys! >>>> >>>> Araujo, I've cc'd Larry at iXsystems. He's already setting up >>>> to take a look so maybe you guys can coordinate ;) >>>> >>>> Cheers, -matt >>>> >>>> >>> Dear Matt and Larry, >>> >>> Would be great if we could be in touch and have some >>> coordination to work together and bring it up on FreeBSD. So, >>> let me know your plan guys, and let's sync our tasks. >>> >>> >>> Best Regards, -- Marcelo Araujo araujo@FreeBSD.org >>> >>> >>> >>> -- ========================== Larry P. Maloney Senior Systems >>> Engineer Cell: 408-893-0294 >>> >>> >> >> >> -- Marcelo Araujo araujo@FreeBSD.org >> >> >> >> -- ========================== Larry P. Maloney Senior Systems >> Engineer Cell: 408-893-0294 >> >> > > -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) iQEcBAEBCAAGBQJQZV2ZAAoJEG80Jeu8UPuz4yMH/RJbLrdeu37BeHi7Q/vf+ok3 ve4sPTF+k8R9qRRdKJ+xP52V2LeBJdT7TqxHA0+4YTwmGVjZlCaetXy8yUDysoth VA3EGaWFSzdTGV03xZ00KGcLfgH4uMUhN/7AADSr8g0h/ipBYQouzJ07hNe5l+0/ gSJd38FX3vmTI/PmcMHVey2+sa493jWpREnSzrtQs98kxkARGObL4G6bQCf4jEYA iqfqRb4ssyI65kh80MiCYbCYK0J0lnDkfDQjLNK44J7kY/S6Nk4+nKu/SdU7zLc4 aEWvIfZ8dScM1OGhYCOrZ+NaI6M2+yzeSbb0mNkK+x9VnNb55TzrtHW+zhnCuUs= =6WZA -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 08:59:39 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D9113106564A for ; Fri, 28 Sep 2012 08:59:39 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 881398FC1C for ; Fri, 28 Sep 2012 08:59:39 +0000 (UTC) Received: by qafi29 with SMTP id i29so6653343qaf.13 for ; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=LZaLQl9vunWdzEbUzJhDRIyoQ6Zl+BeaSN/i5SxlmVc=; b=I3CxXI3hTBhjhS9510Z8bjvT+aR/3jprLYm40s47aW3laeuAP61vSADfc7Pk54A6Rc qkB6G3hSTatFqHgVBzs9SOpFciQ8Nt+zTU+i6SKlvf0Zhk6U+2NQULV/i0Etjt6sc1Mb 7tFNPB0ZoM+Ew0ehsh7wh04IPMLU7jGgCoAa4G6Yo+AytQqgEyHj8aTx6lJhxpfNo2V6 7hLxIC/QY3qoLCIsXbINIYJWzIM0P/tgCrvfeijB08hsN1EOG2oQ/UuqglN35MVWu3Yh b7kt2/MwdsB7yCqU6h4Gq2EhETBw+pFrmfX4c35sur/Rm27yb6yvE/EJaLyHw3H2YY9V lpww== MIME-Version: 1.0 Received: by 10.229.69.67 with SMTP id y3mr4115070qci.136.1348822778716; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) In-Reply-To: <50655D99.8090703@delphij.net> References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> <50655D99.8090703@delphij.net> Date: Fri, 28 Sep 2012 16:59:38 +0800 Message-ID: From: Marcelo Araujo To: d@delphij.net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , larry@ixsystems.com, zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 08:59:40 -0000 2012/9/28 Xin Li > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi, Marcelo, > > On 9/27/12 8:25 PM, Marcelo Araujo wrote: > > Hello dear Larry, > > > > I'm wondering if you could talk with me this Friday. If yes, let me > > know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, > > my time zone is UTC/GMT +8. > > > > Would be great if we can talk around 11PM(Taiwan) at my side, it > > will be 2PM(California) in your side, just let me know few hours in > > advance if we can talk or not. In case you can't, we can > > re-schedule to the next week. > > I believe these time was wrong. 23:00 CST for you is 8:00 PDT for us. > > 14:00 PDT for us is about 5:00 CST in your morning. > > 17:00 PDT for us is about 8:00 CST in your morning. > Delphij, You are right! I always get confused with timezone. So, if I'm right now, let's setup this at 8AM(Saturday, Taiwan), 5PM(Friday, California). Is that OK? In case you want reschedule, feel free! Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu Oct 4 12:31:40 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1642E106566C; Thu, 4 Oct 2012 12:31:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 024BA8FC0C; Thu, 4 Oct 2012 12:31:38 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA17803; Thu, 04 Oct 2012 15:31:36 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506D81A7.8030506@FreeBSD.org> Date: Thu, 04 Oct 2012 15:31:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: Nikolay Denev References: <906543F2-96BD-4519-B693-FD5AFB646F87@gmail.com> <506BF372.1090208@FreeBSD.org> <506C4049.4040100@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 04 Oct 2012 12:35:21 +0000 Cc: freebsd-fs , Pawel Jakub Dawidek Subject: Re: nfs + zfs hangs on RELENG_9 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 12:31:40 -0000 [restoring cc to fs@] on 04/10/2012 14:32 Nikolay Denev said the following: > I have procstat only for the nfsd threads from the moment of the IO hang. > And this is the only one with "arc" : > > 1422 138630 nfsd nfsd: service mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x390 arc_lowmem+0x77 kmem_malloc+0xc1 > uma_large_malloc+0x4a malloc+0xd9 arc_get_data_buf+0xb5 arc_read_nolock+0x1ec > arc_read+0x93 dbuf_read+0x452 dmu_buf_hold_array_by_dnode+0x16b > dmu_buf_hold_array+0x67 dmu_read_uio+0x3f zfs_freebsd_read+0x3e8 > nfsvno_read+0x2e5 nfsrvd_read+0x3ff nfsrvd_dorpc+0x3c0 Oh, very important stack trace. Earlier Nikolay Denev said the following: > PID TID COMM TDNAME KSTACK > 7 100192 zfskern arc_reclaim_thre mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x428 > _sx_xlock+0x51 arc_buf_remove_ref+0x8a dbuf_rele_and_unlock+0x132 dbuf_evict+0x11 > dbuf_do_evict+0x53 arc_do_user_evicts+0xb4 arc_reclaim_thread+0x263 fork_exit+0x11f > fork_trampoline+0xe To me this looks like a deadlock caused by a FreeBSD add-on to ZFS: arc_lowmem handler. I think that this is what happens: The nfsd thread does read, arc_read_nolock finds a buffer in a ghost cache and calls arc_get_data_buf while holding a hash_lock (one of buffer hash locks). arc_get_data_buf needs to allocate some memory and, as luck would have it, there is a memory shortage. Low memory handlers are invoked (directly) and one of them is arc_lowmem. arc_lowmem simply kicks arc_reclaim_thread to do its job and then loops sleep-waiting until memory shortage is less severe. arc_reclaim_thread tries to evict some buffers and, as luck would have it again, it attempts to evict either the same buffer or, most likely, a different buffer that hashes to the same lock. So arc_reclaim_thread is blocked on the arc buffer lock. While the nfsd thread holds the lock, but waits in arc_lowmem for arc_reclaim_thread to make progress. Eventually the held lock stalls other threads that attempt to grab it, the stall propagates to txg_sync_thread threads and all ZFS I/O stops. -- Andriy Gapon