Support Grsecurity/PaX
|
Jason A. Donenfeld
Dear Core Infrastructure Initiative:
I do consulting for several different security companies. The uniform advice across the industry is: if you want to deploy Linux securely, be sure to be using a Grsecurity/PaX kernel. This advice is given time after time in reports authored by several security companies for whom I consult. And companies who take their security seriously wind up using Grsecurity/PaX. This isn't just security-scare-speak rattling the saber; this is pretty much essential and true. As somebody who works day in day out developing exploits and finding vulnerability, I can tell you that Grsecurity/PaX makes my work considerably more difficult. The defenses Grsecurity/PaX offers are real, and are useful. They're doing important work, and as a whole they're moving everybody forward. Simply put, anybody serious about security uses a Grsecurity/PaX kernel. Yet, as far as I can see, the developers (CC'd) receive basically no funding, and since the source they release is GPL, the thousands of companies who deploy Grsecurity/PaX are not compelled to give any financial support. As such, the financial situation of the developers is in nearly constant peril, and this core Linux project lives under a consistent existential threat. Were it not for Grsecurity/PaX, I would probably not be using the Linux kernel in my mission critical infrastructure, in favor of some flavor of BSD that's copied Grsecurity/PaX's techniques. Or simply in favor of a smaller, less performant, more minimal BSD kernel, that would be a big hassle, but would have a smaller attack surface. However, since there is Grsecurity/PaX, I feel a bit more comfortable deploying Linux. And I think many in the security industry feel the same. So, Core Infrastructure Initiative - please - consider supporting them. I'm not affiliated with them in anyway, but in stumbling across CII, and reviewing what CII is up to, it struck me that there's currently a glaring omission in supported projects. Grsecurity/PaX ought to be at the top of the list. Thank you, Jason Donenfeld
|
|
|
|
Dan Kohn <dan@...>
Thanks for the comment. Has anything significant changed since 2009?
toggle quoted messageShow quoted text
https://lwn.net/Articles/313621/ If members of the PAX team would like to apply for a grant to break up their work and work with upstream to get it included in mainline Linux, CII would be happy to consider it. https://www.coreinfrastructure.org/contact -- Dan Kohn <mailto:dan@...> tel:+1-415-233-1000
On Wed, Aug 19, 2015 at 9:12 AM, Jason A. Donenfeld <Jason@...> wrote:
Dear Core Infrastructure Initiative:
|
|
|
|
Jason A. Donenfeld
I believe there have been significant changes since 2009. I'll let the Grsecurity/PaX developers chime in on this, I guess.
It's important to note, though, that it shouldn't be considered relevant whether or not their work is mainlined. This is an important core project that's extremely widely used by everybody who cares about security. Ensuring the work goes on is of *critical* importance. Not only that, but work done in Grsecurity/PaX _does_ make its way back to mainline bit by bit in the form of individual patches that are a rework of features pioneered by Grsecurity/PaX. In other words, mainlined or not, keeping Grsecurity/PaX around with healthy funding is of the utmost importance for the Linux ecosystem at large. We need them. (And I believe they very likely need you, CII.)
|
|
|
|
Dan Kohn
On Wed, Aug 19, 2015 at 10:03 AM, Jason A. Donenfeld <Jason@...> wrote:
It's important to note, though, that it shouldn't be considered relevantCII follows the philosophy of Linux development that long-term, out-of-mainline patches are problematic because of the maintenance issues and lack of peer-review. Of course, the Grsecurity/PaX team is welcome to continue as they have been, and the GPL allows them to charge for consulting services. But the most likely scenario for a CII grant would be to facilitate inclusion into mainline. -- Dan Kohn <mailto:dankohn@...> Senior Advisor, Core Infrastructure Initiative tel:+1-415-233-1000
|
|
|
|
Jason A. Donenfeld
On Wed, Aug 19, 2015 at 4:15 PM, Dan Kohn <dankohn@...> wrote:
CII follows the philosophy of Linux development that long-term, Please do try to consider Grsecurity/PaX as not "just another out of tree patchset" but rather a mission critical project that serves a real world benefit in addition to pushing the bounds with the research that goes into it. No matter the debate on the various kernel development practices and philosophies, nobody in the security world disagrees with the fact that keeping Grsecurity/PaX around as a healthy project is very critical. Regardless of the mainline discussion, Grsecurity/PaX, as it exists today, *is* *core* *infrastructure*.
|
|
|
|
Dan Kohn
On Wed, Aug 19, 2015 at 4:15 PM, Dan Kohn <dankohn@...> wrote: On Wed, Aug 19, 2015 at 10:21 AM, Jason A. Donenfeld <Jason@...> wrote:CII follows the philosophy of Linux development that long-term, Please do try to consider Grsecurity/PaX as not "just another out of treeJason, if CII funded Grsecurity/PaX for a year or two, it would keep the project going, but then what? It is unlikely that CII could fund the project indefinitely, so it would remain an unhealthy project. By contrast, if we funded mainlining the most important parts of Grsecurity, then the thousands of companies and individuals that develop Linux would take responsibility for maintaining it (and improving it) indefinitely. The parts that didn't get accepted could still be maintained out-of-mainline by whatever companies or individuals chose to do so. -- Dan Kohn <mailto:dankohn@...> Senior Advisor, Core Infrastructure Initiative tel:+1-415-233-1000
|
|
|
|
Jason A. Donenfeld
On Wed, Aug 19, 2015 at 4:43 PM, Dan Kohn <dankohn@...> wrote:
Jason, if CII funded Grsecurity/PaX for a year or two, it would keepThat's a nearly reasonable objection, but I think it's a bit narrow of a vision on how many open projects work. More generally, it's "how can a small but essential open source project be supported?" One answer is by merging with an already funded project, like the Linux kernel itself, that already has plenty of commercial investment, with paid developers. Another, more accessible, answer is -- by receiving funding when it can, using that funding to improve the project, and have those improvements result in more interest, and therefore more funding and grants down the line. It appears that many projects work this way. For example, OpenBSD seems to be supported by generous donations and grants (one of which has come from the CII, IIRC). The longer these projects go, the more likely funding from various places is to pour in. It seems that having CII fund Grsecurity/PaX for a year or two would indeed result in increased momentum, more interest, and therefore a steady stream of funding and support from elsewhere going forward. This, anyway, is the route that most smaller open source projects take, that do not directly have developers who are paid by commercial entities. And on top of this, there's the obvious point: a year or two of funded Grsecurity/PaX work would result in research and practical security solutions that would be a massive benefit to all. I suspect there are additional counter arguments as well that I am missing (others can chime in?), because were it the case that it's not worthwhile to fund something for a year or two as you described, many projects that do now receive funding would become woefully ineligible.
|
|
|
|
Dan Kohn
On Wed, Aug 19, 2015 at 10:59 AM, Jason A. Donenfeld <Jason@...> wrote:
That's a nearly reasonable objection, but I think it's a bit narrow ofJason, I do see your points, and the Grsecurity team is welcome to apply for funding whether they want to upstream their work as patches or keep it out-of-mainline. I am pointing out that sustainability is a major factor that the CII steering group evaluates, and that mainlining the work appears to be a much more sustainable path. Separately, I would encourage you (and them) to take a look at a sister project to the CII census, which is our best practices program. https://github.com/linuxfoundation/cii-best-practices-badge One criteria we might add in the future is around mainlining code. -- Dan Kohn <mailto:dankohn@...> Senior Advisor, Core Infrastructure Initiative tel:+1-415-233-1000
|
|
|
|
Jason A. Donenfeld
On Wed, Aug 19, 2015 at 5:11 PM, Dan Kohn <dankohn@...> wrote:
Jason, I do see your points, and the Grsecurity team is welcome toHopefully the Grsecurity/PaX developers are actually reading this thread and do consider applying! One last point, and then I'll fade away a little bit. There is a set of projects that are core infrastructure and are worth funding. There is a set of projects that should be mainlined. I think it's entirely reasonable that while these sets might overlap at times, they won't overlap all the time, and that's okay. Of course there are worthwhile kernel projects that are not a part of mainline. And in this case, one of those projects happens to be core infrastructure, and certainly worthy of a CII grant. I play jazz guitar; here's an analogy. Sometimes artists make pop music, and this sells many albums, and everyone is happy. Othertimes artists make jazz music, but nobody cares about jazz, so artists often can't sustain themselves. There are a few artists who do very well though -- say, Pat Metheny or Herbie Hancock. If your musical direction fits with theirs, you might get lucky and they'll bring you on tour (into the mainline!), and then you'll be set for life. But if not, you'll have to struggle. Fortunately there are a huge amount of foundations, institutions, conservatories, patrons, and so forth who help fund talented upcoming jazz artists. Often times these artists live grant-to-grant (shows don't pay well; cds are pirated). With the help of these patrons, some jazz artists have a chance, and eventually become the next big deal. Sometimes they don't become big, but having their artistic voice in the scene remains an inspiration for many other artists, and so these patrons have supported a good cause here too. Now this analogy breaks down, as there are many other odd ways for jazz musicians to make a living. But hopefully the goofy comparison will demonstrate that there are indeed very legitimate grounds for funding non-mainlined kernel code. Okay, thanks for listening; I'll let others take over the discussion from here.
|
|
|
|
Dan Kohn
On Wed, Aug 19, 2015 at 11:32 AM, Jason A. Donenfeld <Jason@...> wrote:
Of course there are worthwhileOne last question for you: could you name other such projects (not necessarily security-related). Nearly every other out-of-mainline project I'm aware of has eventually either merged or died out. -- Dan Kohn <mailto:dankohn@...> Senior Advisor, Core Infrastructure Initiative tel:+1-415-233-1000
|
|
|
|
PaX Team
On 19 Aug 2015 at 13:37, Dan Kohn wrote:
Hi Dan, On Wed, Aug 19, 2015 at 11:32 AM, Jason A. Donenfeld <Jason@...> wrote:Let's hope you only had bad luck and that's why you aren't aware ofOf course there are worthwhileOne last question for you: could you name other such projects (not any of these ;) http://lttng.org/download/ http://www.sysdig.org/wiki/how-to-install-sysdig-from-the-source-code/ http://open-mx.gforge.inria.fr/download/ http://knem.gforge.inria.fr/download/ http://download.savannah.gnu.org/releases/davfs2/ http://www.openafs.org/release/index.html http://www.asterisk.org/downloads/dahdi https://www.virtualbox.org/wiki/Downloads https://www.jetico.com/linux/installation.html http://zfsonlinux.org/ http://aufs.sourceforge.net/ http://cryptodev-linux.org/download.html http://loop-aes.sourceforge.net/loop-AES/ https://www.rsbac.org/download http://cdemu.org/about/vhba/ https://github.com/vmware/open-vm-tools http://scst.sourceforge.net/downloads.html http://sourceforge.net/projects/xtables-addons/files/Xtables-addons/ As for the requirement of mainlining grsec, it's not possible since we know right off the start that some of the features and other changes are not acceptable at all (say, all the x86 segmentation based code). Second, for the potentially viable pieces this would be a multi-year full time job. Is the CII willing to fund projects at that level? If not we all would end up with lots of unfinished and partially broken features. Third, you're actually wrong as to what is needed for mainline acceptance, it's most definitely not enough to dump the code on them and let the community figure it all out and take care of it. If anything, the exact opposite is true, for any non-trivial amount of code there has to be a pledge for long-term maintenance from the submitters (so regardless where grsec stays, in-tree or out-of-tree, the maintenance burden would still be ours for some years at least, with corresponding need of funding). Fourth, you mentioned the potential futility of not funding grsec indefinitely. It begs the question why it is then worth doing the same for the fuzzing project or Frama-C. I hope noone at CII believes that a year or two of fuzzing and static analysis will exterminate all bugs so what happens after their funding runs out? Or are you suggesting that they can be funded indefinitely but for some reason grsec could not be? In any case, it is your money and decision at the end and we thank you for your attention at least. However it is also clear that we have a different understanding of what constitutes securing the core infrastructure. cheers, PaX Team
|
|
|
|
Shawn C
Hi Dan,
toggle quoted messageShow quoted text
It'd be a tough work to break up the features of PaX/Grsec and push them into upstream, separately. The fact is vanilla kernel community don't care about mitigation. People who runs GNU/Linux servers for security critical scenes has been suffering from vanilla kernel guys's philosophy of A-bug-is-a-bug for years. Some brilliant developers doesn't even treat "bypass methods" as a threat: http://cybersecurity.upv.es/attacks/offset2lib/offset2lib.html http://www.openwall.com/lists/oss-security/2014/12/08/3 IMOHO, PaX/Grsec is the only option we have for mission critical infrastructure if we do want to continue using GNU/Linux.
On Wed, Aug 19, 2015 at 9:44 PM, Dan Kohn <dan@...> wrote:
Thanks for the comment. Has anything significant changed since 2009? --
GNU powered it... GPL protect it... God blessing it... regards Shawn
|
|
|
|
Kevin P. Fleming (BLOOMBERG/ 731 LEX)
(sorry for top-posting, our message system doesn't believe anything else is possible) The funding for the fuzzing project is intentionally short-to-medium-term, with two goals: helping to learn the 'state of the world' (which will feed into the census, and future support/funding decisions), and producing better and easier-to-apply fuzzing tools so that project teams can run them on their own software to identify flaws as early as possible. It's definitely not a long-term 'fuzz test the world' project. The funding for Frama-C is for further development of the tool (and related tools), not funding to do static analysis of large bodies of open source software. As a result, neither of these are suitable comparisons to the request for funding of grsecurity/PaX. Such a request would be more comparable to funding OpenSSL, ntpd, etc. Those are also not open-ended funding commitments, but are based on achieving improvements (in some cases, milestones) to continue funding. It's certainly possible that the SC would consider a funding proposal for grsecurity/PaX, but such a proposal would need to include reasonably well defined goals/milestones, and there's no question that the committee discussion would include the topic of the in-tree/out-of-tree status of these tools.
From: pageexec@... At: Aug 19 2015 17:56:13 To: Jason@..., dankohn@... Cc: cii-census@..., cii-discuss@..., spender@... Subject: Re: [cii-census] Support Grsecurity/PaX On 19 Aug 2015 at 13:37, Dan Kohn wrote:
|
|
|
|
PaX Team
On 21 Aug 2015 at 18:19, Kevin P. Fleming (BLOOMBERG/ 731 LEX) wrote:
Hi Kevin, first of all, thanks for your detailed response and new information that I was not aware of. However as much as it clarified some points, it also raised new questions. You see, I brought up the issue with fuzzing and static analysis because Dan Kohn said this earlier: Jason, if CII funded Grsecurity/PaX for a year or two, it would keepThe way I read this response suggested to me that long-term viability is an important (and possibly deal breaker?) factor in your funding decisions. Now you are saying that it was not for the mentioned projects. This leaves me confused as I do not know what applies and does not apply to a project such as ours. You also said that grsecurity was not comparable to fuzzing/static analysis and is more like a standalone(?) product. I beg to differ here as we produce much more than just a kernel patch albeit it is perhaps less advertised. Namely, due to the nature of our proactive defense mechanisms (both runtime and compile time), they are also good at catching bugs (almost always with security impact) and we have found and fixed a number of them for the past few years. One would think that exposing such technologies to a wider audience would have a much bigger impact on everyone's security than our own limited efforts. cheers, PaX Team
|
|
|
|
Kevin P. Fleming (BLOOMBERG/ 731 LEX)
I believe I may not have communicated my thoughts as clearly as I should have, so I'll attempt to clarify :-) First, it's absolutely true that long-term viability is a component of funding decisions. A goal of funding any project is to allow it to reach a point where it can be self-sustaining at a level that allows it to produce high-quality software, respond well to its users' demands, and continue adapting to the changing needs of the community. Some projects have had difficulty reaching that point on their own, so CII funding has been put in place to get them jump-started, but the CII doesn't plan to be their sole funding source indefinitely. That's why I said that a funding proposal would need to include clear objectives, so that there's a visible path to being self-sustaining. When I commented on the comparison to the fuzzing and Frama-C projects, I wasn't commenting on the nature of the software involved, I was commenting on the structure of the funding proposals (hypothetical, of course, in the case of grsecurity/PaX). The funding proposals for the fuzzing project and Frama-C, by their nature, have definite horizons, and the goals are to produce tools that the community will adopt in such a way that CII funding is no longer needed for the projects to survive and thrive. The CII might still fund them in order to allow them to grow at a more rapid pace, or to explore research that might not be supported by other community members, but those would be *new* funding proposals. So, my point with in both situations is that the CII steering committee is unlikely to review, and definitely not approve, a funding request that is essentially just employing the project's developers to continue what they are doing. In each case as we've considered a funding proposal, the most important aspects have been measurable, objective improvements in the project's health (measured in a number of ways, of course). While I certainly can't speak for the entire committee, I have no doubt that a significant item of discussion around a grsecurity/PaX proposal would be the effectiveness of funding a project whose codebase is not incorporated into its 'host' project, and whether such incorporation would be a suitable objective. Clearly, if these tools *could* be merged into Linux proper, they would likely see even greater adoption, and they'd benefit a much larger audience than they do today.
From: pageexec@... At: Aug 21 2015 15:40:49 To: Kevin P. Fleming (BLOOMBERG/ 731 LEX), Jason@..., dankohn@... Cc: cii-census@..., cii-discuss@..., spender@... Subject: Re: [cii-census] Support Grsecurity/PaX On 21 Aug 2015 at 18:19, Kevin P. Fleming (BLOOMBERG/ 731 LEX) wrote:
|
|
|
|
Tom Ritter
I'm pretty far from the kernel development community. I know the
generalities we've seen in this threat about different communities attitudes about mainlining, and I understand that pretty much everyone is either frustrated with the situation or given up on it. I just wanted to weigh in add support to the notion that PaX/grsec is a critical piece of security software, is very highly regarded, and it's always the first thing we recommend to people when they ask "How an we harden our systems?" Then they almost never do it. It _has_ been very frustrating over the years that it has not been mainlined, but I would rather it exist out-of-tree and be made as available as easily and broadly as possible than not exist at all. Maybe the answer is going to the distros and making it easy to switch to a patched kernel to drive adoption, or maybe that's a horrible idea. I think it would be wonderful if something could be done - but I don't know what the exact plan could be. -tom
|
|
|
|
Meredith Whittaker
Seems like there are a number of thoughts, and a general consensus that grsecurity is useful and used and deserves support. Cool! OK. What's missing is a proposal, tying funding to specific outcomes. I think this is a next step that would help narrow this conversation, and allow CII to vote on funding during its next Steering meeting (Sept. 17th). Cheers, Meredith
On Fri, Aug 21, 2015 at 5:29 PM, Tom Ritter <tom@...> wrote: I'm pretty far from the kernel development community. I know the --
Meredith Whittaker
Open Source Research Lead Google NYC
|
|
|
|
Jason A. Donenfeld
Hello PaX Team, Spender, After spending a few weeks meditating on this thread and its responses, it strikes me that the best thing to do might be to, in fact, apply to the CII. The initial discussion was met with quite a bit of resistance from Dan, but the ensuing feedback from the community at large has been overwhelmingly in favor of funding Grsecurity/PaX. And actually, I think Dan's early responses form an important part in the development of critical CII policies. It is a relevant question -- how should out of mainline projects be handled, or more generally, how do other Linux kernel projects coexist with Linus' in terms of CII, and even more generally, what is the relationship between the CII's funding and a project's long term financial sustainability? These are all important questions that do need to be addressed during the CII's meetings. But, if anything is certain in all of this, it's that Grsecurity/PaX should/must receive funding. It's a sentiment echoed extremely widely throughout multiple communities and industries that rely on Linux. That means that when the CII does sit down to work out these interesting and complicated policy questions, they will do so with the goal in mind that whatever their policies are, they must allow for the funding of Grsecurity/PaX. I think this is a very good position to be in. For this reason, I believe it makes sense to put in an application for CII funding. From my assessment of the matter, I do imagine that the committee will be in favor of Grsecurity/PaX -- and why shouldn't they be? -- and that direction will help them formulate the necessary policies to ensure that the CII is useful for critically essential projects like Grsecurity/PaX. Jason
|
|
|