####################################################################### 1) What happened 2) Doubts and conclusions 3) Round II ####################################################################### ---------------- 1) What happened ---------------- The 29th August 2012 I have noticed the blog post of Aaron Portnoy on his company's website blog: http://blog.exodusintel.com/2012/08/29/when-wrapping-it-up-goes-wrong/ UPDATE Aaron removed that blog post on ExodusIntel (why?), you can find a backup made the 30 Aug 2012 here: http://aluigi.altervista.org/misc/EIP-2012-0001.htm It's a format string vulnerability in EMC Networker on which he worked at the end of June and sold it to his customers: 24 Jun 2012: Just finished an exploit for a nice windows server-side format string bug I found. Now I have to wait for the patch before dropping a blog. and This vulnerability (and accompanying exploit) was detailed in our Exodus Intelligence feed and distributed to our customers in June. Aaron was the manager of ZDI TippingPoint, the company that buys information about undisclosed vulnerabilities to create filters for its IPS and then reports them to the vendors. In short he had the job of supervising all the new vulnerabilities submitted by the external researchers, managing the reviews and so on, he has been my direct contact at ZDI for anything I asked (and I complained and annoyed him often) about the submitted vulnerabilities and many other things. From now on I will refer with the name ZDI to TippingPoint and obviously to its owner, HP. The problem is that the vulnerability disclosed by his new company (ExodusIntel) is the same bug I reported to ZDI the 25th March with the id aluigi0216, the following is the full changelog visible about the case on my ZDI account: EMC NetWorker nsrd format string (nsrd_1) Case ID: aluigi0216, Status: CLOSED Case Opened 2012-03-25 12:41 GMT-6 A case has been opened and added to the queue for review. Case Reviewed 2012-05-15 13:33 GMT-6 This case has been reviewed. Case Closed 2012-08-28 14:28 GMT-6 Investigation of this case is complete. It shows 3 information: - the date in which I opened the case, I found the bug the 23th March and reported it two days later - a message that I have never seen on ZDI "This case has been reviewed." and the following is the answer they gave to me when I asked info about this weird note: That is a by-product of us internally transitioning the state of this particular case in our issue tracking system. It can be ignored. The case is still be reviewed and you should have some feedback soon. Sorry for the confusion. Regards, - - The ZDI Team - the 28th August the case has been closed after my request of closing all my pending cases, confirmed also publicly with an announment made on twitter: 27 Aug 2012 I no longer contribute to @thezdi or ANY other third-party external "program" it may be also possible that the case was closed because EMC patched the bug, it just changes nothing and in any case it's after my leaving So ZDI kept my bug in its hands for over 5 months and during all this time I have NEVER received a closing or rejection of the case and, more important, NEVER a duplicate case offer which is the only thing they do a bit more quicker because take less time to see if other researchers or the internal team has already found/reported the same bug, moreover if the report contains detailed information or directly the root cause (like mine). ZDI is interested only in the root cause of the vulnerability, which is perfectly logical and I agree at 100% with them (they don't need exploits), and the following is the content of the case I submitted as aluigi0216: Application: EMC NetWorker (Legato) http://www.emc.com/backup-and-recovery/networker/networker.htm Versions: <= 7.6 sp3 (7.6.3.2 Build 860) Platforms: AIX, HP-UX, Linux, Solaris, Windows Bug: format string Format string vulnerability in nsrd (program 390103, process 124) exploitable through an unexeistent username. The final string passed to lg_sprintf without the format argument contains both the username and the hostname provided by the attacker: 0043BAE3 |. 50 PUSH EAX 0043BAE4 |. 8B46 08 MOV EAX,DWORD PTR DS:[ESI+8] 0043BAE7 |. 6A 00 PUSH 0 0043BAE9 |. 50 PUSH EAX 0043BAEA |. 6A 00 PUSH 0 0043BAEC |. 68 C44B4B00 PUSH nsrd.004B4BC4 ; ASCII "%s %s event: %s" 0043BAF1 |. 68 08070000 PUSH 708 0043BAF6 |. 68 60015000 PUSH nsrd.00500160 ; ASCII "42502 15 %s %s event: %s 3 0 14 access control 0 7 warning 49 165 78099 85 gstd process running as user '%s@%s' is not in the '%s' list of NetWorker server '%s' 4 13 14 EVIL_USERNAME 12 2 xp 11 18 2344:administrator 12 9 EVIL_HOSTNAME" 0043BAFB |. 68 06A60000 PUSH 0A606 0043BB00 |. E8 6BE60500 CALL 0043BB05 |. 55 PUSH EBP 0043BB06 |. 6A 31 PUSH 31 0043BB08 |. 68 54104A00 PUSH nsrd.004A1054 ; ASCII "%s" 0043BB0D |. 6A 00 PUSH 0 0043BB0F |. 68 07A60000 PUSH 0A607 0043BB14 |. E8 4BE60500 CALL 0043BB19 |. 50 PUSH EAX 0043BB1A |. 8D4C24 7C LEA ECX,DWORD PTR SS:[ESP+7C] 0043BB1E |. 51 PUSH ECX ; "gstd process running as user 'EVIL_USERNAME@xp' is not in the 'administrator' list of NetWorker server 'EVIL_HOSTNAME'" 0043BB1F |. E8 5CE50500 CALL ; format string vulnerability No problems on the *nix platforms while there is only a note about Windows where it's not possible to use %n but it's possible to exploit the vulnerability as a classical buffer-overflow as shown in the proof-of-concept. As clearly visible from the snippet (but still not clear to some people in the new Aaron's team, scary) the problem is the calling of lg_sprintf without the necessary "%s" format argument. Obviously the description was accompanied by a proof-of-concept for demonstrating the vulnerability, oh and in case you are asking, no ZDI does NOT want weaponized exploits because they need only to write the IPS rule about the bug (everything is confirmed in various mails, it doesn't change the final offer and saves my time). If we watch his post on ExodusIntel we see the same code: ; lg_sprintf call with %n format specifiers and a dest of ecx: 0:000> r nsrd+0x3bb1f: 0043bb1f e80ce50500 call nsrd+0x9a030 ; lg_sprintf 0:000> da poi(@esp+4) L10 00999a30 "%n%n%n%n%n%n%n%n" 0:000> ub @eip L1 nsrd+0x3bb1e: 0043bb1e 51 push ecx 0:000> r @ecx ecx=00126f00 Just to clarify to people outside the security scene, doesn't matter what RPC program/opcode or what way you use to reach the vulnerability: longer, shorter, pre-auth, post-auth, better, worst... nothing! The only thing that matters to ZDI and any other normal security researcher in the world is the location of the vulnerability, the root cause: lack of "%s". When I noticed it I have immediately asked info about this curious "coincidence" on Twitter: @exodusintel @aaronportnoy suspicious that "your" EIP-2012-0001 is the same bug I reported to @thezdi the 25 March when you worked for them Now what happens is really weird because most of the tweets made by Aaron have been edited multiple times during the discussion and some of them have been even deleted for unknown reasons, anyway: Aaron: @luigi_auriemma @exodusintel Incorrect, my bug was discovered long before March, but wasn't realized as exploitable until after I left note: he specified a certain 2012-02-20 but he modified it, then he confirms that he was aware of my case that has been NEVER tagged as duplicate or rejected. immediately he asked to send him my original ZDI report so he would have showed me where they were different... what? practically this second tweet confirms what I said before and the fact that he knows perfectly about what I'm talking. note that this tweet has been deleted by Aaron my reply to this second tweet was: @aaronportnoy you got access to the report for 2 months at ZDI and you have just admitted you were aware about my case here there was another tweet of Aaron that has been deleted, unfortunately I don't have the original but judging my reply it was still something similar to his previous one (different RPC blah blah blah) my reply: @aaronportnoy @ExodusIntel @thezdi ZDI has never rejected my report for 5 months and you were the manager there, so stop finding excuses Aaron: @luigi_auriemma @ExodusIntel @thezdi lol, like I said, I discovered the bug more than a month before your submission. What is your point? my reply: @aaronportnoy @ExodusIntel @thezdi I repeat, my case has never been closed for 5 months as duplicated or rejected. aluigi0216 is the proof Aaron: FTR, @luigi_auriemma's bug was submitted 1 month AFTER mine in a diff RPC interface and diff RPC opcode. Other than that, identical (lolz?) oh wait how it's possible that now he knows details about my case on ZDI if he asked my report just before? maybe he has access to a dump of the ZDI db? mah Note that he no longer works at ZDI and he is even a competitor so he can't see my cases (phisically and legally) my reply: @aaronportnoy @exodusintel @thezdi it's the SAME bug, stop finding excuses. you are stealing undisclosed bugs from ZDI db... shame Before making some conclusions and finally talking about ZDI it's interesting to show these 3 tweets: digitalbond "RT @aaronportnoy: I discovered the bug more than a month before your submission < while you were at ZDI? How then did @ExodusIntel get it?" Aaron "@digitalbond @ExodusIntel The bug at ZDI was closed as unexploitable. Any bug closed at ZDI remains the submitting researcher's property." digitalbond "@aaronportnoy @ExodusIntel hmm, ok, but you were an employee. Tipping Points' invention/IP agreement must be very unusual." As noticed by DigitalBond, Aaron is saying that the vulnerability he claims to have found in Februrary was reviewed by himself when he was at ZDI and he classified it as non-exploitable (remember that ZDI needs rules for the IPS and not weaponized exploits) but magically 4 months later it become exploitable exactly when he opened his new company. So ZDI lost a genuine vulnerability because his manager said it wasn't a bug but the same manager made money with it. ####################################################################### ------------------------- 2) Doubts and conclusions ------------------------- Simply reading Aaron tweets, the modifications he made on them, the tweets he and his friend Rolf deleted and the lack of a serious explanation ("lol" is far away from the explanation I deserve) we have the following situation: - Aaron was perfectly aware of the bug I reported the 25th March 2012, he knew even its details - Aaron and his team at ZDI (both during his work there and after his leaving) have NEVER rejected or considered my aluigi0216 a duplicate for 5 long months. now the question is why my case wasn't kicked out if he already knew it was a duplicate? if it's true that he found it before me why not rejecting mine and saving mine/his time? he has claimed to have had it under his hands so kick out my case, why wasting my time for an useless review? - I have suspects that my case aluigi0216 has been delayed in some ways by ZDI, the doubt comes from that weird note on the case's changelog why ZDI has left that note in May but has never contacted me back soon as they promised? I waited 3 months! note that I have asked updates about that case multiple times but everytime they claimed to be full of work (the same thing said from when I joined in September 2010, boring) - there are no proofs and cross-checked references of the claimed internal work of Aaron on the same bug in February. but even in that case it's not possible to make yours a bug that has been reported by an external researcher to the company for which you are the manager and there is a signed agreement with him. In short: - why a manager closes a vulnerability he claims to have investigated for selling it 4 months later through his new company? - why ZDI allows ex-employees to take out the internal work that was done at ZDI and was deliberately classified as "garbage" (closed and abandoned by ZDI)? - what HP thinks about TippingPoint giving vulnerabilities out to a new competitor? - why Aaron took the bug in his pocket for 4 months if it was unexploitable? if it's garbage just trash it and forget it, wasn't it enough to ZDI but is it enough for ExodusIntel? Does HP agree? - why he reported and sold it even if he was perfectly aware that I found the same bug and reported it to ZDI when he was working there? doesn't he imagine that probably I would have said "something" when it went public? it was a bug reported by an external researcher who gives 100% of trust in ZDI and was still under review when he left the company, additionally he closed his February work and so with what face you sell it to your customers? I talk about agreements and ethic, oh come on - does Aaron still have access to the ZDI database? Aaron no longer works for ZDI from May 2012 but by reading his tweets it looks like this is more than just an hypothesis... maybe he has only an excellent memory able to remember thousands of ZDI cases and their details, sure... first he asks to provide my case and then he says it's a different RPC, that has nothing to do with the bug since it's just a tentative to fool non-technical people to think it's a different thing... blah. - when I joined ZDI I signed an agreement that at point 4.1 states various obligations that the company must honour versus the researchers for the period of time between the reporting of the bug and its acceptance/rejection: "...TPTI hereby agrees: (i) to hold your Vulnerability Information in confidence and to take all reasonable precautions to protect such Vulnerability Information (including, without limitation, all precautions TPTI employs with respect to its confidential materials), (ii) not to divulge such Vulnerability Information to any third person, and (iii) not to make any use whatsoever at any time of such Vulnerability Information except to the extent contemplated by Section 3.2 above. (note by LA: 3.2 is the "License to Evaluate") You agree that the foregoing clauses (i), (ii), and (iii) shall not apply with respect to any Vulnerability Information (A) after one year following the disclosure thereof to TPTI, or (B) that is or (through no improper action or inaction by TPTI or any affiliate, agent, consultant or employee) becomes generally known to the public, or (C) that was in TPTI’s possession or known by TPTI prior to receipt from you, or (D) that was rightfully disclosed to TPTI by a third party, or (E) that was independently developed by TPTI without use of any of your Vulnerability Information, or (F) was or is provided by you to third parties without similar restrictions. ..." so we have that during the evaluation of my case the information about the vulnerability have been divulged to third parties, in this case the customers of a competitor company owned by an ex-employee. if Aaron worked on the same bug in February (which is perfectly possible and I WANT to trust him) but it was closed then we have that point (E) is unaffected because officially for ZDI the DVLabs has NEVER found or owned this bug: as Aaron says in its tweet, it has been closed and its property went to himself so my evaluation was perfectly valid. in short, during the evaluation of the bugs ZDI and the researcher have a deep trust relation that can be truncated only if the bug is already known privately/publicly or they reject it: aluigi0216 has been alive till 28th August for 5 months. - other than having violated the agreement I have received also a monetary loss caused by the lack of an offer from ZDI and the impossibility to use these information on my own In conclusion any word that Aaron Portnoy tries to say to defend himself leads to a negative effect versus himself: - if it's false that he worked on the bug on Febrary it means he stole my case (well not directly), but with the current updates this doesn't seem the case. hopefully, or I want to hope - if he worked on the bug in February the effect is even worst because means he closed his case deliberately, delayed my case, took out a property of ZDI to a competitor (the bug was real) and many other things listed above Note that in both the cases my problem is NOT versus Aaron because my agreement and contact was with ZDI TippingPoint which is directly with me. This is very important to take in mind, I have never had an agreement with Aaron and he has no obligations with me. Aaron is responsible only versus TippingPoint and HP because he was their employee and so was their job to avoid similar situations that at the end affected me negatively: ZDI lost the bug and I lost everything. My job is only to point out publicly the weird thing that happened and trying to understand why and what happened. Stop. I have sent a mail to ZDI asking a statement about the fact to know also why it happened but I have receive no reply yet. I want to know who did the error: Aaron, ZDI or both. Little personal question to Aaron: but with all the researchers of this world who submitted bugs to ZDI why did you select the same bug submitted by the most picky person you know in the security field? Oh come on... ####################################################################### ----------- 3) Round II ----------- Finally a non-"lol"ed reply from Aaron: http://thunkers.net/~deft/misc/dramallamas.txt Now let's start checking what doesn't match and show what ZDI thinks completely differently. First point: i found the bug in question mid february and only noticed the stack overflow properties (not the format string). A stack overflow is not the same of a format string, the title of my submission was clear: EMC NetWorker nsrd format string (nsrd_1) The stack overflow is a secondary vulnerability that I considered and reported also in my submission but the main one on which I focused was the format string (ZDI would have closed one of my submissions if I would have reported them separately). So Aaron found/noticed the format string after the "late april" when his submission on ZDI was closed because he left, but please read later regarding what ZDI thinks about the February case. Then: in the latter claim he states i sat on his bug for 5 months. as i stated earlier, i left 5 *weeks* after his submission. ZDI was ZDI during and after Aaron, I was saying that ZDI (with and without him) has never closed/rejected my case for 5 months. Just a note, some people of the new Aaron's team migrated to the new company weeks after him. And: its odd to me that luigi let zdi sit on his bug for 5 months (as 0day) Aaron knows perfectly the reviewing times at ZDI with some of my bugs for which I received an offer after MORE than 9 months (yes "0day"), so talking about sitting on the bug for long time is really inappropriate. I have complained tons of times about the absurd times at ZDI and Aaron knows it perfectly because I was in contact with him almost each week. And now more chaos. I asked a statement from ZDI and late yesterday I received some lines where ZDI states something completely contraddictory: =================================================================== From: Luigi Auriemma To: zdi@tippingpoint.com Subject: Your statement Date: Thu, 30 Aug 2012 02:56:54 +0200 X-Mailer: I'm waiting for your statement about what happened with Aaron and aluigi0216. You know exactly about what I'm talking and I want your official statement about this thing. Please don't find excuses, this is a serious matter. Very serious. --- Luigi Auriemma http://aluigi.org =================================================================== From: Zero Day Initiative To: Luigi Auriemma Subject: RE: Your statement Date: Thu, 30 Aug 2012 22:48:36 +0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Luigi, ZDI's records reflect that this vulnerability was discovered and submitted internally prior to your submission in March. In regards to why your case was not closed as a duplicate, we are looking into it. - - The ZDI Team - -----Original Message----- - ... cut ... =================================================================== From: Luigi Auriemma To: Zero Day Initiative Subject: Re: Your statement Date: Fri, 31 Aug 2012 01:27:49 +0200 X-Mailer: Aaron claims that his case was closed because "unexploitable" and so he owned full right over it, in this case you can't close my case as duplicate because you don't have another primary submission. Aaron is very clear on this matter: The bug at ZDI was closed as unexploitable. Any bug closed at ZDI remains the submitting researcher's property. Secondary, are you ZDI or ExodusIntel? How it's possible that a bug owned by ZDI (false, you owned nothing as Aaron confirmed with both his words and the facts) is arrived to your competitor ExodusIntel? Then I have an agreement with a point 4.1 which has been violated because you state that during the evaluation period you and your employee agree to not disclose information about the submitted bugs. NOBODY except you were aware of the bug on which Aaron was working internally. Additionally point (E) isn't affected in this case because you had no rights over that bug. Maybe you can find more interesting to read my doubts in the second section here: http://aluigi.org/misc/aluigi0216_story.txt I want to know what happened without covers between you and Aaron, you are my direct reference and so I don't care about what Aaron did: my situation is that a vulnerability I submitted to you in full trust for 5 long months has been sold by an external company owned by your ex-manager and I have received nothing, not even the respect of being warned about the situation of the owned/non-owned bug Anyway judging your reply seems that Aaron is not much out from ZDI, so probably I need to ask explanations to HP. >... cut quoted e-mail ... So: - ZDI states that the bug is owned by ZDI and claims that their action would have been closing my case as duplicate - Aaron stated that his bug was closed at ZDI when he left - now Aaron states that he found the stack overflow so it's not the same format string I submitted at ZDI Most of the doubts and questions in the previous section of this text are still valid, the Aaron's reply has added only more doubts to this soap opera. So can someone tell me what people do at ZDI? Why ZDI and Aaron claim to own the same bug which in reality it's "half" or even not the same bug? I have submitted a vulnerability in order to receive an offer or a rejection and now, after all this senseless chaos, even the involved parts Aaron and ZDI say the opposite. I only asked an explanation with a simple tweet and I have more doubts than before. I'm external to ZDI so anything happening inside ZDI is unknown to me and to any other person, I can judge only what I see and what happened and there is something weird and wrong, maybe just a coincidence or some errors of ZDI to review submissions but in any case ZDI (with or without Aaron) did the chaos and I have lost my submission. If I see that the manager of the company to which I gave my submission publishes the same bug (or bugs fs+bof) and instead of giving me valid explanations, replies with his team in a unprofessional manner, that's not fair. Because what happened and happens inside ZDI is ZDI business only. #######################################################################