Interview: Ashay Chaudhary on the SQL Server JumpIn! Camp
Well, December did finally slow down a bit and I had a chance to catch up with Ashay Chaudhary, the Program Manager for the Microsoft Drivers for SQL Server for PHP. We had a lot to catch up about because we haven’t found much time to sit down and chat since the JumpIn! Camp last November. Ashay, with the help of the esteemed Josh Holmes, brought several PHP developers from several countries to the northwest corner of the U.S. and put them in a room with a bunch of Microsoft developers for a week. I was there for most of the week, so I have a feel for how things went. However, I was very interested to sit down with Ashay and get his perspective on the camp…
Brian: I was at the latest JumpIn! Camp, but for our readers can you tell me what JumpIn! Camp is and who was there?
Ashay: JumpIn! Camp is a relatively new effort from Microsoft – a new way to intersect Microsoft technologies with PHP open source applications, a new way to foster a mutually beneficial relationship between these two groups. At these camps, there are engineers/developers from some Microsoft web related technologies and several PHP applications/projects. No marketing stuff, it's all about code. And some healthy discussions on various topics, of course.
Microsoft has hosted 2 JumpIn! Camps so far, one in Zurich and the other in Woodinville near Microsoft's Redmond headquarters. Since the concept is relatively new for Microsoft + PHP, the 1st camp (in Zurich) was more of an exploratory event where developers from various projects like Doctrine, Agavi, CakePHP, Joomla!, phpBB, Moodle, osCommerce, Drupal and Project Sonar learned about Microsoft web technologies and then could use them in their projects if they thought it would add value to their applications. The Microsoft experts on those technologies would often help them leverage the technology for their scenarios.
At Woodinville, the focus was more on completing a sound foundation – just core technologies of the web stack and then Azure, Microsoft's cloud offerings. This time there were seven engineers from SQL Server, and a couple from IIS and Azure, who provided the necessary guidance/expertise to the PHP developers so that the applications can run on IIS and/or SQL Server. At this event, core developers from Doctrine, Kohana, Lithium, osCommerce, MODx/xPDO, XHGUI/XHProf, ImpressCMS, TangoCMS, Typo3, and Formulize worked really hard to understand the platform, and then how they could leverage its strengths.
Brian: Who’s idea was JumpIn! Camp and how did the idea become a reality?
Ashay: Microsoft has been working with the PHP community for a few years now … folks from the PHP community and Microsoft would get together and discuss what needed to happen to build Microsoft platform support into PHP applications, and then go work on the items agreed on separately. After a few cycles of this method, we had made good progress but it clearly wasn't enough. During a recent event in December 2009, Sam Moffat (from the Joomla! core developer team) expressed interest in exploring support for SQL Server to Joomla! (along with many other Microsoft technologies) and we managed to carve out some time from very busy schedules. The progress made when two developers with expertise in their own areas sat down together undisturbed for about half an hour surprisingly exceeded expectations. Little did I realize that this would be the seed of what was to transform into the SQL Server Jump In! Camp.
As SQL Server is a very popular product, one of the most widely used databases, and there have been a lot of asks from PHP developers to support SQL Server in their applications. However, I observed that many of the PHP projects were running into challenges in supporting SQL Server as they were unfamiliar with it. But they didn't have access to the SQL engineers like Sam did. Most developers with a passion for the technology they work on love to see their technology being adopted and used, our team wasn't any different. The seed was already planted, and soon developed into the idea of an event where motivated expert developers would come together and coordinate their expertise to get working code. Since I didn't find similar events out there to learn from and model after, I discussed the idea with several folks within the company that work with the PHP community and received great feedback. I factored this feedback in designing an event model/framework that I felt would succeed.
At the same time, our Developer Evangelism folks in the field were also looking for ways to take their engagements with the PHP community to the next level and, coincidentally, arrived at a similar conclusion. Since they work with several Microsoft technologies, they put together an event where key Microsoft web application related technologies were represented by their respective experts, and several developers from various PHP projects got a chance to try them out with the required assistance – the Jump In! Camp at Zurich (read more about it here). It was an extremely well organized event and incredibly successful. Having represented SQL Server there, I observed keenly and identified the key elements that made the camp successful and applied it to my model/framework. While there were still a lot of unknowns, I had sufficient confidence that it would be successful with good execution.
Ideas by themselves don't amount to much without energy. That is, one has to act on them. I pushed my idea to my management, and was very fortunate to gain their support quickly.
In the summer we were heads down finishing up v2.0 of our driver where we added our PDO driver for SQL Server (see my blog post for additional details), so this sat on the backburner for a while. Once we had our v2.0 released, I was soon free to start planning for my event. Some of the unknowns were relatively easy to plan for, but others were complete guesswork (and I really had my fingers crossed for a long time). In many ways, it felt like this 2nd camp was completely different from the first, like the 2nd camp was a completely new journey. Yet, it was a maiden voyage that I was motivated to launch, fly, and land successfully.
Brian: Would you consider this JumpIn! Camp a success?
Ashay: No matter what I think, it is far more important what the participants think – both, Microsoft and PHP developers. And the feedback I've received is that it exceeded their expectations. I concur, but that's secondary. It wasn't perfect; there was a misstep or two but nothing catastrophic.
What's more important is what made it successful. Fortunately, I had some vacation time accrued that I really needed to take after the camp and gave me the time to decompress and introspect to ponder this question. To me, it really distills down to the following…
Right attitude. Some participants probably had concerns as this was the first time they were interacting with the evil empire at a venue disconcertingly close to the death star. I must give them all credit for being receptive and really keeping an open mind without compromising their integrity. Open to listening to others, open to sharing their thoughts, willing to learn from others, willing to educate others, willing to exploring this new trail we were on. I should also add that PDO was not on the radar for many of the projects, but many were ready to consider it for a better multi-database solution and the overall value it provides to their project.
Right environment. I can't expound on the environment enough. Sure the hotel was really, really nice and the food really good, but there was much more to it. The informal yet collaborative environment, "work hard play hard" ethos, and the very minimum of legal constraints really helped the participants come together as a unit that marched ahead at a fantastic velocity.
Right incentives. You heard me right, incentives. But hear me out, I don’t mean buying loyalties. Essentially, the right incentives are very important for motivation, and motivation is an incredible energy source. So having the right incentives is crucial. First, it is really critical that everyone benefits from the effort. While those skeptical, or even hostile, to Microsoft will see that only Microsoft benefits from this, that couldn't be further from the truth. Windows and SQL Server have a very broad reach, and many companies are not comfortable introducing new environments in their network for a PHP application that fulfills their needs. Adding support for SQL Server and IIS helps the PHP application reach a huge server/user base at enterprise scales. Microsoft benefits too with more licenses sold, and the users/companies benefit when they can deploy the PHP applications of choice on their infrastructure and fulfill a need. It also enables the application to grow from using just the fundamental capabilities of SQL Server as a relational database to leveraging additional capabilities of the product like high availability, replication, manageability, and business intelligence.
Right people. This is probably the most important part of the equation – without the right people present at the camp, none of the other factors matter. Folks from the application community with the right level of expertise in the database and web server components of the application, and the right experts from SQL Server and IIS. These in turn need to have sufficient critical mass in each project (and the camp overall) to provide the balance required to make good decisions quickly and sustain momentum.
The thing to note here is that we had a good amount of these to begin with, and the positive energy from everyone increased it even further.
Brian: You alluded to a willingness to both learn and teach on everyone’s part being a key to the success of the camp. What were some of the things that you personally learned during the camp?
Ashay: Personally, I was surprised at the effort it takes to make an application multi-database capable when it was not designed for it from the beginning, the development environments and preferences in the PHP community, and the effectiveness of our interoperability tools.
There were obstacles to overcome with schemas, APIs, SQL syntax, and web environment configuration. The SQL Server Migration Assistant made it quite a breeze to migrate schemas and "complete" queries – while most were done within an hour or so, some folks needed to correct their existing schemas/queries and then re-run the migration tools. Some knew which driver they wanted to use, native or PDO, which made the porting task somewhat easier. I was a little surprised to find some folks looking for an equivalent API when the job is done by an SQL query – something quite obvious to those familiar with SQL Server, but not for those coming from other databases. I also learned a lot into the coding styles and expectations of the PHP developers, something that will help us in our API design in the future. I was also surprised at how easy it was to connect to SQL Azure. Not only did the migration assistant help with creating resultant schema/queries that would work with SQL Azure, the changes required in the values fed to the _connect APIs were trivial.
The effort to migrate SQL syntax was definitely not trivial, and I was surprised at how much effort it was for some. This was particularly evident for those that concatenated a query on the fly – since our migration tools could not be leveraged in this dynamic scenario, the effort here was quite laborious.
While I anticipated most of these obstacles, having the right people in the room to answer unanticipated questions helped us make great progress. I guess I am saying that the learning was in finding out the exact position of the needle on the dial. Now I know how to categorize the projects and what velocity I can expect from each of them, and thus have a better agenda planned out for supporting PHP applications.
I learned a lot from the first-hand view into the tools and working styles and preferences of many PHP developers…why they do things one way vs. another way, what they typically look for, and so on. Of course, 17 PHP developers cannot accurately represent the whole community, but it's a great starting point. Such insights are invaluable in getting the "softer" and "intuitive" parts of design right, and I look forward to using them in the future.
Brian: I know you can’t speak for the PHP developers that attended the camp, but what do you think (or perhaps hope) that they learned during the camp?
Ashay: On the technology side of the equation, I hope they learned about how SQL Server & SQL Azure works and how to get good value out of it. For example, we spent some time discussing and demonstrating the query plans created by the query processor/optimizer, some best practices like using parameterized queries, prepared statements, stored procedures, etc., and then how to leverage this in their projects. Some had a chance to understand and discuss features like Full Text Search, Semantic Search, Spatial, and some features coming in our next version, Denali (like the OFFSET clause defined by ANSI).
We also had the opportunity to use some great tools for SQL Server: the Migration Assistant mentioned in a previous question, SQL Server Management Studio which is probably the most used one, the AutoProfiler tool which provides tons of valuable information for the developer or database admin. Of course, last but not least, we also got the chance to use some of the IIS tools for PHP including the PHP Manager, Web Platform Installer, WinCache, Windows Azure Companion, etc.
Most important, however, is that they got a chance to see the human side of Microsoft. At the end of the day, we are people just like everyone else and that we are technology "nerds" too. We love to take on challenges; we are passionate about the technology we are working on, we learn from our experiences and apply it to making our products better for developers, admins, and end-users.
Brian: Would you say there were some common lessons learned?
Ashay: Of course. Like I said, we are developers at our core and thus have a lot in common with other developers – technology "nerds" that love to write code, love to see our code/product solve real problems and become popular, love to learn new technologies – to varying degrees, of course. We learned that we have our differences in philosophy and the way we do things, but we also learned that when we manage these differences like professionals, we can actually work well together. Open source and proprietary source can co-exist and interoperate well if we decide to do so, and that developers from both camps can work together effectively and can mingle socially without compromising our core ideals.
Brian: On the first day of the camp, I missed the Nigel Ellis’ talk about two historical moments in the evolution of SQL Server. I later heard that was an excellent talk. Would you consider it a camp highlight? What were some other highlights?
Ashay: Nigel is an incredible person; this was the first time I had the opportunity to hear him talk about his experiences and journeys in SQL Server. Having been in this relational database space for over 17 years, the depth and breadth of this experiences and knowledge is unbelievable. Nigel chose to highlight 2 major events in SQL Server's history – the big SQL Server v7.0 rewrite where 80% of the code was re-written while still maintaining the backward compatibility guarantees our customers demanded, and how our enterprise experience evolved into the design and implementation of the Database as a Service (DaaS) - SQL Azure. I would certainly consider it one of the highlights of the camp. Many of the participants were quite fascinated by his stories, and got a small insight into what it takes to build world class software.
Let us face one fact, when you get talented and motivated developers in a room for a few days, more code gets written! So another highlight is the passion and dedication demonstrated by all the developers. The fact that most projects could demo at the camp itself, and that many projects completed their work at the camp itself (or expect to do so soon) is something all the participants should be proud of.
Another highlight is the osCommerce solution. Harald and Mark had already started work on their v3 code with its own database abstraction written. I was very impressed with their ability to re-evaluate their design for their long term goals, accept the finding that they needed a better abstraction layer, evaluate other solutions including the frameworks present in the room, and then design a new finely tuned database abstraction layer to make best use of each supported database. In the case of SQL Server, they implemented their core queries as optimized parameterized queries (or stored procedures), using the query plans to optimize the query syntax. They also went ahead and wrote their first automated unit test suite!
By mentioning osCommerce only I am probably doing a disservice to the other projects that also did some phenomenal work, I'll save these for later.
Brian: What aspects of the camp didn’t go as well as you’d hoped? What would you have done differently?
Ashay: This is a great question Brian, there were two occasions that made me cringe inside and a few small issues that we corrected right away. One of them I could address right away at the camp itself, one needed to be done during the planning stages itself. A few suggestions came thru written feedback after the event, and we'll certainly factor these and other lessons learned into our planning.
As I said before, there were a lot of unknowns. While I knew that all would be at the same stage at the beginning, but we had no data to make reasonable projections on the rate of progress over the next couple of day. Based on interest expressed on Day 2 on some of the SQL Server features (Full Text Search, Semantic Search, Spatial, Reporting Services), we got the right Microsoft experts in these features to show up at the venue on Day 3 – those who had completed the basic relational data storage part could understand, discuss and prototype these features. A last minute change in the Reporting Services expert, traffic delays and demo gremlins made this session a truly forgettable experience - I wish I could turn back the clock and make the required changes.
However, it wasn’t just the demo gremlins that were a problem. It became clear at some point that the intense desire to finish and demo applications and have these feature presentations in the same room became a challenge. Participants may have been interested in the presentations, but they also wanted to write code. I quickly found another room at the hotel and moved subsequent presentations to it – those interested in the topics could then have the in-depth discussions on these features without distracting the others.
Another issue was topic selection for presentation/demo/discussion. It would have been good to create a list of these and then get the participants to vote, this way the content up for discussion will be more appropriate. I received this feedback after the camp, so this is definitely something we need to do in the future.
Lastly, one of the consistent pieces of feedback I received was that the overall agenda was too intense. Many of the projects were together in a facility at the same time (and the likelihood of the same group coming together again is quite low), so they'd have loved one free evening to just mingle and learn more about each other and their projects.
Brian: Anything else you want to add?
Ashay: Yes, absolutely. I've had a few queries on when we will have the next one, and many have expressed interest in attending it. First, I was out on vacation for most of the time after the camp and am in the process of collecting all the feedback and figuring out next steps.
However, I'd like to iterate that the two camps were the beginning of a good relationship with the developers from these projects, and we need to foster them for mutual benefit. We are off to a great start and it will be a shame to lose the momentum.
I'd like to thank several groups. First, all the participants for being really awesome – it was these folks and their attitudes that really made the difference. Second, my management for believing in me by supporting it, and then trusting me to do it right. Third, Josh Holmes for being a terrific partner and colleague. And last but not least, Gloria Sanchez (our event manager) for her attitude and energy in making this a stellar event.
Thanks Ashay! I look forward to similar events in the future.
If you’d like to know more about Ashay, check out the interview I did with him last May: Interview with Ashay Chaudhary, Program Manager for the SQL Server Driver for PHP.
Thanks.
-Brian