With the new OpenSearch-based federation capability in Search Server 2008, you can integrate any external search service that can expose results as an RSS feed. In this podcast Jon Udell discusses search federation with Richard Riley and Keller Smith.

Richard Riley is a Senior Technical Product Manager for Microsoft Office SharePoint Server 2007. He is responsible for driving Technical Readiness both within and outside of Microsoft and specializes in the Enterprise Content Management and Search features of the product.

Keller Smith is a Program Manager in the Business Search Group at Microsoft. He designs and manages new enterprise search features in the areas of Federation and End-User UI. His passion has always been to improve the lives of users through exciting new ideas in software.

Links

Enterprise Search Blog Search Gallery Location Definition File Schema

Q: What's the lineage of this search server?

A: The technology that was built into Index Server, way back in the NT4 option pack, has grown and diversified into various products, including desktop search and SharePoint. They've split apart now, but the common DNA is there.

Q: What differentiates this search server from its predecessor?

A: We found that customers wanted to use the search capability without buying the whole SharePoint product. So we split the search features into Microsoft Office SharePoint Server for Search. People could buy that and use the search features without the full MOSS functionality. Search Server is the next version of that.

Q: What were the domains over which MOSS 2007 could search?

A: Anything you could crawl. Out of the box, SharePoint plus other content sources we had handlers for, including Notes. Or you could go to the effort of writing your own protocol handler, or business data connection. But if you couldn't find a way to index it yourself, there was no way to connect to the data.

Q: So how does federation change the game?

A: Instead of indexing the content, you're leveraging an external search engine that already exists. That engine returns results back in an XML format we can render.

Q: I was fascinated to learn you're using the OpenSearch mechanisms and formats to accomplish this. I did an early implemention for Amazon A9, and it was trivial since I already had an RSS feed coming out of the search engine I wanted to integrate. Is that still how it works?

A: Yes. Any search engine that emits an RSS feed, you can connect to. It takes about 5 minutes to set it up. You take the query URL, put in into a federated location definition (FLD) file), and away you go.

Q: I guess the part of OpenSearch people will be most familiar with is the description that drives the search drop-downs in browsers. It's a little package of XML that defines the template for the query. You must be using that in Search Server as well, when it acts as a client to federated sources.

A: Yes, exactly. SharePoint is behaving as a client, just as IE is. When you create a federated location definition, you're creating one of these OpenSearch description files. But, we add some schema changes for the triggers that SharePoint uses to know when to send queries to that location. And we add the XSL used to render the results. So we extend the OpenSearch schema to make it more useful to SharePoint.

Q: When you start shipping queries over the net to multiple federated sources, you start running into issues of sequencing and latency. How do you deal with that?

A: You add federated locations as web parts. And you can choose whether to load them synchronously or asynchrously. Everything synchronous will be loaded first, and then the queries are sent off to each asynchronous web part.

Q: And you'll use AJAX to weave in results in as they arrive?

A: Right.

Q: One of the sources can be SQL Server. How does that work?

A: You need a simple connector that exposes an RSS feed.

Q: In the case of SQL Server, there's the option to do structured search. Can I pass through an XPath query?

A: Well, it's up to you to write the connector. If you want to accept XPath in the query, and return results on that basis, it's your code.

Q: What I like about this is that the act of creating an OpenSearch RSS feed on top of a source is just plain useful, independently of Search Server.

A: Absolutely. We use that in SharePoint Search, and also in Search Server, you can get an RSS feed of any result set. It's great for alerting. Set up a fairly restricted search, and your RSS reader will get new items when they appear.

Q: It's great that you're using OpenSearch this way. Was there any debate about it?

A: There are many ways to connect to other sources, but we felt there was a need to federate out in a very lightweight way. OpenSearch already had a scheme that was relatively well adopted, and served our needs as a base, though we did extend it as I've mentioned.

Q: How do I control the results display?

A: You can customize the XSL, so anything you can retrieve from the source you can format in any way you want.

Q: Can I extend the results metadata?

A: Yes, you can override the OpenSearch defaults, specify which fields you care about, and use those in your XSL.

Q: And, Search Server is free?

A: Yes, just go download it from microsoft.com/enterprisesearch.

Q: How far can you go with the free version?

A: You can install the express version with either SQL Express or SQL Server. With SQL Express you can run up to 400 to 500 thousand documents. With SQL Server, you can run to millions.

Q: What about federation? Will there be a cap on the number of sources?

A: No limit on sources. The only difference is that the express version requires you to install all the search services onto a single server. With the licensed version you can spread those across machines.

Introducing Live Mesh

In this audio version of a Channel 9 video, Ray Ozzie discusses his role as Microsoft's chief software architect, and the role of Live Mesh as one aspect of an emerging Internet-oriented platform.

Ray Ozzie is Microsoft's chief software architect.

Links mesh.com Video of this interview on Channel 9 Abolade Gbadegesin on the architecture of Live Mesh Demo of Live Mesh on Channel 10 Mike Zintel on Live Mesh as a platform Background on FeedSync

JU: Hello Ray! Thanks for joining us.

RO: It is great to be here Jon.

JU: So, it's been about 3 years since you joined Microsoft, initially as CTO. People tend to wonder what it's like coming from a company of 300 to a company on the scale of Microsoft.

RO: I've had the luxury of career working for small companies: Software Arts in the early days, and a couple of startups in Iris and Groove. But Lotus ended up being acquired by IBM, so I was at one big company before coming to Microsoft. It's tremendous in terms of the potential impact that someone can have. I think everyone at Microsoft tends to be here because you want to have a tremendous impact, and certainly that was a tremendous draw.

What I really do enjoy about the role as CSA, is being at the juncture of business strategy, product and market strategy, and technical strategy. I have the opportunity to work with not only the executive team on larger strategic issues, but also with the product teams at fairly detailed technical architectural levels. As an engineer, it is really fascinating, and I've met a lot of great people.

JU: People also wonder what it's like to step into a role formerly occupied by Bill Gates. What kind of continuity will there be, and how might you want to reshape the role?

RO: Bill is a very unique individual. There will never be another Bill. He has got a tremendous palette of talents, both technical as he applied at Microsoft, and non-technical in the role he's moving into. In shaping the role after July, when he won't be here full time anymore, he really split the role into two pieces. Craig Mundie takes over long-term issues, research and things like that. And I have taken over most of the technical and product strategy related to products that'll ship within a couple years.

My background is different than Bill's was. I've been a lot more hands-on in the product design for a number of years. I'm dealing with broader issues than I've dealt with in the past, but my background in product development gives me a lot of grounding in terms of working with a development team. And I think Craig and I make a good pairing in terms of filling his shoes.

JU: How do you balance the need to span a vast spectrum of activities and the need to go deep on things?

RO: Time management -- attention management -- is really the biggest challenge. The pace is fairly brutal. At the beginning of the year, I'll kind of plan out how much of my time in hours I want to spend in different categories of things. There's some allocation for the rhythm of the business and high level strategic things. There are allocations in terms of time I want to spend with product groups.

And then there's a fraction I didn't initially realize I had to be as intentional about, but sometimes you have to create white space because, like a task scheduler that has too many ready tasks, you can thrash if you spend all day dealing in a reactive mode to the incoming issues, the incoming communications. Sometimes you have to create some white space in order to think and understand what is going on in the environment. I can do that by going away, by traveling to our international offices. Bill had something called Think Week that we are continuing in a slightly modified form going forward. And there are other ways.

JU: Is one of those ways maybe to sometimes focus deeply on particular interests of yours? If so, what would some of those be?

RO: Well, the problem is most of my interests are technically related and so in theory I would just go write some code. I don't do that anymore, though, and honestly the best way that I've found to clear my mind really is either to go to a conference that's a little off the beaten path, or just travel somewhere, maybe with my wife, that is not technology related, just to clear it out and re-prioritize. It is probably something that everyone has to do. In the old days when I did code, I used to have a 4-hour rule that said: "Do not write code unless you can at least have 4 hours of contiguous time where you will not be interrupted." Otherwise you end up introducing more bugs than the code you are writing. In a way, this is kind of the life management equivalent.

JU: So, in the talk that you gave at MIX, you introduced the interesting phrase "utility computing". I got to thinking that although "web 2.0" is the meme of the current era, people may have forgotten that for quite a while, Tim O'Reilly was actually trying to establish "Internet operating system" as a meme. That didn't really stick, and now it's come around again as "web 2.0", but "Internet operating system" is a pretty evocative phrase, as is "utility computing." We have talked about some things that are coming. We're going to talk more about a part of that initiative here, the Live Mesh announcement, but I wonder if you could reflect a little bit on what an Internet operating system could be.

RO: Maybe I should just step back and talk about the environment a little. I mean, when I got into this business back in the 70s, utility computing was really hot, it was called the mainframe. We had these raised floors, freon cooling. It really was a utility. We used virtualization. There was "time sharing". They were boring terms, but we were used to treating computing as a utility, and the PC revolution was all about empowerment and kind of getting back some of that personal feel, getting control of building solutions for things that might be really meaningful for you.

So the pendulum swung to the personal, and then with the web, when the web first emerged, it's odd, the nature of how technology is shaped is based on the constraints of the environment, whether it is computing constraints, communication constraints, and so on. The early web grew up in an era of dial-up, of 56K dial-up, so a lot of the way the protocols are structured, where computing was located, were based on that balance of computation and storage on the back-end, a really thin straw, a smart terminal that we call a browser on the front end, and that's how it was born.

Nowadays, we've got increasing ubiquity of broadband. We've got this big fat pipe so we can send more data. You still can't be chatty, but we can send more data back and forth, and it gives us, as architects, the ability to revisit what should be the right balance between, for any given solution, of what's on the back end and what's on the front end. We have amazing computation abilities on both sides. We have amazing storage abilities on both sides. So now, in this unconstrained environment, really the question is: what is the right way to build a solution? Application models have had begun to evolve that start to take advantage of some of these things on both sides, and I think really when we talk about utility computing now, what we are saying is, if you are building a solution now, what is the right way that back-end utility should expose its resources? What business models? What application design patterns are appropriate for the cloud? Map-reduce-like patterns, pure horizontally scalable patterns, are much better for that back-end.

What should the front-end programming model be like? We started with the PC in a model of one computer for some subset of users. Bill and Steves dream was to have a computer on every desk and in every home, and we have gotten into that point, but now we have gone beyond that point. Every individual has a phone and a PC. Many people have multiple PCs at home. They might have a PC at home, at work. We have got computer-like devices in our cars, sitting underneath our TVs with the set-top box. People have home security systems. There are lots of devices around, and I think now is the time to reflect, what is the right programming model for the client environment that we have got? What is the right programming model for the cloud? And at least from Microsoft, how can we built tools and services to help developers build great businesses, to build great solutions, using both those back-end and front-end resources?

JU: From that perspective, we see some interesting offerings emerging in what we can broadly call the Internet operating system space. Amazon surprised me quite a lot in the last couple of years in the things they have done. I don't think people were too surprised by the Google announcement more recently. I would invite you to reflect on the kind of company that Microsoft historically has been, and therefore, the kind of approach that Microsoft can take to this problem as it might compare to the kind of approach that these other companies can take.

RO: I have no products announcements to make. [laughs]

JU: I understand that. [laughs]

RO: I'll just reflect in our approach, and compare and contrast if you like. Microsoft's approach...I can tell you this because I was a developer for most of my career on the outside of Microsoft...I've been here for 3 years, but I had a relationship with Microsoft as an ISV since roughly the beginning the Microsoft. I met Bill and Steve in 1981 when I was first coming out to talk to them about some DOS issues. Microsoft, I believe, has always taken a perspective of ... its DNA is as a platform company. And in order to have a successful platform, you've got to have successful ISVs, people who are being selfish about their solution, what they are trying to deliver, but we have to -- semi-selflessly and semi-selfishly -- serve those people. We've got to build a good business, but we've got to do so by serving those people and letting developers build great businesses. So in any platform, any utility computing environment that we would consider, we would be taking a broader perspective.

We would look at a 20- or 30- year horizon and say: How is this all panning out? What is the broad range of developers out there? What does the new-age ISV look like? It's a web ISV. There are also client ISVs, but client code is changing, it has cloud interconnections now. What does a VAR look like these days, a solution VAR? What does an enterprise developer look like? What is the enterprise environment going to look like when it's transitioned from an on-premises data center to one that factors in both an on-premises data center and the cloud. Perhaps there would be some businesses, small to medium size businesses, that might shift completely to the cloud for their back end. But most major enterprises would have some kind of hybrid. So when we step back and look at tools, languages, application design patterns, operating systems, and runtimes, we kind of look at it and say: How will we design this for the way that the environment will shape over the next 5, 10, 20 years? As opposed to what does the web look like today, what are the capabilities today. I think Amazon has done a great thing in terms of opening people's eyes to the power of, coming from the ground up, what does it look like to make raw resources, raw VMs, or blobs, available to a developer. I think they've done all of us a great service, and themselves. Google's recent announcement, I think, is actually the inverse. It's done a good service in terms of looking at an individual developer and saying: "Hey, for a specific problem, what is a very simple way of getting into this cloud game with a relatively constrained pattern and model, but doing it in a fairly slick, seamless way." I think those are both interesting viewpoints and ultimately the answer that the broad developer audience wants will be a combination of those and many other things.

JU: Good. So in that context, we have just announced Live Mesh, and when I first saw it, I worried a little bit that people would see it in comparison to a lot of things which on the surface, it compares to. It can look like a FolderShare kind of thing, it can look like a screen-sharing kind of thing, it has those aspects. But in fact, this is one example of some platform-like capability for which those things are really trivial applications that have been layered on top. We can talk about Live Mesh, so let's talk about it.

RO: Live Mesh began with the perspective of saying, what is the environment that we are in today, and that we'll be in for the next who knows how many years. As users we're in a multi-device environment, and we need to cope with these many different devices. Each one of us at home ends up being a kind of system integrator, if you want to get simple media sharing scenarios done between devices at home. You might have different contact-sharing things between your phone device and other things that you are dealing with on your web or on your PC. If you're in a productivity context, you have document-sharing scenarios both among people and among devices.

Each one of use has had challenges, if we have multiple PCs and multiple devices, figuring out how to get the most recent version of that application installed on all the right devices that we're the system managers for. On the enterprise side, we've solved this quite well with things like SMS that lets an enterprise push things out to many desktops and manage desktops, but we haven't actually solved that problem for individuals. It hasn't been a huge pain point for individuals, but now it's becoming more of a pain point.

That's one aspect that started us down the path of Live Mesh. We basically said, the OS as it is right now, the OS for the phone, the OS for the desktop, the Xbox, the OS for a Zune, the OS for the PC, are all designed more or less to expose the resources of that device to developers and to users, but they are really not designed in concert with other devices. What is going on in the web is mostly done serving the web, and the browser is largely disconnected from those devices. If we were designing an operating environment for users or developers today, looking forward, it would probably look a little bit different. It would look like something that would bring those devices together for the end user. And so that is one thing that Live Mesh does. It brings together your devices. You use the web as a hub to claim your device. You securely identify yourself as an authorized user of this device. Multiple people can own a device as authorized users and each person can have many devices.

Once you've said that's your device, it enables many things. It enables centralized health monitoring and status reporting. It enables settings replication across your devices in computers where you think appropriate. It lets data flow among those devices, whether files and folders, or other things that I will talk about in a minute, like feeds. And it lets applications be configured and potentially licensed across your device mesh.

And in solving the problem of getting things to work across your devices, the same kind of technologies can be used for multiple people. So if you share a folder of documents, if you are working on a set of documents on your desktop with someone else, those same technologies that are used to synchronize that folder across devices can be used for me to share with you or other people. So from the user's perspective, we think that Live Mesh can really transform your experience with multiple PCs and things like your phone to make the experience very seamless in that way.

Now let me just come from the developer's perspective, Live Mesh is actually a platform. What you see with Live Mesh when you download it is a very small piece, from the user's perspective, of what it actually is, because it was built to enable innovation in a variety of ways. You can kind of think of what you see as the shell. If Windows or an OS has a broad sort of capabilities that is exposed by its APIs to developers, the shell, the command line of an OS or the Finder or the desktop within Windows is a thin exposure of that to users. For Live Mesh, file and folder synchronization is that small amount that gives the user a taste for the capabilities of this platform.

JU: You've talked with me before about a couple of distinct application patterns. I think one of them is going to be intuitively obvious to people because the folder and file synchronization thing is something that people already do. So people are going to kind of get that if you drop a thing here, it shows up there, and hopefully they'll be delighted to find out that subtler kinds of things than whole files and folders can also participate in that synchronization. And they'll be interested to see how they can then bring people into the equation, with sharing. All that I think is what people will get at first glance. I think what will be less obvious is the way in which websites can use Live Mesh to optimize the communication of stuff down to individuals and groups of individuals. Since that's less obvious maybe we should take a moment to spell that out.

RO: Sure. You can look at it from two perspectives. Live Mesh is a way of enabling rich applications on the PC to get their settings and their data across devices, as you just said. But it's also a way for websites to be able to efficiently extend their function down to a world of devices. The PC for sure, but also phones and other devices. One of the things that we inadvertently stumbled upon in Groove was that enterprises wanted to use this technology to help them extend the functions of their websites out to a world of devices. That isn't what Groove was designed to do. It was more designed as a peer sharing mechanism. So one of the things that Live Mesh is all about is essentially, from day one, providing a centralized infrastructure such that this platform that's on all of the clients goes to this one service in the cloud to manage, all under the covers, all the synchronization. Now the actual data may flow peer-to-peer, it might flow relayed through the cloud encrypted, but one thing that is for certain is that an arbitrary web site won't have to deal with the complexities of synchronization. They can develop an application, using technologies that they are familiar with -- web development technologies -- and develop a piece of that application that gets downloaded to the client, that has local storage synchronized with the web site, they can update the application and the updates get distributed transparently...

JU: Or maybe it's just data.

RO: Yep, could be data.

JU Let's talk about my bank's web site and my travel web site, two websites that I frequently do business with. In both cases there's data exchanged, and I would love for that data to be exchanged in a fully synchronized and reliable and transparent way. What you're saying is that both of those web sites, and any other of web sites that I transact with, can pretty straightforwardly get into the game of plugging their pipes into my mesh.

RO: That's right. They can plug their pipe into your mesh, and it's through mechanisms that most web developers these days are becoming increasingly familiar with, such as feeds. Some people might remember a few years ago there was a technology that we introduced, an extension to RSS called SSE (simple sharing extensions), that eventually matured -- with the help of the partner community -- into something we now refer to as FeedSync. It's essentially RSS and Atom extensions to a technology that was initially developed for publishing, where you have a list of items that get updated, and through a publish/subscribe mechanism, the updates get sent out. FeedSync extends that to make it bidirectional. You can essentially do crosswise subscriptions. I subscribe to you and you subscribe to me on the same feed. We can both modify it, and make sense of the results, and understand how conflicts are dealt with.

By using this very simple technology, we connect the web site to our cloud, our cloud to the clients, the clients to each other. It is just a very simple thing. The base model of what is an application within the Live Mesh environment begins with essentially a feed of feeds. One feed represents a logical thing that a site might be exchanging with the client. That's called a mesh object. It's a feed of feeds. A developer can new up one of these things, and its elements are other feeds. An application can develop as many feeds as it likes. Some of those feeds are hard-wired to be things like the list of members, or the list of devices in the feed, but then the application can develop many more.

JU: The way I'm thinking about it is that the sub-feeds are basically custom objects. If it's a calendaring application, those might be calendar events. In banking applications, they might be transactions. But the notion is that the same infrastructure that's synchronizing files and folders can also synchronize these custom objects in the same way.

RO: That's exactly right. In what they see today, if you open up a folder, what we would refer to as a mesh-enabled folder -- it's one of these mesh objects. And in essence every file within that folder is an element, it as an item within the feed. The file itself is the enclosure. The metadata of the file -- its name, its modified date and so on -- are a standard schema that represents the item. Then there's a news feed that you'll see on the right hand side if you open up one of these folders, that's another feed, and each of the entries in there is another item, and so on.

We expect that developers will develop feeds that suit the needs of their specific application, and we deal transparently with the synchronization of those elements. The user interface offers a very simple consistent way to help users manage conflicts -- if the application says that the user should be the one to deal those conflicts.

JU: There are a lot of interesting degrees of freedom here. My bank's website has a RESTful interface to this stuff, but so does my mesh client. In fact, I think people will be surprised and delighted to discover that you can hit the localhost with REST calls -- and that's putting stuff in, as well as getting stuff out.

RO: Right. We made a decision, from an API perspective, that developers would prefer to learn one way of dealing with the mesh, and that the tooling would be easier if we had one way of dealing with the mesh. So the web version of Live Mesh, what's running up in the web, and what's running on the client, are symmetrical and the same code. So on localhost, in a secure way, an application uses REST calls to invoke -- we call it MOE, the mesh operating environment -- or it calls cloud MOE to do what it needs to do.

JU: I think I can figure that out. [laughs]

So the synchronization piece is interesting. You've obviously been around this track a few times before. This time around, how is it different, how has it evolved from things you did in Notes or in Groove?

RO: Well one thing that's different is that I didn't build this software. I sponsored it, I had a good degree of input. But a very talented team, with talented leadership, formed and rose to the task. I sponsored it, and there is certainly a DNA trail you can follow from Plato at University of Illinois to Lotus Notes to Groove and now into Live Mesh. And I was fortunate to have the opportunity to interact heavily with that team, when I had the time to do so, which was right after I came on board, when I was still in the CTO role. But they took these base concepts and really ran with them, and developed it into a much richer thing than I could imagine.

But the DNA elements are the basic sync model, the basic interaction model. The biggest difference between Groove and Notes was that Groove embraced the concept of ad-hoc interaction much more in terms of inviting people into a shared environment. So those invitation models are essentially borrowed from Groove into Live Mesh. So if you are a Groove user, you will feel very comfortable with that model in dealing with Live Mesh.

JU: So, Mesh.com is the place to go to check it out but where do the developers find the SDK and everything they they need to know to actually work with it?

RO: We're bringing out the Live Mesh software right now because it's a preview, we need to begin getting user feedback, we need to begin testing the scale of the back-end. You can architect and plan these things but you can't actually just light them up at hundreds of millions of users overnight. So there's a progressive rollout that begins today. What you won't find on mesh.com is the developer kit. We're beginning a series of systems design reviews with smaller sets -- but increasing sets -- of developers over the course of the summer. The official rollout of the dev platform, and broad availability of the dev platform, would be at our PDC, our professional developer conference, this fall. So as a user, look at Live Mesh now. As a developer, stay tuned, look at the screencasts that we've done, they'll show what we can do from an application perspective, but really, come to the PDC, go to the PDC web site when it happens and play with it. Both from the perspective of extending a rich application to the web and to other devices, and also extending a website out to take advantage of the power of Windows.

JU: This is been extremely useful. Thanks, we appreciate it.

RO: It's been fun, thanks.

Pablo Fernicola is a group manager at Microsoft.  He runs a project focused on delivering tools and services for scientific and technical publishing, with a particular interest on the  transition from print to electronic and web based content, and its implications for collaboration, search, and content discovery in the future.

In this interview, Pablo explains how a new add-in for Word, now available as a technical preview, helps authors and publishers of scientific articles work more effectively with one another, and with online archives like PubMed Central.

Pablo Fernicola

Links Technical Computing @ Microsoft - Scholarly Publishing Download details for the Article Authoring Add-in Pablo Fernicola's blog: ex Scientia

JU: Hi Pablo, thanks for joining us to talk about a new Word add-in for authors of scientific journal articles. It's an interesting story about applying the XML capabilities of Office, and also about the evolution of journal publishing. How did this project get started?

PF: It's an incubation project. Three people had an idea: Jean Paoli, an XML pioneer, Jim Gray...

JU: Oh really? I didn't know he had been involved.

PF: Yes, he and Jean really pushed to get this started, and they both recruited me for this project. It's been a little over a year since Jim disappeared, and that was a big blow, considering his key role.

And third key person is Tony Hey.

JU: We should explain that Tony runs what's called the technical computing initiative, and is very involved in figuring out how Microsoft can help various people in the scientific community address computing and information management challenges.

PF: Right. Scientific authors in many disciplines use Word to write articles. We looked into how to simplify the workflow, streamline the process, and lower the cost. And not just for the authors, but also for the journal publishers.

JU: It's been true for a long time in publishing, and not just scientific publishing, that there have been real challenges getting that Word content converted into the kinds of long-term formats we need: XML that's richly decorated with metadata.

Publishers have tended to use strategies that involve giving people templates that try to use styles to control what's in the document. But since Word 2003, and especially since Word 2007, there have been a set of XML capabilities which have made possible a much more robust approach.

PF: That's right. Before Word 2003, styles were the best you could do. And people got quite far by relying on them. But they were very fragile. When you copied and pasted, styles would bleed across. It was hard to disentangle that when you converted the file.

JU: That's part of the problem. And part of is that, along with the content itself, there's a process involving the metadata, and that process is divided between the author and the journal publisher. It's a shared responsibility, and you need an information management system that embraces that division of labor.

PF: Also: What kind of user interface do you present to these different groups? There are really three groups. First the authors, who are subject-matter experts but don't know anything about the publishing process, and shouldn't have to know.

Second, the journal editors. They're also subject-matter experts, but they also know about the structure of the journal, and about the metadata they need to apply

And third, you have companies and vendors who do backend tools and services, as well as the folks who work on the electronic archives. With the move from print to electronic journals, the role of the archive becomes very significant. Either the journals have their own repositories, or you have centralized repositories at university libraries or larger institutions, for example the National Library of Medicine with PubMed Central, or Cornell with Arxiv.org.

That group is very technical in terms of understanding file formats, elements, and properties.

JU: But even those folks shouldn't necessarily need to master all of that. They'd rather spend their time on math and physics, not the minutia of XML publishing.

PF: That's right. The way the pipeline is set up today, you start with a Word document, and then at a certain point you convert to XML, and from that point on, all the editing happens in an XML editor.

JU: So in biology and medicine, the format defined by the National Library of Medicine, and the one you're supporting in this Word add-in, is called the NLM DTD.

PF: Yes. It's not only used by PubMed Central, but also a lot of the commercial publishers are using it for their archival format. And we're also seeing it used by publishers in other disciplines, for example law and social science.

JU: Really? It's general enough for that?

PF: It is fairly general, and I'm really impressed by how the community related to scientific, technical, and medical publishing is not reinventing the wheel, but instead leveraging something that's in common use.

A significant point is that the format usually does not encode any presentation elements. It's all about the semantics and the metadata, not about what font or background color. As you try to preserve data for the long term, for centuries from now, the presentation is not relevant, it's the content that matters. You can always generate a presentation from it.

JU: So as we see in the accompanying screencast, you've created an add-in that presents editing enhancements both for authors and for editors. The interface for the author helps that person fill in the template and also apply those metadata elements which are appropriate for the author to apply. Then there's a separate interface for the editor. Explain a bit about how this can change the workflow.

PF: If you start from the author side, a key premise was requiring less effort to produce a valid document. You want to avoid having the author round-trip with the editor, back and forth, because they didn't fill in all the required information.

JU: And that happens a lot?

PF: Yes. And it's not just failure to provide the required information. We want to make it easier to provide the correct information. Consider co-authors. You'll likely work with the same ones over and over. You want to avoid having to repetitively enter that information, and avoid having errors creep in. Remember: As we move to electronic publishing, search becomes key. It's the main way people will find articles. To have good search results, you need to know the information in the articles is good. If the last name of the author is misspelled, it's harder to find all the papers from that author.

JU: In terms of the consistency of author information, you can help with this Word add-in by normalizing the metadata editing process, but there still has to be a reliable disambiguated set of author names which are managed by the publishers, and ideally by a federation of publishers, and ultimately even more broadly than that.

PF: Correct. If we look down the road, we see something like a global directory, but we're not there yet. We have to build up to that. When you look at the add-in, we're taking small steps that will get us to at least a better baseline than we have today.

JU: Or, given that the world is moving to that baseline anyway, will help make it quicker and easier to get there.

PF: That's right. If we think of the authors, the key thing is to provide a very simple interface. As we consider features, if they look complicated we'll drop them. One of the prevailing rules is: Don't duplicate Word UI. If there's a way to do tables or equations or reference lists in the Word UI, we'll use those. We don't want to provide a lot of new UI for the authors to learn.

JU: What I find interesting, here, from a workflow perspective, is how people in different roles are touching different pieces of data and metadata. Historically that's been a one-way process. Once the article is converted into the NLM format, it's typically not available to go back to the author for editing in the original context. So the person at the journal has to be responsible for round-tripping change requests.

Similarly with the editing of the metadata. The author might want to make some changes, the journal publisher might want to make some changes, and those things tend to happen in disparate environments. What this is showing is what has always been the promise of robust XML editing on the desktop. You can bring all these chores into a common environment. The unit of workflow, the document, is something that can flow to different people in different contexts, and be modified in different ways, but it hangs together as it moves through the process.

That's a big deal, and it goes far beyond the specific domain of scientific and technical publishing.

PF: Right. And in addition to keeping all the data together and providing a simple interface, publishers have told us that as they move to electronic-first, they expect the cycle times to shrink. With the current disconnected tools and formats, that's hard to achieve. If you want to make a quick revision and send it to the journal, it may be too late because they've started the process of conversion, and once that starts there's no stopping it.

And to your point about other domains, people have told us they want to use this for things like grant requests as well, moving away from article content to other kinds of content that can benefit from the structure and validation.

JU: The problem is almost universal.

PF: Yes, anytime you want to validate content, or preserve it for a long time, these capabilities are relevant.

JU: So 2003 was the first major deployment of XML capability for Office and for Word. We haven't yet seen as much use of that capability as I'd expected. Why?

PF: The biggest challenge was that XML wasn't the default format. You had to have authors do special things to produce XML. Also, if you think of the NLM formats, they contain things that aren't part of normal Word content or UI. In Word 2003, extending the document content, or extending the UI, wasn't as easy as it has become in Word 2007.

With Word 2007, you end up with a set of things, in a single installation, that bring all the enabling capabilities together at the same time and in the same place.

JU: So what you did have, in Word 2003, was user-defined schema, but you're saying that wasn't enough, and that the newer capability of including arbitrary chunks of XML is more flexible for this purpose?

PF: Yeah. There's two parts to that. There's content within the document, so the ability to have new XML elements that are part of the document, and that's more robust and expressive in Word 2007's Open XML format. Then there's the ability to have other XML data packaged within the file. Custom XML is what that's usually called.

JU: And that's the method you're using for the journal metadata?

PF: Right. And since this is all defined as part of the Open XML format, and since the packaging of the file follows the standard as well, developers can build their own tools to create metadata, access metadata, or even create the whole file, they can.

JU: So this is a first cut you're putting out for publishers to experiment with, and to help you refine the templates they'll deploy to authors?

PF: Yes, this is a technology preview for evaluation and feedback. The idea is that the publishers will create the templates themselves.

JU: Who are you working with?

PF: We're talking to many different journals, publishers, and archives. Each constituency has a different set of interests and requirements. Journal editors care a lot about the templates, but folks at PubMed Central and Arxiv care more about how the metadata gets validated.

We expect a beta shortly, and a 1.0 release by late summer. It'll be a free add-in for Word.

JU: Well thanks Pablo. I fear that this will only seem interesting to the relatively small number of folks who have a direct interest in scientific, technical, and medical publishing. But I hope it will be apparent that it's much broader. You hinted at that when you mentioned that the NLM format, despite having been invented for the particular purposes of certain disciplines, is being taken up by people in legal and other disciplines.

I'm excited about it because I care about publishing and metadata and robust information systems and open formats, and this brings all those things together. I'm glad to know that it's happening, and I'm glad you're working on it.

PF: It's really proving the value proposition of XML, and show how it's coming of age in a mainstream production environment.

JU: Yep. For those of us who've been thinking about this for a long time, there's been a tendency to get frustrated and feel like it'll never happen. But it just takes a while for things like this to make their way into the mainstream, and this is a great example of that.

Well, thanks Pablo!

PF: OK, thanks!

In this podcast, MSR researcher Catharine van Ingen and Berkeley micrometeorologist Dennis Baldocchi talk with Jon Udell about their collaboration on www.fluxdata.org, a SharePoint portal to a scientific data server. The server contains carbon-dioxide flux data gathered from a worldwide network of sensors, and provides SQL Server data cubes that help scientists collaboratively make sense of the data.

Dennis Baldocchi is a professor of biometeorology at Berkeley. His research focuses on the physical, biological, and chemical processes that control trace gas and energy exchange between vegetation and the atmosphere. He also studies the micrometeorology of plant canopies.

Catharine van Ingen, a partner architect with with Microsoft Research in San Francisco, does e-science research exploring how database technologies can help change collaborative research in the earth sciences. She collaborates with carbon climate researchers and hydrologists.

Links Fluxnet website MSR news article

JU: Dennis, you're someone who's pulling together a worldwide network of CO2 monitoring stations. Can you briefly explain how these devices work?

DB: Sure. Let me give you a bit of history. Back in the late 1950s, David Keeling made some of the first measurements of carbon dioxide concentration -- on Mauna Loa in Hawaii, in the Arctic, very remote locations. They saw an increase in the C02 concentration in winter, and a decrease in summer. The increase is due to respiration in the biosphere, the decrease is due to photosynthesis. And on top of this they saw a trend due to fossil fuel combustion and logging of tropical forests.

These measurements were just C02 concentrations. As atmospheric scientists, we know that changes in the atmospheric concentration are due to fluxes. We measure actual fluxes: moles of carbon dioxide, per meter squared, per second, between the atmosphere and the biosphere.

We do it with a combination of sensors. One is a three-dimensional sonic anemometer, which measures up-and-down and lateral-and-longitudinal motions of the air, ten times a second. And simultaneously with new sensors we measure instantaneous change in CO2 concentration.

JU: So it's a combination of sensing wind speed and sensing atmospheric gas.

DB: Absolutely. We measure a covariance between the two, and theoretically that's related to the flux density.

JU: And this population of sensors has been growing for 15 or more years?

DB: Yeah, my old lab in Oak Ridge, Tennessee made some of the first sensors we were using in the early 90s. Around then a company called Licor started making a sensor that's about 15 centimeters long and shoots an infrared beam from source to detector. The air can blow through this sensor, and it's low power, doesn't need pumps, so it can be deployed in the middle of nowhere. Many of us run with solar power, so we have a PC that pulls an amp, then the sensor pulls another amp, so for two amps we can run a flux system.

JU: As Catharine points out, there's a long tradition of large-scale collaboration in some scientific disciplines, but it's relatively new in other areas, and it sounds like this is one of those.

DB: Yeah. I was a grad student in the 80s and I remember my professor having a desk full of data. People would knock on the the door wanting to borrow it, and there was always some reluctance, it was really a single-investigator culture at the time.

In many ways I credit our Italian colleagues, they were really gregarious and good at hosting wonderful workshops that started bringing people together.

JU: So Catharine, how did Microsoft get involved in building out the scientific data server that supports this project?

CvI: It was serendipity. We had met folks at the Berkeley Water Center two ways. First through Jim Gray's interest in e-science and database applications. Second, one of the current heads of the Berkeley Water Center is an old friend of mine from grad school, Jim Hunt. We were talking about doing a hydrology project, then somehow my colleague at BWC on the computing side, Deb Agarwal, ran into Dennis, and we started talking.

Dennis fit all of the criteria for how I like to engage with scientists. He was desperate, he had a problem that he didn't know how to solve, and that was important, because it meant he was willing to talk to us and teach us things.

Also he had enough data to make things interesting for us. It's not petabytes, but we're talking about the hundred-gigabyte range, and the dataset is extremely diverse. I find it fascinating from an informatics point of view because it's a true scientific mashup to do the data analysis. You're taking the flux data that Dennis just described, as well as a lot of site properties, and other things from the literature, and trying to bring it all together.

JU: There's a whole range of what you folks call ancillary data, which describes soil and vegetation and other aspects of the environment.

DB: To give you an example, the meteorological data, from a database point of view, is fairly simple and regular. Our loggers give us half-hour data, so you get what's essentially an Excel spreadsheet. The rows are timestamped for each half-hour, and the columns are temperature, flux of water, solar energy, and so on. But it gets complex when you weave in the ancillary data. For example, you need to know the population of leaves that control these fluxes. You may measure that in a half-dozen spots, a half-dozen times per year. Then you need to understand leaf photosythesis, and that's another set of measurements, and then soil texture, carbon, and water absorption, and all these measurements are at different depths, different times, it gets really complex.

CvI: Another interesting aspect, from our side, is handling time. We all think time is linear...

DB: [laughs] Not according to Einstein...

CvI: [laughs] ... well ... so, since we're dealing with plant information, plants photosynthesize during the day. So rather than using wall-clock time, using the plants to tell us about day or night was really fascinating. In effect we're deriving a time window based on the time series data themselves, and for informatics folks, this was more fun than a barrel of monkeys. We've generalized the concept now, and applied it to a couple of other disciplines. Handling time has turned out to be one of the biggest areas of learning.

JU: So what is FluxNet, actually, and how does the data get into the scientific server that you've built?

DB: It started at a workshop we held in Italy in 1995. From that, regional networks started blossoming. First off the ground was the EuroFlux network, then AmeriFlux in about 1997, then over time the Asians, the Canadians. NASA funded us for two cycles, and then things dried up as they decided to go to the moon and to Mars. Most recently we've been funded by NSF, which is funding a whole bunch of ecological networks. On the side, there's been funding to Oak Ridge National Lab, through NASA, to maintain the data acquisition and archive system. And then Deb and Catharine joined in to build value-added products through this FluxData project.

Sometimes I think we're like Tom Sawyer, we've got this fence to paint and all these people are helping us paint it.

JU: Or like stone soup.

CvI: It is like stone soup. From an informatics point of view, the way we think about it is that the data starts with tower owners -- and Dennis is a tower owner as well as a project overseer -- and flows to one of the network repositories, or directly to Oak Ridge where the data is stored.

JU: OK, so your site, www.fluxdata.org, is not the repository, it's for analysis...

CvI: Yes. There are data archive centers, funded primarily by NASA, where you can contribute data, and where data is stored. The challenge for the scientist is to get from the raw data to the science, it's a classic last-mile problem. So the data flows from the repositories to the folks in Europe who are doing gap-filling and uniform processing, and it flows back to Oak Ridge for long-term storage, and it flows to us.

We then make it available to researchers to download, and we provide the value-added summary products. So we're not at the front end gathering data, and we're not the archive, we're in the middle, solving that last-mile problem.

JU: Part of that solution is to put the stuff into data cubes. Dennis wrote somewhere that while these have been used in financial analysis for a long time, their application to scientific analysis is new. It might surprise some people to learn that this way of looking at data isn't common in the scientific world.

CvI: It actually isn't. OLAP databases, data cubes, have been around for a long time. I think I first saw one in the early 90s. But that was really commercial data, it was about finding how to make coupons for Oreos and milk. Scientific data is different in a couple of respects. First, it's much more dense. You tend not to always buy Oreos and milk together, but Dennis always reports CO2 flux, temperature, and precipitation together. The other difference is that a lot of the analysis for commercial data is not at the leaf nodes, it's about annual sales. Whereas a lot of science is actually at the leaf nodes, it's about looking at statistical variation in the half-hourly data.

So we end up building different-shaped cubes.

DB: And let me add that we'll present this data with gaps, for several reasons. One is that if there's a thunderstorm, it might cause the instrument to malfunction. Another is that we have to comply with meteorological steady conditions -- for example, steady winds. So we apply a lot of quality assurance to the data set, and that produces gaps, but any user of the data wants a continuous record. So we need to find ways to fill those gaps.

We also want to partition the fluxes, so we can understand mechanisms. We measure the net ecosystem exchange, but there's a component due to photosynthesis and a component due to respiration. By separating out day and night data we can derive these components, so there's all this value added to the data from the archive.

JU: So I looked at some of your pivot tables, for example on sites by vegetation -- how are those being used?

DB: To do cross-site analysis. For example, we're interested in how length of growing season may affect net carbon exchange. When I did this analysis before I met Catharine, I had to open a bunch of spreadsheets and cut and paste, cut and paste. With the cubes, you press a button and the data's there. It really allows you to do a lot of quick what-if questions, and be creative. It makes our work quicker and easier.

CvI: We're also doing a fair amount of sorting. You can sort along vegetation types, to see the difference between croplands and grasslands. We also know each of the sites that is a boreal forest, so you can look at just those, or just tropical forests. If the database has 900 site-years, you can select just the 200 that you need for a piece of analysis.

JU: Is it fair to say that until this was brought together it wasn't possible to do this?

CvI: It was possible, but just really tedious.

DB: Back when the network was small, we did a workshop in 2000, and we had about 100 site-years of data from 30 sites. It was easy to be clunky. But now we have 900 site-years from 400 sites, and you just can't use the old methods. We have to go modern.

JU: What kinds of collaboration effects are you seeing? You've written that it's a big challenge to motivate scientists to contribute the ancillary data in a standard way. Getting the stuff in front of people like this, in a common presentation with explanations about what all the variables mean, and how to report them, should help get everybody onto the same page.

CvI: I see a couple of things. First, we're starting to hear from individual tower owners asking us questions, and telling us what's wrong. "I'm sorry, my site isn't really at that lat/lon." Or: "My leaf index is really this."

They see their data being used in papers: we're hosting access for about 60 paper-writing teams. As the papers come to fruition, we're actually tracking what sites they're using, so it's possible to go in and find out who's using your data.

DB: It's motivating. I know my post-doc is so excited when she finds out people are using this data.

JU: That explains why you have an update feature on the site?

CvI: Absolutely. We know there are corrections that need to be made. Treating it as a living, breathing data set, and being able to respond in an organized way to changes...

DB: As more eyes look at it, they can help us fix it. Especially our own data. You look at it and don't see the problem, but when someone tries to use it...oops. In fact we found a problem with our solar heat flux recently. We were doing the correct calculations from 2000 to 2003, then we changed algorithms, and the staff changed, and all of a sudden there was a glitch in how the data were being processed. Finally some scientist from UCLA wanted to use the data, and he plotted it up, and found the problem. So now we're correcting that.

CvI: One of the things that happens when you plot data over time is that you can see any errors in time reporting. One site was off by a couple of months. The data looked fine when you plotted just that site. But if you plot it by nearby sites, suddenly you see the problem. That's the kind of processing -- bringing the data into focus -- that we're engaged in right now.

JU: So you've got the data online, and tools for viewing and updating the data, but there's also a conversational infrastructure. You have a blog, there are places for people to add comments and have discussions, and all of that is kept together with the data. Catharine, you've said that the role of data curation in science is emerging, and will be key as we increasingly see these mega collaborations with hundreds or even thousands of people working on the same data. You need an environment in which those conversations can be centralized in the same way the data is centralized.

DB: There's also almost a traffic-cop role too, just to avoid redundant efforts. There are several obvious ideas, and multiple groups may want to pursue them. In the long run it's a waste of effort if people are doing the same redundant analysis, and only one paper may get published. If we can get these people to talk to each other, and interact, that's critical.

JU: As Catharine puts it, investing the same effort in publishing data as you would in writing a paper is something that's not yet socialized.

CvI: No, it's not. We see again and again how difficult it is to put the data in a box and tie a bow around it, so people can reuse it. It's very hard, but very important, long-term, for a lot of these environmental problems.

DB: So Catharine, by marking these data sets and giving them some kind of provenance, is this a way scientists can get credit for the work?

CvI: Well, the challenge isn't only enabling that, but also teaching the funding agencies that it's just as important.

JU: Exactly. I've talked to Timo Hannay about this -- he's the guy who runs the web stuff for Nature Publishing -- and this is a huge interest of his. Science is an enterprise that runs on people getting credit for publishing papers, not data. I gather that often papers are published as a thin gloss on a data set, just to get the data out there. There hasn't been a model for publishing the data itself. The fact that the data from somebody's individual tower can be traced back, and then traced through its use in follow-on papers -- that's huge. Your post-doc can not only get excited about other people using her data, she can get credit for their citations of it.

JU: So, the climate effect of C02 is obviously a hot topic. What have we actually learned at this point?

DB: One paper used this network in combination with remote sensing to see how carbon exchange across Europe responded to the drought and heat wave in 2003. So here was this network poised to measure how the whole biosphere responded to this climate assault.

The network has also been successful with what we call emergent scale processes. One that came out strongly is that plant canopies respond to light more efficiently if the light is diffuse, as opposed to when there are clear skies. That's a process we haven't seen before.

Another thing we found, because we have continuous records, is that if there's a summer rain event, microbes turn on immediately and produce huge amounts of respiration that we never envisioned before. Scientists in the past would miss these extreme events, but by having continuous measurements we can see how the system responds.

JU: But you wouldn't argue for long-term trends in the 15 or so years of data you've collected?

DB: If there are long-term trends, they seem more related to ecosystem dynamics. Many of the forests under study were disturbed at the turn of the century, so they're going through that natural cycle of growth, maturity, and decay. Those ecological features lay on top of any potential climate trends.

JU: So it's more about having an infrastructure in place that allows us to have the data in hand, and then make some predictions?

DB: Yes. Now in fact, one of the things we are seeing is a change in the length of the growing season. As things have gotten warmer, the spring comes earlier, and it's really affecting carbon uptake in the citrus forests. But the big unknown is that if you have an earlier spring you might also get a summer drought, so you have an increase in carbon in the spring, and a decrease in summer, and the two factors may cancel out. But with our measurements we can see the mechanisms, we can understand and parse out what's happening and why. Whereas in the past, scientists would cut down trees and get tree rings and take one integrated snapshot for the whole year. But they wouldn't understand why, because those tree rings were also affected by drought and temperature and ozone and elevated C02 and other issues.

CvI: It's really a great time to be doing this stuff, because you're at the juxtaposition of social need, scientific need, and the availability of cheap technology.

DB: And our NSF grant encourages to do outreach, so this is a great opportunity to do that.

CvI: Jim Gray always used to point out that the post-docs are the ones in any collaboration who most embrace new technology, and move the entire collaboration forward. Knowing the guys over in Europe that's certainly true, and you can see it happening with your own post-docs, Dennis.

JU: So how are these cubes getting built, Catharine? What was the collaboration between you and the scientists?

CvI: We're lucky to be starting with a data set that is very well processed. As to building the rest, Dennis gave us, gosh, I looked at 300 hundred of his graphs. I also got a similar collection from two of his other colleagues. I went through all the graphs and papers to try to understand how the data is manipulated and displayed.

DB: That's a good idea. I didn't realize you did that.

CvI: Oh yeah. [laughs]

DB: That would be helpful, because you see the kinds of products we're trying to create from these databases.

CvI: Absolutely. I started by classifying the graphs into time-series graphs, scatterplots, and then everything else. Then I waded through how everything was sorted, searched, filtered, trying to figure out how to organize the data to enable that class of graphs.

DB So Catharine, there are a bunch of graphs I'd like to replot with this new database.

CvI: Well Dennis, you and I should have lunch and we should figure out how to rip out a bunch of graphs.

So, along the way we realized that scientists will often make 50 graphs, through away 48, and keep two. The ability to make a lot of graphs rapidly and simply usually requires some kind of scripting, and that's where you start leaving Excel and going into MATLAB or another scientific analysis tool.

DB: Yeah, I'm using MATLAB a lot nowadays, and I'm seeing things I never saw before. I like having the script files because it gives me some history of what I was looking at.

CvI: That's why we decided to connect MATLAB to the cube, so you can browse the reports we make in Excel, or go directly through MATLAB. Again, it's solving that last-mile gap to the scientist's house.

JU: Well this has been great, thanks!

DB: Yeah, thanks. Catharine, we should get together and talk about some graphs.

CvI: Thanks Jon. And thanks Dennis. Are you in your office? I'll call you later this afternoon.

Kyril Faenov is the General Manager of the Windows HPC product unit. Before founding the HPC team in 2004, Kyril worked on a broad set of projects across Microsoft, including running the planning process for Windows Server 2008, co-founding a distributed systems project in the office of the CTO, and developing scale-out technology in Windows 2000. Kyril joined Microsoft in 1998 as the result of acquisition of Valence Research, an Internet server clustering startup he co-founded and grew to profitability by securing MSN, Microsoft.com and some of the world's other largest web sites as its clients.

Rich Ciapala is a program manager in Microsoft HPC++ Labs, an incubation team within the Windows HPC Server product unit. Rich joined Microsoft in 1992 and has held a number of different positions in technical sales, Microsoft Consulting Services, the Windows Customer Advisory team and the Visual Studio product team.

Links Microsoft HPC++ CompFin Lab

Kyril Faenov and Rich Ciapala discuss a new HPC++ Labs project that enables students to run computation-intensive experiments involving large amounts of financial data.

JU: What Rich just demoed, which we'll show in a screencast, is how a financial model can be deployed to a server that acts as a front-end to a compute cluster. It's a nice easy way for students to use a model developed by a professor, select a basket of securities, run a very intensive computation on them against large chunks of data, and get answers back in an Excel spreadsheet. The bottom line is that the students can run an experiment using a level of computing power that was never before so easily accessible.

KF: Yeah, because of the complexity involved in deploying systems like that, acquiring the data, and curating it, a lot of universities don't have this kind of infrastructure in place. So for a number of students who haven't done this before, this will make it available for the first time. For others who have, it will make it quite a bit easier.

JU: Now these are not computer science students who are learning about high performance computing, and about writing programs for parallel machines, these are students who are learning about financial modeling, and this just makes a tool available to them that can accelerate that.

KF: Precisely. Most of our HPC customers are scientists, or engineers, or business analysts, not computer scientists. They're folks who use mathematics, statistics, differential equations ... sometimes not even math directly, but applications that encode these mathematical models to do research, or engineering, or risk modeling, or decision making. To them it's just a tool, and they want to use it in the way they use PCs today, as transparently and straightforwardly as possible.

JU: What's the situation today for most people? In the case of the covariance model Rich showed in the demo, if it weren't being done like that, how would it be done?a

KF: You can do it in Excel, or MATLAB, or SAS, on the workstation. So you'd acquire the data, and use your preferred tool ...

JU: ... and wait a long time ...

KF: ... and wait a long time. And if you want to do a significant amount of data -- like a year's worth, for a large number of stocks -- it might not even be possible at all.

Or you might load it up into a server, but then you have to figure out how to write an application, how to deploy it out to the server, then figure out how to submit the data to the model, pull it back, integrate into the visual analytic process.

This multi-step process is exactly what our HPC customers are running into. They're expressing the models and doing the design on the workstation, using any number of tools. They do the analysis of the results, and visualization, on the workstation. But large-scale computation runs somewhere else. It might be in their organization, it might be out on the Internet, but it's a very disjointed process.

JU: There are clusters out there in academia, and there are people doing these kinds of things, but the point is that hasn't been woven together yet.

KF: That's right. In 2004 the U.S. government published an assessment of U.S. competitiveness in high performance computing. The first recommendation was, and I'm quoting:

Make high performance computing easier to use. Emphasis should be placed on time to solution, the major metric of value. A common software environment that spans desktop to high-end systems will enhance productivity gains.

That's what we're starting to see in the HPC community. Not just getting the systems running as fast as possible, but figuring out how the workflow, the creative element of the scientific process, can be optimized.

JU: So, Rich and I talked about the particular model used in his demo is in a class called parameter sweep, which he distinguishes from the more distributed and chatty kinds of applications. In this case, you can send a batch of data down to a node, it can think about it for a while then give back an answer, and there doesn't need to be much communication. Is that the optimal scenario for this architecture?

KF: Actually, it's optimized for a broad range of HPC applications. In fact, the major goal of the first release of the product, Compute Cluster 2003, was MPI-style [message passing interface] applications. There are a lot of these in engineering and in the environmental space. You're modeling some kind of physical process, and you build a mesh or grid that takes a large physical process or body, partitions it, does computations on local areas, but then has to frequently exchange data across the partitions. Think about a car crash simulation. You might partition the hood of the car into a lot of pieces, every one computed separately, but as the deformation is happening the forces need to be exchanged. Or weather modeling, where heat exchange happens across partitions.

JU: There's a high degree of data interdependence.

KF: Exactly. When you you have an interdependent problem, you use MPI for that. We worked with the team at Argonne National Labs that releases the open source reference implementation of MPI, and we've adopted that in our product, optimized the performance and security on Windows, and integrated it into the stack.

JU: Right, I knew about the MPI layer in the cluster product. But it seems that the system we're looking at here, for professors to enable students to experiment with financial modeling -- that one is targeting the other class of application

KF: Right. There is a large class of what we call embarrassingly parallel problems, a lot of statistical analysis falls into that category, and media rendering, where you have a lot of independent tasks. And that's what we have here, because every pair of instruments that needs to be compared is an indepdendent task. What you need to do is spray those tasks across a cluster. We have a solution that makes that much more approachable.

JU: So in this case, that entails mapping the input parameters to a set of work items.

KF: Correct.

JU: OK. And outside the financial domain, where else will this style be popular?

KF: We'll see this in a range of disciplines. This particular example uses data from an external source -- in this case, the stock market -- and it's looking for patterns of correlations between different signals. This paradigm is broadly applicable. If you think about, for example, clinical research, where you have data coming in from hundreds of patients, where the data includes many parameters about their health condition, and you're looking for disease markers or drug reactions -- you're doing correlation analysis among the diffeerent signals.

Or you might have data coming in from sensors deployed in oil and gas pipelines for safety monitoring, or environmental sensors, everywhere you have instruments producing high volumes of data, where you need to find patterns in data, and optimize the scientific process of developing models that produce insight into the data.

JU: Would you say that these embarassingly parallel problems are low-hanging fruit?

KF: Very much so. And there's another class, Monte Carlo simulation, a method used very effectively across a range of industries to statistically explore different scenarios, for risk analysis and predictive model. It's used in financial services, like insurance, but things like process management in factories can also use it, or logistics chains.

JU: So for the current example, give us a sense of what skill set is required of the professor in order to create the model and make it available to students. There's some .NET programming involved, right?a

KF: Rich, do you want to take this?

RC: Well, you pick your .NET language of choice, and your development environment, which may be Visual Studio. We're making the data available in terms of LINQ, so you need some understanding of that, although for the queries typical of these applications it's fairly basic. And in fact, since it's integrated into the language and you get things like syntax completion, it's probably easier than writing SQL.

JU: There's a framework provided, what does that include?

RC: It does two things. First, it forces you to define the interface for your model in such a way that you can easily build, for example, an Excel front-end to send input and retrieve output. Second, it shows you exactly where you need to do the splitting of the tasks into work items, where you do the spraying of work items to the cluster, and where you put the code that does the covariance and correlation calculations.

KF: The professor focuses on writing the analytics parts, and doesn't have to worry about the fairly complex workflow skeleton that submits the data to the cluster, partitions the work, accessing the results, and then performing the final reduction.

JU: So can focus on creating the pivot table, or using MATLAB, which is where I'd rather be spending my time.

KF: Yes, in a domain you're expert in.

JU: So, who are the guinea pigs for this system?

RC: Our first two are the University of Washington, which did the model we demonstrated, and the University of North Carolina in Charlotte.

JU: Kyril, I know you have big ideas about where this can go. Why don't you paint the picture?

KF: When we started the HPC team at Microsft, we realized it's an actively evolving space. But Microsoft is fairly new to it. Without the benefit of 20 or 30 years of experience, we felt we needed to do something that would help us develop expertise and build up an understanding of not just the technology, but also the usage patterns. So we worked with, and funded, 10 universities worldwide, and that's been very helpful.

We've also created an internal team whose mission is to do incubation. The goal of this team is threefold. First, to prototype and demonstrate the end-to-end solutions that our HPC customers will find beneficial, and what Rich has demonstrated is an example of that.

Second, to help us explore the trend that we see as HPC becomes more and more data-driven. There's still the world where you run simulations, of car crashes or weather. But a lot of new applications are mining data for insight, and doing it in a computationally intensive way. That changes the formula for how HPC is used. In many cases it's becoming impractical to put clusters in customer locations, if you have to ship terabytes or petabytes of data around. Data repositories are starting to act like black holes, if you will, that are pulling computation towards them.

JU: I'm sure that's true in the climate area...

KF: Climate, biology, astronomy, geosciences, everywhere that you start accumulating tremendous data sets. We think there's going to be way that Microsoft can help customers optimize how these services are built, because there's no established architecture today.

JU: Jim Gray was always talking about how it's becoming necessary to Fedex hard disks around the world because there's no other way to move the data to the computation. But instead you're proposing to move the computation to the data.

KF: That's right. We want to incubate a few of these high-value data-centric services, and demonstrate the best practices for doing that while providing free access to academic institutions. That'll help us understand what's involved in operating these services, and potentially we might imagine Microsoft running a few of them.

Then the third goal for the incubation team is to flow the requirements for doing these things into software, so that customers can do this as easily as possible themselves. One of the challenges today is that there's a dichotomy between these very large-scale Internet services being built -- by Microsoft, Yahoo, Google, and others -- but they're in their own world. Customers can't take a slice of that infrastructure and deploy it in their environments.

At the same time, we keep on building off-the-shelf software that people install on their infrastructure, and we're just now learning what it takes to run HPC services using that software. So we want to make sure there's a tight coupling between the team that builds the prototypes and runs the services, and the team that implements off-the-shelf software, such that we run our services using the products that we build. And at same time, we want to make it a turnkey operation for customers to stand up these services themselves.

JU: That's a key point, so let's underscore it. We're seeing the emergence of a small set of what I call intergalactic clusters, which are one-of-a-kind things, and they are not replicable. They do interesting and powerful things, but you can only do things with them on their terms.

Your notion is that you want to maintain parity, and ensure that you can always replicate what's happening in the cloud if you need to.

KF: Exactly. For example we just talked about the gravitational pull of data. Imagine you have an astronomy site that accumulates a petabyte. You can try to put it on one of these intergalactic clusters, but that's maybe not what you want. Maybe the most optimal thing is for you to stand up a 1000-node cluster with each node having a terabyte of disk. We want to enable that. We want to be able to tell our customers: Here's how we run this large-scale data-driven HPC applications, and here's how, within a day or two, you can stand up one of these yourself.

JU: So you even see some potential consumer applications for this, don't you?

KF: Sure. Think about search. We can only find answers to questions that have already been answered. But imagine if your questions require novel insight to data. For example, Microsoft HealthVault is starting to accumulate a lot of health data.

JU: Right, so what are my cancer survival prospects given the specifics of my case, and in light of a large body of data about other people?

KF: Or help me do a predictive analysis on my risk of flood or hurricane damage, not for the region in general, but for my house, given the weather and geographical information that's available, and maybe given a few sensors that report data specifically for my house.

To enable these applications, you have to create a platform that makes it possible to curate data, and develop applications that run on top of it. What you see in the service we just demonstrated is a first example of that.

JU: OK, thanks guys.

In this screencast, Rich Ciapala demonstrates Microsoft HPC++ CompFin Lab, which integrates Microsoft HPC Server, a central market data database, and Microsoft productivity products to provide university courses with an online service to publish, execute and manage computational finance models.

In this podcast, Jon Udell chats with Vittorio Bertocci, author of Understanding Windows CardSpace. The discussion traces the evolution of the identity metasystem, explores the rationale for CardSpace, and considers the unsolved problem of public online identity for individuals.

Vittorio Bertocci is a senior technical evangelist for Microsoft Corporation. He works with Fortune 100 and major G100 enterprises worldwide, helping them to stay ahead of the curve and take advantage of the latest technologies. He is the primary author of Understanding Windows CardSpace: An introduction to the concepts and challenges of digital identities.

Links Vibro.NET: Vittorio Bertocci's blog Understanding Windows CardSpace

JU: What I particularly liked about this book is the lengthy introduction that sets the context, not just for CardSpace but for previous iterations -- what problems did they solve, what problems did they not solve, and why does that lead us to the architecture we have now.

For example, you discuss SSL client certificates. I remember thinking, in 1996 or so, when that capability was present in both Netscape and IE, here we go. No more passwords. Obviously that didn't happen. But why not?

VB: The SSL client strategy, from a cryptographic perspective, is perfectly sound. But it's a paradigmatic example of how technology alone cannot solve a problem that involves human interaction.

The certificate is a construct that's made for computer scientists. It says that the subject is the rightful owner of a certain public key, which doesn't really resonate with my mother or my sister.

JU: But it didn't have to be presented that way. It could have been presented as, here is the managed card -- in modern terminology -- that you will use when you go to the Staples website.

So maybe it was just too early. Or maybe the nature of that certificate didn't lend itself to the embedding of assertions in an expressive and flexible way.

VB: Yes. Certificates cannot be managed cards for two reasons. One is practical and could have been easily changed. The metaphor could have been friendlier, as you say. But the other thing is that a certificate is a primary token, your credentials rather than your identity. It is the mechanism for proving that you are the person entitled to that specific key. If the certificate is given to me instead of you, it's the same. There is nothing in it that says it's you.

Your identity is instead something that is about yourself. When you use a managed card, you are leveraging a relationship that you have with somebody -- your airline, your government. It's true the certificate could be the enabling mechanism for expressing this relationship. But suppose I am a customer of Alitalia, and I have a card in my wallet that, when I show it to the right people, enables me to enjoy certain advantages that are part of my identity as a customer. But my relationship with this airline, the fact that I'm entitled to a certain right, can change.

JU: Yes. So if the right is hardcoded into the certificate, that's fairly static. As opposed to the more dynamic nature of the identity metasystem, in which attributes are exchanged on the fly.

VB: Exactly. The attributes that make sense in a specific context -- like if you do or don't have a certain privilege should come down dynamically. Embedding them in the certificate is dangerous. I have this conversation often with governments. They tend to think of transporting online what they already have offline. So if you have a passport, it's basically like a cached token. It's something that says yes, you can travel, yes, you are Italian. But online it's really better to give this information on the fly, for a number of reasons.

One reason is that you can encrypt the information directly to the relying party. When they gave my my passport, they didn't know that I would go to Iceland, or to the United States. They just gave me a blanket permission to travel. But online, I can present my passport in context that says I want to go to the US, and then the token that says yes, this person wants to go to the US, can be encrypted directly for the US embassy. Whereas a blanket permission, cached for use by everybody, would have to be accessible to everybody, which is dangerous.

JU: Right. You also do an analysis, in this chapter, of Kerberos, and how it has desirable properties but doesn't scale for the Internet. Can you explain that?

VB: Kerberos itself is really the basis for many of the interactions that we use. So this idea of having an entity that knows about you, and can make assertions about you, is there in Kerberos. The problem is practical. Kerberos is one specific technology. As such, it's something you can't impose on everybody. It's a system, but if we want to talk to everybody, we need a metasystem. We need to abstract the capabilities of Kerberos in a way that does not force every participant in a transaction to speak with Kerberos itself.

Also, Kerberos has a very authoritative view of the world. It is made for domains where one entity has complete control of everything and knows the keys of everybody.

JU: The omniscient key distribution center.

VB: Exactly. And the KDC knows not only about the subject, but also about the relying party. It has all the keys. In our world, that's not the case. When we say user-centric federation, we actually mean that it's the user whose choices instantly create a federation between the identity provider and the relying party. This is possible only if everybody has their own keys.

JU: And also if the claims that can be expressed are represented by URIs and are independent of any actor in the ecosystem. So if an identity provider and a relying part agree to synchronize on the use of some claim, and someone can provide that claim, conforming to that schema, then you can dynamically bring together a transaction.

VB: Absolutely. This is probably the main point. It's so important that in the metasystem we even take into account the case where we may not be able to pull that together. So we have the concept of claim transformers. If an airline needs a specific claim that cannot be produced by a known identity provider, but is available in another form, then we have mechanisms for bridging. But the general idea is exactly what you said. We should reach an agreement, at least for specific domains, about common claims.

This is actually pretty close to the idea of the semantic web. Although in my opintion, claims makes it more actionable. The semantic web tries to do everything, but with claims we are in a very specific area.

JU: So, in the media nowadays, you tend to hear the terms OpenID and CardSpace used almost interchangeably. In one a sense in which that's not inappropriate. There's a single-sign-on aspect where the two overlap, and in fact complement one another. But it would be helpful to spell out the deeper differences. This idea of sets of claims, and claim transformation, is one of the things that distinguishes the metasystem from what's happening, at least so far, in OpenID, at least as far as I understand it. The use cases for OpenID are mainly sign-on, and now with version 2.0 there's a move toward attribute exchange. Can you explain how the metasystem differs from what OpenID does now, or is likely to do in the near future?

VB: OK. Now I'm not an OpenID expert, so I hope any naiveties of mine will be forgiven. From what I know, every interaction happens by means of browser redirection. I find this extremely useful, because OpenID is actually a kind of omnidirectional identifier, which is something that sooner or later we have to deal with. Whereas cards are metaphors that help me to do things that are unidirectional. Every time I use a card, it's for a transaction specifically with one relying party.

The same happens with OpenID, but you have the perception that there's a URI which describes you. This opens the way to future developments which, in my view, we desperately need. What we see happening with Facebook is just a signal that the industry needs to do for omnidirectional identifiers what we are now doing for unidirectional identifiers.

JU: Can you define those terms?

VB: The idea is that your identity, or identity in general, can have different audiences. An omnidirectional identifier is something you use for being recognized by everybody. So if you go to the Verisign website, using HTTPS, their certificate declares their public identity.

Then you have unidirectional identities. So if I land on a website that, for business purposes, asks my age, then I obtain a token specifically for that website. We call this unidirectional. The flow goes straight to that website and nobody else.

JU: And this will map to attribute exchange in OpenID.

VB: Yes, they're very close. The point is that when you use a card today, or OpenID, you're in a unidirectional context. You're transmitting attributes to one specific relying party.

But in the case of OpenID, I have my account, vibro.openid.com, and it's a URI, it's my identifier, and it's omnidirectional in the sense that everybody knows it. While in my cards, there's nothing that I tell to everybody. So I think OpenID is a good starting point for thinking about an ecology of omnidirectional identity. How do I handle identity that I want projected everywhere, not just to a specific relying party?

So for example, Facebook Beacon. In my opinion that's a symptom of our need to think about omnidirectional identity.

Also, the concept of an identity provider -- in both CardSpace and OpenID -- is for giving you attributes about yourself. I go on a website, I want to buy wine, I am the one who is asking the identity provider to certify me. While in the world of social networks, the requester of an identity may be somebody other than me. If somebody is looking at my profile, it's not me. But the request is still for identify information about me. This is an area that needs thought. As an industry we did an excellent job with unidirectional identity, and the ecosystem for both CardSpace and OpenID is vital. But we haven't yet found the laws for omnidirectional identity. When we do, things like Facebook Beacon won't happen. We need to extend the conversation to include omnidirectional identifiers for users. A website has a public identity. But at this moment, a user's public identity is an imagined phenomenon. You search for yourself and find traces of your identity on the web, or maybe the identity of somebody who has your same name.

JU: Or someone who said something about you. Made a claim about you, in effect.

VB: Exactly. Also, a Gartner analyst recently wrote on his blog that he believes in the near future we'll need to certify the authenticity not only of poeple, but also of things like digital content. I believe that the ecology of identity needs to grow to encompass all of these things.

JU: I've been making this exact point recently. I see the blogosphere moving toward what we have now, at the high end, in scholarly and professional publishing. There, the papers that people publish have digital object identifiers which are being managed over the long haul, so that citations can be reliably managed. And so that claims can be made: this is not just a paper published by me, it was also peer-reviewed by these three other people. You start to build up a fabric of claims where the subject is the digital object, not necessarily the person.

Was this where you were going with omnidirectional identity, that I'm broadcasting these kinds of claims.

VB: Yes. With OpenID you have an omnidirectional identifier, or at least a handle you can use to gain these identifiers. We can do it also with cards, but we don't push it as a metaphor. Nor is OpenID pushing it as a metaphor, it's just a side effect. But I believe it will be useful.

Anyway, that was a long digression. Now I can get back to your question about how OpenID relates to CardSpace, and how they can work together. OpenID is very handy because it lives in the cloud, and it's easy to access. It doesn't intrinsically require passwords, which is fantastic.

JU: Yes. I have a completely passwordless OpenID account at myopenid.com now, and it's wonderful.

VB: It's beautiful. If I have both passwords and cards...

JU: ...there's still a weak link.

VB: Sure. If somebody calls me and says, can you please give me your username and password, and I give it, well, then, having the card didn't help me much. With cards only we eliminate one of the key weaknesses, not of OpenID itself, but of any browser-based interaction.

That said, the fact that you never leave the browser is a limitation. In many situations, like for a blog, it's perfectly OK. But people are not very good at interpreting the clues and understanding if they are on the right page. It's very easy to get redirected to the wrong place. We can put in safety mechanisms, but if the website is the complete master of what goes on in this universe, there will be attack vectors that you cannot avoid.

JU: Sure. This is the principle of consistent user experience, which is one of the seven laws. Point taken. You can't enforce that without a branded, consistent chunk of UI.

VB: But even if every OpenID provider were to decide that the UI for authenticating is exactly the same, if it's all within the domain of of HTML and JavaScript, then whoever initiates the experience can make you believe whatever they want, because they control your only window on reality.

When you use an identity selector -- not necessarily CardSpace -- your identity interaction happens outside the browser. The browser only asks the selector for a token.

Furthermore, an identity selector can secure things at the message level. The token you obtain can contain claims, but can also contain keys that you can use for securing messages, using WS-Security.

JU: This is one of the key distinctions. The protocols for OpenID are very light, and that's attractive. It's easy to get things done, it's quick, there aren't stacks of WS-* specs. That's clearly a reason why it's gaining traction. The identity selector piece is separate from the protocol complexity behind the glass, and we can talk about those things separately. One could imagine the very lightweight protocols of OpenID grafted onto identity selectors -- well, we have that now, I can use CardSpace as a front end to OpenID -- but the protocols being spoken are still very simple.

On the other hand, your chapter about WS-Trust, WS-Metadata Exchange, WS-Federation, that's the kind of thing that makes people want to lie down and take a nap.

[Laughter]

So what about that? How do you delineate the value of the heavier protocols, and how do you compensate for the difficulty of making effective use of them? Where's the sweet spot?

VB: In terms of the difficulty of making use, I would disagree. Every single time you use a card, behind the scenes you have all the standard negotiation with WS-Trust and WS-Security, and yet you are blissfully ignorant...

JU: ...as a user.

VB: Yes. From the user's point of view it is absolutely clear.

Now if you think of the complexity of Kerboros, or even TCP/IP itself, with its backoff algorithm when it has to retransmit packets, those things are pretty damn complex, but you don't care. They sink inside the platform. And in this area too, we are sinking into the platform. I'm sure you can remember a time when you had to install TCP/IP, or write applications for a specific monitor.

So, WS-Trust may be complex. In my opinion, not so, but then, my license plate is WS-STAR.

JU: Really?

[Laughter]

VB: Yes. So I'm biased. But in general, the idea is that those protocols are more complex because they're trying to address a broader range of scenarios. So for example, there is no assumption of HTTP. Everything happens at the message level. So things can work on any present or future transport protocol.

JU: Although in practice...

VB: In practice, today, it's HTTP, and in fact we are optimized for HTTP.

JU: So, I'm a complete agnostic. I see scenarios where REST makes sense, and scenarios where WS-* makes sense. The latter, to me, always comes down to cases where you have declarative policy. It's not just a conversation between a couple of endpoints. There's a set of transactions embedded in a policy fabric, and by being able to flow through intermediaries, which can make claims transformations, which can assert policies, which can require that certain kinds of credentials are used in certain contexts, which can audit and monitor and do all those kinds of enterprisey things -- it's that class of scenario for which this more advanced functionality is designed.

I think the problem is that it's easy to say, look, we have all this stuff on the web, and the web just works, therefore this is the right and only and best way to do it. Whereas if you talk to people who are involved in, say, the secure exchange of medical information, and there are multiple stakeholders asserting claims and policies about how that information is going to flow, then you do get to this place where you need stuff that's just harder. It is irreducibly harder to meet those requirements. And I could be wrong, but I don't think is saying that OpenID aims to occupy that ground.

Then it becomes a question of where you get the support that enables you to do those things. Microsoft is putting together a strong story around the framework, the tools, WCF, so if you want to live in that ecosystem and can operate homogeneously, then it's great. But things never are homogenous, so that gets to the issue of interop.

VB: All the scenarios you mention are within the scope of WS-*. But also, now, we want to be able to do more complex things directly from the web. Things like accessing your bank account, or using your financial information to apply for a mortgage, or accessing your medical records. Those are all things that require enterprise-level guarantees, and areas where WS-* can help.

Part of my job is flying around the world, talking with governments and other big players interested in this new generation of technology. I can tell you that they are very protective of their data, and they need to provide very strong guarantees to their citizens, their patients, their customers. OpenID is an extremely specialized animal. It's great specifically for the web. It's a child of our times. People are tired of remembering many different credentials, and who can blame them. OpenID is a great way of overcoming that problem.

Then there are scenarios where you need to be able to model existing business relationships. With WS-Policy and WS-Metadata Exchange, their power is the ability to describe a situation that already exists, so that you can leverage online what you already have in place in the offline world. So if I'm a citizen and that fact is expressed in terms of a managed card, then I can use my privileges online automatically. I don't have to renegotiate everything with every relying party online.

With WS-* you can express these things, and since it's a meta-protocol you have a decoupling layer that enables you to describe your business situation without committing to a specific encryption or authentication technology. And here we come to interop. This is one of the most heartfelt topics in this area, and there is a constant effort to keep the stuff real. If you check Mike Jones' weblog, self-issued.info, he talks a lot about this effort. He's involved in organized, for every identity-related conference, parties in which everybody brings his own technology and we build the Cartesian product of everything talking to everything else. They publish their results to a wiki, and I can tell you it's impressive. That table, which started pretty much red, is working toward green at a steady pace. And every time they hold a new event, new players come to the table.

JU: So the part that's easily visible to folks now is CardSpace to OpenID. Anyone can set that up, use it, and see what it's like. The part that's not visible to people, but that you see in your travels, visiting governments and businesses, are these more advanced scenarios. At what point will this become more visible? Because until it does, it all feels kind of abstract and remote, doesn't it?

VB: Absolutely. So, it's really hard to answer. In the last two years, we engaged with every big name you can think of. Everybody's extremely interested, because they can see the disruptive potential. But it's hard to say. What I can tell you, and it's a matter of faith, so you can choose to believe me or not, is that a lot of people are really serious about CardSpace, and are building prototypes and pilots that are internally up and running.

JU: Fair enough. So, we haven't said a lot about the book specifically, but having written one myself, I know the incredible level of effort and commitment that it takes. Your title is Understanding CardSpace, an