IOT

Pegasus Mission Annual Report – 2016

wmattlong2015 IOT, Mark Nichols, Matt Long, Near Space, North American Eagle, Real-Time IOT, Uncategorized

Overview

It has been a remarkable year for The Pegasus Mission as we flew to new heights in the upper atmosphere and traversed a desert at extreme speed. We engaged over 20,000 in a global audience in 60 countries with captivating adventures centered in STEM and experimental real-time technologies running in the MS Cloud. Our team grew to 60 volunteers and partners that showed unwavering support and demonstrated the power of the human spirit to challenge assumptions about is possible. The payoff was massive real-time engagement with a global audience and acceptance by the media for our collective efforts. Over 25 million people viewed 30 plus articles written about our adventures in near space and in the remote Alvord desert.

Pegasus IIUpperAtmosphere

We made history with Pegasus II by flying 2 hours into near space and bi-directionally communicating with a global audience in real-time. Using apps for Web, mobile, and Media Services, thousands of people experienced the flight from 60 different countries as we transmitted video, telemetry, and received messages from well-wishers through our experimental technology running in the MS Cloud.

The impact was overwhelming and it deeply connected with a global audience of interested parties. Our charter of challenging assumptions about what is possible and making research an experience became a viral meme in 2016 as soon as Pegasus II lifted itself into the heavens.

North American Eagle Partnershipimg_0279

Our partnership with the North American Eagle (NAE) team built on our experience with Pegasus II and enabled us to synthesize technology to enable users to experience extreme speed in different type of hostile environment, a desert devoid of infrastructure. Using satellite communications and our experimental technology in the MS Cloud, we gave 15,000 people a real-time experience of the drama of extreme speed, i.e., 478 mph, across a remote desert floor. The outpouring of the emotional user messages we received told the same story as Pegasus II…it had deeply connected.

It is with endearing gratitude that we thank the NAE team, Ed Shadle, and Jessi Combs for allowing us to participate in a remarkable risky pursuit of a challenge that only a few will ever take, extreme speed.

Accomplishments – 2016

The Pegasus Mission derives itself from a trifecta of concepts; research, experimentation, and human consumable experiences that form a powerful gestalt when combined with taking risks to achieve what has not been done before.   Our accomplishments in 2016 demonstrated that these challenges resonate broadly across a spectrum of interest levels. Technically, we showed how it was possible to build an open-system of heterogeneous system actors on a global basis that could behave organically and communicate in real-time with the latency of an online video game, and did this will great scale and economic feasibility. We simplified communications and decoupled the system actors, which allowed us to build unique experiences that touched people from impossibly remote and hostile environments. We took the initial steps in democratizing IoT and we were rewarded with the endless appreciation from the audiences for taking them on risky scientific adventures where success is never a guarantee. Together with our partners and our dedicated team of volunteers we brought a piece of an amazing future into the present.

Looking Forwarded – 2017

2017 is shaping up to be an incredible year. Our top priority is the total eclipse of the sun occurring over North America on Aug. 21. This is a one-in-a-lifetime event. We will be creating a remarkable global event with “Flight into Totality”, where we will fly to 80-100K feet into the upper atmosphere and film the total eclipse from a floating and self-stabilizing platform.  We will provide Web and Mobile apps for a global audience to participate in the flights. To remediate as much risk as possible we will be launching redundant flights for 4 different time zones. This will be fulltime work to engineer, test, promote, and prepare for an opportunity that will not reoccur. We will expand on the number of ways users can participate and leverage more technologies. This is complex mission and time is short to begin the work. We plan to build video documentaries and promote them as we prepare for flight to amplify messaging, grow the audience, expand our connections, and capture mind-share. Using the videos and attaching to social media, we plan to build an audience that will become participants as we attempt to deliver another risky and perhaps astonishing mission to near space.  This is considered an “at-risk” project due to requirements of time and funding that we must pursue.

We plan to re-engage with North American Eagle for another run at history. One of the lessons learned from NAE was how well it connects with people, but how little has been done to message the enormity of the challenge or the engineering required to safely travel at extreme speed.   We believe that we can build a story around the challenge versus the land speed record, and maximize real-time engagement of a global audience. Our impact will be to deliver the extreme speed experience in real-time as the NAE makes it runs, and increase the ways we creativity engage with the audience.

In summary, we will be engaging more people for these events in real-time, telling true stories that capture the imagination, and putting leading edge technologies to the ultimate test. By doing what has not been done before, we energize, collaborate, partner, and go forth to challenge assumptions about what is possible.

Dare Mighty Things

Advertisement

The Story of Challenging the Land Speed Record with North American Eagle

wmattlong2015 Ed Shadle, IOT, Land Speed, Mark Nichols, Matt Long, North American Eagle, Piraeus, Real-Time IOT

The Pegasus Mission’s involvement with North American Eagle (NAE) started with the idea that Pegasus team could provide a real-time view into what was occurring with the vehicle to a global audience. This would be done by placing a device inside the cockpit of the Eagle. The hope was that we could transmit telemetry and receive user messages, i.e., bidirectional communications, from around the world over a 10.5-mile course in a remote location, no electricity or reliable cellular communications. We would do this is milliseconds, not seconds, without staging data or any polling, using our advanced experimental operational technology, Piraeus.

The reason this challenge is important, it is something that commercial environments experience, and is difficult at low latency. Companies engaged in natural resources generally are operating in remote locations and getting granular information into and out of these locations is certainly a challenge in just a few milliseconds. NAE not only provides this environment, it provides the opportunity to interface with global audience to test such a system a scale. Pegasus can take a challenging set of commercial circumstances and turn it into a participating and fun experience for the audience with NAE. The audience becomes part of the experiment, i.e., experiential research.

Here’s the catch…it is not possible with “stock” technology, and nobody has ever tried what we were about to do, especially at 400+ mph in a remote desert. We believed it was possible, but you must do it to find out and learn from it.

NAE team of about 40 dedicated souls on an ongoing mission to break the world land speed record. They are challenging a ~20-year record set by Andy Green, and doing it on shoestring budget. NAE is the place where STEM meets human experience to achieve the impossible. The NAE team must continually leverage their resourcefulness and constantly innovate to get the vehicle optimized for speed and cooperate with the pilot. The NAE’s effort is nothing short of heroic. It is with dogged determination that team manages a complex world of mechanics and engineering to perform a set of tasks seldom, if ever, experienced. Pilots, Ed Shadle and Jessi Combs, use their unique pilot/driver skills to operate the Eagle at extreme speed. Their skills are a glorious trifecta of athlete, engineer, and steely astronaut.

The meeting of the minds between NAE and Pegasus Mission intersects in that both teams are trying to challenge assumptions about what is possible. There is no guarantee of success, except lessons will be learned and used in the future for improvements. The secret is having the WILL to try at this at the intersection of STEM and the limits of human capability. We are locked together by some ethereal compelling force that overcomes the fear of failure with need to try.

When a phone app or Web site lights up with the real-time telemetry, it is an exciting experience during one of the NAE runs. However, what remains hidden is the difficulty and logistics of execution, which is Herculean. The NAE must transport an incredible amount of materials to the site over an 11-hour journey as well as the massive vehicle. It takes days to setup and requires 40-person team to accomplish. The Alvord Desert is devoid of everything from spare parts, fuel, generators, rest rooms, sleeping quarters, food, and water. If you do not bring it with you, it is not available.

The Pegasus logistics are not as massive, but considerable. We carry a 507 lbs. satellite communications network and another 200-300 lbs. of equipment over 180-miles across the mountains only to be faced with a 32-mile journey on an unpaved road across jagged volcanic rocks to get to the site. Once present, we face a 260-mile commute and restocking provisions daily. This is the price of admission to attempt to meet the challenge.

Once operationalized for runs, i.e., 2-days for NAE, Ed Shadle must attempt a test run to determine the operating condition of the Eagle and its behavior on the desert floor. This a critical test and gives the entire team the opportunity to check-down procedures and equipment. The Pegasus Mission must determine whether our device, field gateway, local network, SATCOM, and applications are performing as expected. The first 2 runs where short and Ed Shadle aborted due to steering issues. This means the engineers and mechanics of the NAE must kick into high-gear to solve the problem before time runs out…the clock was ticking.

The steering issue involves the pilot’s stick commands being responsive to the steering box of the Eagle. A decision was made to try and deliver parts delivered via a daring night flight from the Tacoma, WA area to the desert floor at Alvord. The NAE team would put up flares to define a runaway and have well lighted truck to designate the beginning of the makeshift runway for the pilot. Pilot, John Richardson, sign up for the attempt to deliver the parts to the desert floor with 4500 foot mountains surrounding the area in total darkness.

The NAE team spotted a plane around midnight, but it did not respond to signal flares. It was John, and he was concentrating on dodging mountains. He missed the signals and proceeded further south. Unfortunately, John mistakenly took a well-lite cattle truck as the runway marker and began a very slow descent into the abyss of the darkness. The plane touched down in a field and immediately ripped off the nose gear. The plane then stuck a fence, ripping out about 200 feet of it. The plane was destroyed, but John was unharmed and alone in the darkness near his plane.

The NAE team kicked into rescue mode and began to decipher John’s location. Through a serious of odds events that entailed coincidence and providence, the NAE team determined his approximate location and rushed into action. Several hours later John was at the base camp unharmed. His first words after being rescued, “I got your parts.”

The morning began we a flat tire during our commute to base camp. The NAE team was making corrections to the steering to prepare Ed for another test run. This time in the late afternoon. It worked reaching 362 mph and traveling approx. 10 miles. The Pegasus device maintained contact and we could stream telemetry and receive user messages into the cockpit from the global audience in real-time. Our only issue was the desert heat was killing the GoPro cameras in the cockpit.

The Pegasus team was excited with the anticipation of our functioning system for the next day’s runs. Jessi Combs would be the pilot and attempt to break the women’s land speed record. We began the commute out of base camp only to get another flat tire on a different vehicle (same road). A vehicle appeared out the darkness from behind. It was a Les Schwab Tire Service operator. Confounded by this luck, a large man named “Tiny” exited the truck and offered us help. “Tiny” changed the tire in about 5 minutes and we were ready to continue our daily extraction routine. Refusing to take any money for his services, “Tiny” wished us luck and sped away into the darkness from which him came. 42-miles of desert road and this was the only vehicle we saw that night.

The Pegasus field team Mark Nichols, Dana Johnson, and myself, would be cut to just 2 for the last day. Dana would have to return to Chicago. We entered the entrance point to the desert floor, only to find the private road we used locked…it did not open until 10AM. We needed to find an entrance that allows us access and would not blow out a tire. Jessi Combs and the Eagle were going to run a with or without us…time was short. We lumbered the SUV over a “path” and could gain access to the desert floor. We quickly sped to the base camp only to find the Eagle preparing to move out to the start line.

Mark headed to the Eagle to switch on the device, then turned on the SATCOM which it locked on its satellite. He jumped into the SUV and rolled out with the drone to film from mile 3. I headed to the communications truck with Steve Wallace and connected the field gateway to the local network, device, and SATCOM. We were operational in short order.

Jessi Combs was ready to begin her run with a moderate wind coming from the southwest. This meant the Eagle was being pushed to starboard, i.e., right. Jessi powered up and begin to move. Only a few seconds later powered down again. She felt that the stick was not being responsive to the steering box. With a quick check from Les Holm, she was convinced it was responsive and powered up again. She was going to make a run.

There were approx. 40-people involved in the run and only a few at base camp. The communications truck, where Steve Wallace and I sit is the only place where speed can be determined during a run. Jessi was accelerating and had 4500 of her fans watching the telemetry and sending messages to her in the cockpit. Jessi powered down and Steve Wallace called out on the radio to signal to the recovery team. This was not a speed run, rather a test run. Jessi would have one last shot in the afternoon.

The NAE team moved into high gear to prepare the Eagle for another run. Meanwhile, Jessi huddled in the communications truck with Mark, Steve Wallace, and myself to analyze the stick position and responsiveness of the steering box. Steve pulled up the graphs on a display and Jessi talked through what she experienced. There is a “blimp” in the telemetry suggesting that Jessi’s moved the stick quickly toward one direction. Mark pulled the cockpit camera video from the Pegasus device and we watched to see if Jessi had hit a bump. The answer was no. Steve had to chalk up the “blimp” to a brief power surge and the remaining telemetry appeared responsive to stick commands.

The next item for Jessi was concern that the nose weight may have been lighter than expected. Steve checked the telemetry and saw 800 lbs. on the nose, a safe number from engineering. Jessi is a highly aware driver/pilot and confidence is critical to performance. If she is feeling the nose lighter than it should be, then a correction needs to be made.   This led the NAE team reset the canard on the Eagle to slightly increase the downforce on the nose. The Eagle and Jessi were ready for run 2.

It was late afternoon, when the Eagle was towed to the starting line. The winds had been low throughout our time in the Alvord Desert. They were now blowing and that meant a dust storm. The Eagle was holding for winds and visibility. We had been forewarned about the potential for dust storms, but experiencing it is a different matter. Everything is covered in dust, including yourself. We waited in misery, while was Mark out on the course with the drone battling wind, dust, and cameras falling from tripods.

The winds began to subside and the course visibility cleared. Jessi climbed into the cockpit. The NAE team in position, Steve switched on the onboard telemetry, Mark got the drone airborne, and I was broadcasting telemetry and receiving user messages to and from the NAE.

Jessi powered up and roll off the start line. Her run started on course and she was fast. The power of the NAE is tremendous, speed is only a question of when she aborts and powers down. Jessi knew in that later part of the run she had drifted slightly to port, i.e., left. Her keen seen of awareness, like an elite athlete, told her it was time to abort and powered down. She successfully stopped the NAE only a few feet off the desert floor in the flora. She had pushed it to the limit of speed given her course. Everything was well and racing now over. It was time to get the top speed and fastest mile from Steve Wallace.

Steve calculated the top speed at 477.59 mph and the fastest mile at 471.55 mph. It wasn’t greater than 512 mph, the women’s land speed record, but that did not matter. Everyone done their best and was safe. There will be another attempt, but on another day.

Pegasus field team, now 2, packed the 15’ UHaul with the satellite gear and SUV. It was time for us to extract and head for Boise, ID. We crept across “Tiny’s” road trying to avoid another flat and began the 5-hour drive back across the mountains complete with reflection on what we had witnessed for a week, and covered in dust.

The Pegasus Mission with the help of our new desert racing friends, again plowed new technological ground and bidirectionally communicated on a 10.5 mile stretch of desert floor to a global audience in real-time.

The NAE team, Ed Shadle, and Jessi Combs have pitted themselves in a race for a land speed record. In the process, they have made us all proud members of the Human Race.

 Dare Mighty Things

A Users Guide to Pegasus II

wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT

Pegasus II is an experiment in real-time “Internet of Things” technologies from the edge of space.   The experiment uses a High Altitude Balloon (HAB) as a delivery system to carry a payload crammed with sensors, video, and radios into the remote and hostile region of the upper atmosphere known as near space.  The ascent of the flight will terminate at 100,000 feet above the surface of the earth. The experiment allows users to participate in the flight in several ways.

We are scheduled for launch on Wednesday April 13th at 12pm CDT.

(1) View telemetry from the craft in only milliseconds from when the measurement is taken from anywhere in the world.

(2) Watch a live video broadcast of the flight and launch. Note: Flight video is highly experimental.  We will learn much from this flight in terms of live video performance from remote UAVs.

(3) Communicate with the craft during flight by sending messages to the craft. These messages get recorded into the video record of the flight.

(4) Receive SMS text messages on your phone as Pegasus II makes flight milestones.

(5) View a live map that updates the positions of the chase vehicle and Pegasus II as it flies.

There are 2 ways to include yourself in the experiment, Web and phone.

Best Experience

  • Go to the Web site and sign up for flight notifications.  These will allow Pegasus II to text your phone when it launches and you will no miss the flight regardless of whether you are using the Web site or mobile apps.
  • Follow us on Twitter @PegasusMission.  This will keep you informed on any flight delays and alert you of time-to-launch.

Web Site

  • Watch the live launch and flight video.
  • Watch live telemetry and mapping during the flight.

Mobile Apps

  • Download and install the mobile app by searching your app store for “Pegasus Mission”.  If you have already installed the app, we suggest that you uninstall and reinstall the app to help us ensure consistency.
  •  Connect to WiFi if possible.  This will help keep the connections “hot” and reduce dropped connections.  Reconnections can be slow if the signal is not strong.
  • Watch live telemetry and mapping
  • Send us a user message.  We love getting “Hello from <location>” from the users around the world.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The Pegasus Mission and You

wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT ,

The Pegasus Mission is all about bringing you a near space adventure, an experience in High Altitude Flight that you may have never witnessed before.  It is about you.

You can help us out promoting the flight of Pegasus II Oct 5 by using your social networks to promote the flight and get users to sign up for SMS notifications and the craft will send a text message to users to alert them of the launch.

Pegasus II is high altitude mission to near space providing a global real-time experience to users from the upper atmosphere using Microsoft Azure and experimental technology.

Windows and Android apps available in app/play stores for Pegasus II’s mission to near space.  iOS app submitted last night and hopefully will be available by flight time Oct 5 ~10AM MDT from Cheyenne, WY, USA

Search “Pegasus Mission” to find the mobile apps.

Warning:  This mission has no guarantee of success. The mission team is working in a remote and hostile environment with high sensitive equipment nearly 20 miles above the surface of the earth. However, attempting to try what has never been done before is courageous and noble effort itself in an effort to learn and provide a unique experience to others.

High Altitude Science Firsts

  • Real-time video transmission from a HAB
  • Coordinated global broadcast of real-time telemetry and mapping through Web and mobile apps
  • Global user communications direct to HAB inflight
  • Craft notifications via SMS to users
  • Widely available HAS mission that is publicly and globally consumable
  • Real-time communications to/from HAB

Mission Objectives

  • Low latency, high throughput, linearly scalable and bidirectional communications
  • Mobile app with real-time experiences
  • Web site with real-time experiences
  • Security: Distributed authorization
  • Altitude: 100, 000 feet (19 miles, 30.5 kilometers)
  • Live video feed and onboard video from near space
    • Curvature of earth
    • Edge of visible atmosphere
    • Blackness of space
  • Onboard 3D view (5 cameras up, down, 3Xout)
  • Real-time broadcast of telemetry and mapping
  • Real-time communications between users and craft
  • Telemetry capture
  • Remote Intelligence: Automated flight operations from data center 1,000 miles away.
    • Live camera rotation
    • Delivery system release
    • Low altitude high speed main parachute deployment
  • 2K users watching flight (aspirational)

Resources

Web Site: https://www.pegasusmission.io (Live video + telemetry + maps)
Sign up for SMS flight notifications and craft will text you when it launches.
Mobile apps (Live telemetry + maps + sending user messages to craft inflight)
Facebook: http://www.facebook.com/pegasusmission
Twitter: @PegasusMission
Blog: http://pegasusmission.com
LinkedIn: “#PegasusMission”

Dare Mighty Things 😀

Unique Exchange of Equity

wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT , , , , ,

When I look back at the nexus of “The Pegasus Mission” Mark and I made a single decision that transformed the challenge into something grander than we initially imagined. Because we were working with real-time, open, and distributed systems coupled to IoT, we needed something compelling that placed an emphasis and reason to the participate in the technology. We did not want to trivialize the technology in any way, and that led to a focus on driving a unique user experience that people had never seen before. By focusing on the entertainment value and taking users to a place they could not travel, we could passively couple both research and technology. It is a unique exchange of equity between the mission team and the users. The users get an incredible experience they could not otherwise obtain, and we get to test the premise of the research and the technology while they do it. While we initially believed our thoughts were original, they were not. The concept was originated a long time ago by a great explorer, Jacques Cousteau.

Perhaps you may remember watching “The Undersea World of Jacques Cousteau” on television when you were much younger. The Cousteau team would travel to remote locations and explore the oceans. This was in a time where underwater filming requiring highly customized equipment and scuba diving was not something many people could do. In fact, Cousteau invented scuba diving with his aqualung. You probably cannot remember the name of single researcher onboard their ship, the Calypso, or any of the research they were doing…but you remember the marvel of undersea world shown to you for the first time. We were absolutely enthralled watching the Cousteau team take risks below the surface in order to gain an understanding of what was down there. The act of watching the television special enabled us to be drawn into a marvelous exchange of equity, because that research, that we cannot remember, would never have been possible without us watching it. Ultimately, we all benefited from our own participation because the Cousteau team help make our lives better through the understanding and research that they did in the oceans around the world. It was an ironic twist that we can in some way thank ourselves for making the oceans a better place.   Our mission to near space is fundamentally no different and depends on users watching flight through our Web Site or phone apps.

With less than 2 weeks away from the launch of Pegasus II, where we take you not down into the oceans, rather into the heavens 20 miles into the upper atmosphere. We will test live video onboard the craft for the first time to bring you a completely unique experience from three times higher than where commercial aircraft can travel. You will see the curvature of the earth, the edge of the atmosphere, and the blackness of space in real-time. Additionally, we will stream telemetry from the craft, and map of the showing the position of the craft and our chase vehicle, all in real-time. Finally, we will let you send a 40 character message to the craft in flight and get your personal message into the flight video record.

While the endeavor is not without risk and success is not a guarantee. Unlike a television special, we will be executing the mission live and I can guarantee will it not be “perfect”.  Innovation is an inherently a risky proposition and it takes a courageous team to live it out and challenge assumptions about what is possible…and attempt to bring you an unforgettable experience. We can only try our best and I believe that the having the fortitude to attempt what has not been done before is noble effort in itself.

You can sign up for SMS flight notifications and we will text your phone when Pegasus II is launched https://www.pegasusmission.io

My thoughts on Innovation from nearly 2 year ago. https://www.youtube.com/watch?v=uMVix6p33Zs

Pegasus-II

Aside wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT

We are getting ready for Pegasus-II.  We will target the American West for the flight slightly East of the Continental Divide.  We hope to get some outstanding video of the Rocky Mountains.  Mission parameters will include LIVE video where users will get to see the Pegasus “Eye-In-Sky” using Azure Media Services as well as a target celling of 100,000 feet.  Not to mention an additional 4 or 5 cameras onboard the craft.

We have upped the ante with live video, but also using 38 sensors capable of streaming the telemetry in real-time to Phone Apps (Android, iOS, and Windows Phone) as well as a map to display the position of the Pegasus-II and the chase vehicle in real-time.

Adding to the game plan, we will be allowing Web site and phone app users to place information onboard Pegasus-II while inflight and supply these users with an incredible photo of the craft at its apex as a thank you.

It’s game ON for Pegasus-II and pushing the limits of real-time IOT.

Security in Piraeus

wmattlong2015 IOT, Real-Time IOT

Piraeus’s architecture is very intentional to reduce the amount of effort needed to leverage the system. The security of the system is built on a concept that an “authority” that can manage resources within the system, e.g., create topics or subscription grains within Orleans. Those grains are ACL’d by access control policies such that only authorized callers can access to those grains. The callers can be anything that can present an authenticated security token where the attributes of the security token comply with the access control policy of the resource being accessed.

The architecture greatly simplifies how Piraeus functions and also reduces latency within the system. The key is access control and how Piraeus manages this. Piraeus puts the user in control of what resources can be access by what callers. It does this through distributed access control policies. These polices can be cached, expired, and renewed by such that they can change within Piraeus without every stopping or redeploying the system. This clean separation of access control management from Piraeus enables to it be both fast and flexible.

Piraeus

One of the new technologies, other than Orleans, introduced in Piraeus is Claims Authorization Policy Language (CAPL). CAPL is a powerful, but simple logic-based and security token agnostic language for access control that enables rapid authorization decisions. CAPL policies can be created externally to Piraeus and then consumed to provide control over resource access, which reduces the complexity and increases the flexibility within Piraeus.

Topic grains within Piraeus include metadata when they are provisioned. Two important pieces of metadata are URIs that reference access control policies for the topic and subscription grains. Piraeus uses these references to return CAPL access control policies to ACL the resources from a service. We call this authorization service, Authorization as a Service, or ZaaS. ZaaS is where the access control policies are managed and Piraeus simply uses the ZaaS policy store to get and update the access control policies for the resources.

We want the access control system to be low latency and as such Piraeus will retrieve the policies when the Topic is provisioned. The policies will be cached locally as well as in Redis. When the policies expire from cache, they will be retrieved again and updated in the local and Redis cache. The Redis cache allows Piraeus to scale its gateways and make the access control polices available to the entire system. Once the policy is retrieved from Redis, it is cached locally to further speed up the process. When the policies expire locally, they are immediately refreshed to the local and Redis cache from the ZaaS service.

The security architecture greatly increases Piraeus’s flexibility because identity and attribute stores are completed separated from the system. Piraeus can scale and be leveraged in multiple data centers without the burden of porting these stores and/or synchronization because of this segregation, something that would be not possible with CAPL.

The benefits of the security architecture actually increase when you consider this clean separation.  Because we are using security tokens with attributes (not connection strings), this means that no caller is trusted, e.g., device, user, etc.  This is more in line with the scale provided by federated identity that is commonly used in Web sites.  Like I say, “If you don’t trust billions of people, then why would you trust devices when they will exceed the number of people on the Internet in the near future.”  The Piraeus architecture builds on this and opens the flexibility on how devices acquire and manage security tokens.  Piraeus does NOT manage devices and provide tightly coupled mechanisms for security token acquisition…by design.  The reason is that scenarios are so diverse for device registration and management that it needs to be either customized or leverage any existing product.  Since Piraeus is not a principal in the identity transaction, it simple requires that security token presented be by a trusted issuer.  Access to resources by the caller are based on the attributes of the security token as executed by a CAPL access control policy.  Again, cleanly separating the issue of token acquisition and management from the Piraeus Architecture.  Roll your own, use an existing product, just present a security token signed by a trusted issuer to Piraeus and it functions securely using CAPL to maintain access control to resources.

The Power Of Self-Organizing Systems

wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT

Pegasus-I was a system with the overall goal to broadcast telemetry and control flight operations in real-time. The system was a combination of organized and self-organized sub-systems that interacted with Pegasus-I during the flight. Let’s examine how we structured this and role of self-organizing systems in the mission as well as how we will leverage them for Pegasus-II and Pegasus-III in the near future.

A system is an organized set of things that form a complex whole designed to achieve a goal. The system designed and used for Pegasus-I included seven (7) interconnected actors, Pegasus-I, Ground Station, Chase Vehicle, Web site, and three (3) Azure Blob Storage containers. The nodes could transmit, receive, or transmit and receive information depending on their function. The overall system was composed on nine (9) different sub-systems that managed telemetry and flight operations. These sub-systems communicated with the actors to execute specific tasks within a sub-system.

Three (3) sub-systems were organized, meaning we had predetermined and configured paths for the information to flow. These sub-systems were associated with storing telemetry received from Pegasus-I by the Ground Station and the Chase Vehicle as well as storing the location of the Chase Vehicle itself. These sub-systems were required to be organized because our storage containers in Azure Blob Storage do not connect to Piraeus. Piraeus requires a durable subscription to be configured to any passive receiver. Below in [Figure 1] shows the system actors and organized sub-system graphs for telemetry and location. You will notice that the only organized sub-systems simply ingest telemetry and location from either the Ground Station or Chase Vehicle to Azure Blob Storage. The Web site does not send or receive any information through these sub-systems.

Figure 1

Figure1

Four (4) of sub-systems were self-organizing, which means they organize themselves on-the-fly to create sub-systems that did not exist before. If these new sub-systems create ephemeral subscriptions to receive information, then those subscriptions will only last for the duration of the connection by the caller. They are disposed upon disconnect. The reason we made this choice for Pegasus-I was that certain sub-systems were only concerned with “now”, not any past history of events to or from the inflight craft. This also reduced the complexity of the system design because we could depend on Piraeus to create the sub-system graphs in Orleans on demand and connect them to the appropriate parties immediately.

When the Web Site connected to the Piraeus gateway, it was entitled to subscribe to the same topics grains in Orleans as the Azure Blob Storage containers. This allowed the Web site to create new sub-systems on-the-fly to receive telemetry and location, shown in [Figure 2]. If the Web site disconnected, both the new subscriptions and their respective observers would be disposed and upon reconnection the sub-systems would be recreated by Piraeus within Orleans.

Figure 2

Figure2

This takes care storing and displaying telemetry without requiring complex configuration or maintenance, but what about those critical flight operations for delivery system release and parachute deployment? We used the same ability for self-organization for those also. We only need to configure topics [Figure 3] for the Web site to send specific commands to the responsible parties, Ground Station and Chase Vehicle.

Figure 3

Figure3

Once the Ground Station and Chase Vehicle connection to the Piraeus gateway, the sub-systems were created [Figure 4] and communications enabled from the Web site to these parties.

Figure 4

Figure4

When you look at the entire system and its sub-systems [Figure 5], it is a complex system. However, using self-organizing systems within Piraeus, we were able to take advantage of creating the sub-systems without the need to configure all of them in advance. We only configured seven (7) topic grains and three (3) subscription grains in Orleans to design the entire system and enable communications and storage so users could view the flight in real-time and allow Mark and I to control flight operations.

Figure 5

Figure5

Instead of describing the various uses of self-organizing systems, which are numerous, I want to communicate how the planning for Pegasus-II and Pegasus-III will use them. We want to be able to bring the excitement of high altitude science to people in a very personal way and allow people actively participate in the experiment in real-time. In our own way, it is like being onboard the Calypso with Jacques Cousteau. Doing this requires that we leverage not only a Web site, but also phone apps to broadcast to users real-time telemetry, maps, and streaming video through Pegasus’s eye-in-the-sky. These phone apps will receive telemetry, location and update rapidly from 100K feet in the upper atmosphere to user’s eye. Additionally, we working on concepts to get some user-defined personalized information onboard the craft during flight such that users can directly communicate with Pegasus and see personal message during flight. However, we cannot control the number of people using the phone apps or when the users choose to turn the apps on or off, or simply a phone dropping a connection due to poor reception. Therefore, we need a self-organizing system for these users, and Piraeus supports just that for this type of experience.

-Matt Long

Orleans Above the Cloud – Piraeus Overview

wmattlong2015 High Altitude Balloon, IOT, Near Space, Real-Time IOT

Disclaimer

The “Internet of Things”, IOT, is a big topic and involves many important concepts about systems, e.g., open vs closed, organized, self-organizing, closed and open-loops. All are worthy topics to understand to put a foundation under IOT. However, I will resist these topics and only discuss Piraeus and the role that Orleans has in the architecture for the sake of brevity.

Piraeus is a multi-channel, multi-protocol, in-memory event broker. It enables edge devices or services to connect and transmit or receive information from other system entities without any coupling and without system entities having direct knowledge of each other. It is a high throughput, low latency, and linearly scalable Operational Technology that simplifies the ability for an open-system to achieve its goal.

Diagram of the physical architecture used in Pegasus-I, i.e., how we got the real-time out Piraeus and Orleans.

PIArchitecture

The Piraeus Architecture (Operational Technology)  that was used for Pegasus-I

PiraeusArchitecture

A system can be modeled as a directed graph and such Piraeus uses this simple construct to enable communications through its gateway to components within a system or sub-system.  Information enters Piraeus through its gateway, then is distributed throughout the graph.  The leaves of the graph are connected to either active or passive system components that receive the information.

The components of Piraeus are relatively simple. A gateway exists that can receive information through a variety of channels and protocols. Once this information is received by the Piraeus gateway it is fed into the appropriate graph for that specific information topic. The node that receives this information is a virtual actor, called a grain inside of the Orleans host process. This type of grain is considered a “topic” within Piraeus and topics are graphed to “subscription” grains, which Orleans can communicate. Once the information enters the topic grain is fanned out to all subscription grains associated with the topic. The subscription grains are observable and feed information directly and immediately to the specific channel and protocol that is associated with the subscription.  There is no polling in the entire Piraeus architecture…ever.

That is a high-level summary, so let’s dig deeper into what is happening under the covers. Topic grains are always provisioned prior to communications being established. They describe the head node or ingress resource of a sub-graph as part of a system or sub-system. The topic grain’s job is to act as a resource for subscription grains to attach and fulfill the sub-graph enabling transmitter and receiver to communicate unidirectional. This gives us consistency with how a generalized system would behave.

The subscription grains can be either durable or ephemeral. Durable subscription grains are configured with metadata and attached to a topic grain. The metadata is part of the subscription grain-state within Orleans and therefore requires no orthogonal look up or database to manage that would impact performance. Durable subscriptions are used by either active or passive receivers. An active receiver is one that creates a connection to the Piraeus gateway. Once that active connection is established, the identity of the actor is used to create and associate “observables” with the specific subscriptions grain(s) for that identity.  When information flows into the subscription grain from the topic grain, the observable for the subscription is already established and associated with the channel and protocol of the receiver. This creates a direct pipeline between transmitter and receiver.  A passive receiver is a service that does not connect directly to the Piraeus gateway. When information is passed to these durable subscriptions, the subscription grain will forward the information immediately to the service. Piraeus supports RESTful Web services, Event Hubs, Azure Blob Storage, and Service Bus for these passive subscriptions.  This means it is possible to create, e.g., to create Web service, and start receiving information from Piraeus without doing anything else.

Ephemeral subscriptions can be used with self-organizing systems.  These are systems that can organically change and system actors can enter and exit arbitrarily.  Ephemeral subscriptions only exists for the duration of the active channel.  Once the channel is dropped, the subscription grain and its observable are removed and resources are disposed.  This enables a self-system to organize without prior knowledge of edge system components.  Of course, access control plays a major component on what is allowed, which is not a topic for this post.

Orleans is a radically different type of technology, which is a key enabler for the Pegasus-I mission to near space. It makes possible some of the key concepts around flowing information at low latency and organizing graphs through its concept of virtual in-memory actors, i.e., grains.   Observables in Orleans also give Piraeus a simple way to mate specific receivers to their respective subscription grains without need to leverage any store and forward technologies and makes for a remarkably elegant end-2-end communications story that is simple to use. We were able to get telemetry from Pegasus-I inflight to the Web site for a user to view in about 20 milliseconds on average, going from a radio transmission from Pegasus to our field gateways and over MiFi on our phones to Piraeus.  Of course, we also proved it also works at 85,000 feet, 2.2% atmosphere, and -60 degrees Fahrenheit.

Notable Technical Details

  • Orleans is a high throughput, low latency, and linearly scalable technology that enables Piraeus to easily manage a system graph or sub-graph to enable communications between system components.
  • Subscription grains are either durable or ephemeral in the context of Piraeus.
  • Subscription grains can be associated with either active or passive receivers.
  • Piraeus enables systems to be either organized or self-organizing.
  • Information flow is in-memory by default, but can be persisted if required.
  • Piraeus uses no databases or storage accounts to perform its operations.
  • Orleans grain-state is persisted in a customized Orleans Storage Provider that leverages Redis.
  • Piraeus has a multi-channel and multi-protocol gateway that does not couple channel and protocol between transmitter and receiver.
  • There is no intrinsic limitation to message size within Piraeus

There is a tremendous amount more about the technical detail, which is far to overwhelming for this overview post.  I will try to post more fined-grained detail at a later date for those interested.

Best Wishes,

-Matt Long