Microsoft 365 Groups overview and architecture deep dive

ARUNKUMARAN VARADHARAJAN: Hey, everybody, my name is Arunkumaran Varadharajan, I’m a Principal Program Manager with the Microsoft 365 Groups Team and thanks for tuning into my video, today In today’s session we’re going to be talking about Microsoft 365 Groups, start with a brief overview and then deep dive into the architecture of the Groups platform, and in this session you will get an understanding of the inner workings of the Groups platform, so let’s get started So this is the agenda for today’s video Like we said, we’ll start with Groups and then go to detailed overview And the way we’ll be doing it is we’ll go through one scenario at a time So we have four key flows identified, which is group creation, the life cycle of a group, member management, and administration and policies So let’s start with an overview of the Microsoft 365 Groups platform As you know, Groups platform covers a number of collaboration experiences across Microsoft 365 be it in Teams, SharePoint, Yammer, or Outlook But in addition to just that, the Groups platform is also available in most of our productivity applications like Word, Excel, that enable easy sharing And they’re also integrated with a number of third party solutions in our partner ecosystem, and you see some logos, there The Groups platform is centrally stored and managed from the Azure AD Directory service Now, let’s take a look at the Microsoft 365 Groups architecture So this slide is very similar to the one that we saw previously, but this is designed to build out the key architecture elements Since the slide is a bit crowded, let me start explaining all the different components First on the top is where we have all our user surfaces So the first one is the Admin surface This includes command legs, admin portals Right next to that are the cool hub apps like Yammer and Teams, which are closely tied and powered by the Groups platform Right after is the productivity apps that we talked about earlier So these are apps using which you can share content easily to a group of people And on the top right we have the third party applications that are integrated with our platform So you can see that all these apps are tied to our platform using the Graph API layer For simplification, I just called it Graph API, but you can think of this as a unified API surface that allows all the applications to talk to the Groups platform Now, let’s take a step deeper into the platform and understand what is beneath the API layer Beneath the API layer, we have two key components The first is the storage where we store information about the group, like the name, the group description, the membership, and so on, and for this we leverage the Azure AD Directory Service A second key component of the Groups platform is the policy engine, and when we say policy engine, it means various policies like expiry, where you have the option to say, “Hey, I want to remove groups that are inactive for more than 180 days.” So the definitions of those policies is stored in a couple of places It’s stored in the security, or the compliance center, as well as in the Azure Active Directory And then the policy engine is something that basically runs periodically, reviews the policies, looks at all the groups in your platform, and then makes sure that all these groups remain complaint So now, let’s take a look at one scenario at a time on how these different architecture elements come together So first off, let’s talk about group creation So as most of you will be familiar, you know Groups platform powers the three key communication apps, which is Outlook, Teams, and Yammer One of the things we do know that is when you create a group in Outlook it is automatically not visible in Teams and Yammer Similarly, when a group is created in Teams we don’t make it automatically visible in Outlook and Yammer And there’s a couple of reasons we do this, this is because most often users prefer to be in a single communication platform So we start with one, and then optionally allow people to expand into the other platforms So let’s look at, maybe, an example of how you can go about enabling that First, let’s assume that you’ve created a group in Outlook Your team has been using the email surface, and now you want to adopt Teams and you want to use chat, as well, for your communication This is sort of a common scenario where some of the project teams use email for their external communication for maybe other teams or customers to connect with them, where they use Teams chat for, more often, an internal communication tool The enablement process is very simple You go to the Teams, you say you want to crate a new team, which is the first screen on the left, right after that you have the option of picking, “Hey, I can create from something that already exists.”

So the two options are you can clone from an existing team or start with a group So here you would pick group as the sixth option After that you see a screen that shows you all the available groups So here, you can go ahead and pick the group that you want and that’s pretty much it So now, once you click okay, a new team is in for you and the team is ready to go, and it’s connected to the original Outlook group, which means all the membership and everything’s maintained and synced A similar flow that we see is when people are now starting teams So, you start a team, maybe you have this internal collaboration project going on, and then you realize that you need to have some external collaboration You want to make your team accessible to other groups within your own company, like departments, or even your customers So this is where opening up the team for Outlook is very useful, so this is sort of the other flow where you start with a team and then you say, “Hey, make my team available in Outlook.” So the first option that you have, the first action that you want to do is run this powershell command link, which is Set-UnifiedGroup, where you pick the the identity, which is the group name, and then you set it as don’t hide it from address lists What this does is now this group is visible across your entire company, so people can start emailing it very easily The second one is actually making this group visible in the Outlook left nav for all your group members So this is very similar, you see, Set-UnifiedGroup-Identity as group name, and then you say, “Don’t hide it form all the Exchange clients.” Now this group is visible in all the Outlook lines So now, let’s go back and talk about the visibility problem, right? So we said, “Hey, when you create a new team “we want the team to be provisioned, “we want the team to be visible, “we don’t want it visible in Yammer, “and we don’t want it visible in Outlook.” So, how does that flow happen from an architectural standpoint? Before we go into the flow, there’s a couple of key architectural elements that I want to mention The first one is the datastore So that we talked about, how Azure AD is the primary datastore The second one is the notification framework, which is how will all the apps connect to Azure AD and how do they get notified when something changes, like a new group getting created? And the third one is a key element called resource provisioning options So what, basically defines is, for any new group that is being created, you have the option to define what are all the resources you want provisioned for this group So, with these in mind, let’s take a look at the create group experience So let’s assume we have a user who’s gone ahead and said they want to create a team So you see that the Create Team notification is now being sent to the Azure AD Directory Service Next, the Azure AD Service publishes a reliable notification to all the apps And the reason we call it a reliable notification is because, as you know, the Microsoft service is available in numerous data centers across hundreds and thousands of customers with millions of users in it The word reliable just means that our system’s designed to be fault tolerant, so these notifications never get lost Now, you can see that the notification has been published and the different apps received that notification Now, let’s see what happens Now, based on resource provisioning options, we know that we need to provision a stream, we need to provision a SharePoint site, we need to provision a team, and we need to provision a planner Similarly, you see that Yammer and Contact Manager are not applied, because we know that those are not needed And here you see that the Outlook is sort of in a dashed or dotted line So what this really means is a mailbox is provisioned and this mailbox provides for services like discovery, compliance, and calendar for the team But it is disabled by default, or hidden by default, so that’s why it’s dotted So if you were to go back and run those command legs to expose these in Outlook, then you can start to make this visible, as well So this is one of the most basic and fundamental elements of the Groups platform, which is the Azure ADX Datastore and the notifications as a mechanism for keeping all the apps in sync Next, let’s move on and talk about what are all the different polices that are applied when a group is being created So like we saw earlier in the architecture diagram, we have the datastore and the policy engine There are three policies that are taken into account every time a group is created The first one is the creation policy The creation policy, basically, defines who in my company’s allowed to create a group By default, all users are allowed to create groups, but admins have the option of limiting that So every time a new group is created, our platform is going to check whether that user actually has the permission to create the group and these checks happen both on the UI layer as well as our backend

The second key policy that applies when a new group is created is the naming policy So, quickly, for sure, the naming policy allows admins to define prefix and suffix so that every group that’s created from the fiance department has a prefix, say, finance So that helps distinguish The second one aspect of the naming policy is the blocked words For example, you want to make sure that some inappropriate words are never used, even accidentally, by all your employees So you could use a naming policy to define those So, the way the naming policy works is every time a group is created, at the time of creation we evaluate the user’s submitted group name against the policy and we allow the group creation only if it is allowed Similar to naming policy, a couple of other checks we do is we always make sure that the email address of a group is always unique Similarly, the site-URL is always unique This is sort of a basic thing And these are checks that are always done The third sort of policy we apply at the time of group creation is the new Microsoft IP Labels So, as you might be familiar, this was in preview in last year and a few months back we essentially GAed this feature Microsoft IP labels just provides a very easy way for admins to sort of define templates so that their end users don’t have to think about how to classify content or how to configure the group As you can imagine, there’s two key configurations in a group, which is public/private, which is the privacy setting, or internal/external, which is the guest setting So in order to make it easy for end users, admins can define policies to say, “Hey, this is confidential,” or, “This is external facing.” And just by picking those labels the privacy setting and the guest setting are automatically applied Most of our UI surfaces actually enforce this so this never becomes an error, but this is something we enforce at the platform layer So even when you create a group using our APIs, and if you were to provide a label that were to mismatch with the defined settings then the group creation would fail, and that’s the architecture component of policies at create time Next, let’s look at data residency So as you’re aware, Office is available in multiple geos now, especially Exchange and SharePoint Online In order to capture which geo the group needs to be created in or be homed in, we have a property called preferred data location, or PDL So on the screen you can that I have an example of how the PDL can be specified at the time of group creation So in this example, we’re creating a new group called MultiGeoEUR and we explicitly pass in the the mailbox region as Europe And what this does is it allows you, as an admin, to be able to create a group at a desired data residency In order to make things simple, what we do by default is we inherit the PDL of the creator So, for example, if a user’s based in Europe, any group they create is automatically homed in Europe, for example But you do have the option of changing the preferred data location of the user or of the group using these commands Once a group has been created in one data residency, there’s often a requirement for you to move it It could be that you maybe have some of your users are changing locations or you want to move some specific data because now you’ve started adopting the multiple data residency options, so the way to trigger these data residency moves, is to first is for Exchange, all you have to do is update the data residency of the user And because your move happens automatically, so it’s out of provision and managed by our own service On the SharePoint side, we don’t yet do it automatically So in order to trigger the move you would have to run a command So the command is on the screen, so you just say you want to start an SPO site move and then you call those up And once the command is on, the destination is defined and the data starts to get moved Not all the M365 apps support moves yet, but that is something we are working towards Now, let’s take a look at the groups life cycle As you can imagine, all groups start with a new state when the user creates them, and as the group starts to get activity, we put it in what is called the active state Depending on whether your company has enabled expiring policy, we keep tracking the group to see if the group is actually active or not If the group is not active, then we automatically trigger the deletion process and it gets moved to a soft deleted state In cases where the expiring policy is not involved, a user, as you know, can manually go in and choose to delete a group or delete a team because they are no longer needed

Once a group is soft deleted it is available for 30 days before it gets hard deleted So any time a user clicks restore within those 30 days, the group comes back with all the attached resources and it’s almost like it never got deleted But if the delete is not restored within 30 days then the group goes into a hard delete state which means the entry of the group is completely removed from the directory and also all the associated data is removed The only exception where we don’t remove the data is when retention is applied Now that we’ve gone through the groups life cycle on a high level, let’s dive in and look at exactly how this works out Let’s start by looking at group activity tracking and auto-renewal So let’s assume that there’s a user who belongs to a group in Tenant T1 And this user is using the group across Outlook, SharePoint, and Teams So every time any of the users in the group are using this group, either in Outlook, or in SharePoint, or in Teams, that activity data is continuously tracked And what we do is we look at next expiry policies Let us assume that the tenant has defined a policy of 90 days Now, we look at groups that are due for expiring in the next 90 days and then we look to see whether this group has been active or not So the activity tracking system tells us whether the group has been active or not in the last 90 days And if the group was actually active, then we go ahead and auto-renew the group, which means the renewal date is now pushed by an other 90 days But if the group is actually not active, what we do is we mark the group for deletion We don’t delete the group at that point, the group is marked for deletion and that now gets picked up by the expiry emailing service What the expiry emailing service does is it looks at the list of groups that are up for expiry and then it sends three notifications to the owners of the groups, one 30 days prior, one seven days prior, and one one day prior And these are just options available to the end user to be able to say do they want to let the group expire or do they want to manually renew it? And when the chooser chooses to manually renew it they are taken to the Azure AD protal, where they have the option to say, click renew, and now it’s manually renewed for another 90 day Again, the 90 days is a configurable option that you can define for your own tenant It can be any date more than 30 days Now, let’s take a little bit deeper look into the activity tracker service So the activity tracking service today is enabled at the tenant level so you have the option to say, “Hey, I want to monitor anyone from my whole tenant.” Secondly, the activity tracking service looks for activities only in three apps, which is Teams, SharePoint, and Outlook For example, if you have a Yammer community that is being activity used an you’re uploading files, because those are SharePoint users, the Yammer community’s going to get auto-renewed But if that Yammer community is not seeing a lot of file usage, then that, today, is not part of the activity tracking service And this is, again, just a point in time, we do plan to enhance this in the coming year The second thing that we want to call out is that not activities are considered for auto-renewal, and we try to make sure that only intentional activities get counted Often we run into a situation where you get a message saying, “Hey, the site is inactive,” you go to the site and inadvertently auto-renew it So we don’t want that to happen So for SharePoint, we look at explicit activities to say somebody write the file or somebody edited the file, or somebody actually sent an email for Outlooks in our use, and only in those scenarios do we auto-renew In Teams, it’s not very easy for us to distinguish between intentional activities So right now, we have a bias for retaining the data versus deleting it, so any user activity, even reading messages on the Teams channel prevents the channel from getting deleted Next, let’s look at deletion flow So, in the next few slides, we’ll just walk through, step by step, what happens in the process of deletion Let’s start with a group that is active in Azure AD and the group has four resources attached to it, Exchange, Teams, SharePoint, and Planner First, let’s assume that the group is active in Planner but at some point in time, the user decides that they no longer want this group Maybe the project is over and they want to get rid of the group, so the user goes and deletes the group in Planner So the first thing that happens is there’s a notification that goes in Planner to Azure AD informing Azure AD that the group now needs to be deleted This request is accepted and the group is deleted in Azure AD and after that confirmation, Planner deletes its own copy of the group, but the data is still only soft-deleted for 30 days

The liable notification gets published to all the other apps and all the other apps also follow suit and the data is marked as soft deleted So here, let’s take a look at a few differences For example, first, in Teams the group is soft-deleted for 30 days, no other change On the Outlook side, the group gets deleted for 30 days, but the email address actually gets released This is because if someone else needs this email address they can immediately use it On the SharePoint side, this is slightly different We actually have a longer time window for soft-deletion, which is actually 93 days If you look at it, it’s a little more than three months, and the site-URL is also not deleted The site-URL is maintained We do recognize that there’s a lack of coherence here across the apps and we are working towards fixing that So now let’s assume that the user realizes 15 days down the line that this group the deleted, they need it back Users have two options, they can request you, as the admin, to help restore the group, or they can actually restore the group on their own So newly we allowed a capability called Groups Hub in the Outlook Web experience We would encourage you to educate your end users about it because this is completely self-service So they can go to the Outlook Web experience, open the People and Groups tab, and right there they can see a list of deleted groups where they can restore it in one click Now, let’s assume that the user goes and restores the group in the Outlook Web experience Next what happens is the group is marked as restored in Azure AD, so that is the first step And after that the liable notification goes through to all the endpoints and now all the content is perfectly restored Here, again, the one small caveat is that on the Outlook side, in case somebody else took the email address of the group, then the restore’s not going to work But in most cases, the email address is going to be available, so the group is going to get restored flawlessly Now, let’s look at a case where the group actually has to get hard-deleted So let’s start with the best case, the group has been deleted by the owner and the content is maintained for 30 days in the soft-deletion window On the 31st day our system realizes that this group has been deleted for more than 30 days So we send out a hard-delete notification to all the apps And the group is also deleted from Azure AD Now you can see that the group has been deleted in all the apps, but the content is still available in SharePoint for 93 days This is similar to what we talked about in the previous slide And now let’s assume that the 94th day comes So at this point, the group has been hard-deleted in all the apps and all the content, including the site content is now permanently deleted The next scenario we want to talk about is retention So in case of retention, the admin can go and define the retention policy in the compliance center A common pattern is to have a 7-year retention policy for financial industries So let’s assume that in this example we have a deleted group The group is still in a soft-deleted state and the policy’s set for 7 years Now, on the 31st day, as expected, the group is hard-deleted The hard-delete notification is now sent to all the apps Since Planner does not support retention policy, yet, you see that the group is hard-deleted and all the content in Planner’s removed But in Exchange, Teams, and SharePoint, the group, actually is not deleted Even though the delete notification comes in, because of the retention policy, we don’t honor that, and instead, keep the data And this data is now accessible both via e-discovery as well as the management surface endpoints So, in the first two sections we saw an overview of how groups are created and what is the lifecycle of a group, now, in the third section, let’s talk about member management When we talk about member management, the first experience a user has is discovering and joining groups So, here, let’s take a look at what is the experience when the user tries to join a group Let’s assume you’re in Teams and you click on the Join or Create Team option If you’re a user who does not have creation policy, then you can see that you only get the Join Groups option If you do have creation policy, then you do have the option of both creating a new team as well as joining an existing one Secondly, when you try to join a group, what we do, is we basically run a discovery service in our backend that looks at all the groups, it looks at membership, it looks at usage, and it also looks at what are all the groups that you are not a member of, and then it does a recommendation So this is one other element of the Groups platform

which powers the search and discovery experience So, let’s walk through the different member experiences We’ll start with the most simple and basic one, which is there’s a public group and I, as a member of the company, now choose to go join that group, or it could be a private group where I, as an owner, want to explicitly add someone who needs to be a part of this group The flow’s very simple, you can start in any of the apps You go and make a request to say, “Add group,” “Add member to a group,” so the first thing that happens is we send a notification to Azure AD to say member M1 needs to be a part of this group and that is honored Right after the member addition happens, the next thing that happens is an email flow is triggered So depending on the app, you get an app-specific welcome email and this helps the admin quickly ramp up and educate the user This welcome email, again, is optional So one thing we do want to call out is admins have the option of disabling the welcome email for any group so any new members who get added will not get the welcome email This can be useful when you want to bulk-add members, get the group to a ready state before you open it up for all your users So now, let’s look at the next scenario where we add a user to a private group The use-case here is fairly straightforward You might be a member of a group, and then you realize that maybe some of your colleagues would maybe benefit from being a part of this group, so you go ahead and add them to the group Because it is a private group, they actually need owner approval to join and they don’t get added immediately So here, you can see that the flow’s very similar You start in an app, you make the add request, but right after that, it doesn’t go to the Azure AD Service The member’s actually recorded as a pending member in the group in the Groups platform and we trigger an email flow where the request is submitted to the owner The owner now has the option to either accept or decline it And this has been built using the actionable email support, meaning the owner can look at the email and basically click a button directly on the email to approve or decline and just in one click it’ll get the job done And after that, let’s say the approval does go through, the flow’s similar to the previous slide where the user gets added to Azure AD and they do get sent a welcome email, which is optional, and can be disabled by admins Now that we’ve seen how we can add members to public and private groups, the next sort of membership that we do want to talk about is guest members Guest members are basically users who are not a part of your own tenant and these are people that you need to work with for solving your business scenarios These could be partners, these could be suppliers, these could be vendors, these could be your customers, and the guest platform on Groups is actually quite rich, and we see that remote users increasing by an order of magnitude every year So from an architectural standpoint, what is the first step we consider when a guest is about to join a tenant is the tenant level policies So there’s policies that define eligible domain, meaning we set the domains to a company’s firm, which this guest is invited or can be invited Second is the group level policies, so as an admin you can define which groups can have external members and which groups cannot have external members And this can be useful, for example, maybe on marketing you’re okay with the marketing team working with different agencies, but maybe you don’t want your research department to be working with external folks So you can do these group level configure options The third one is the inviter role, meaning who in your company’s even allowed to invite guests? This is just one more way of locking down guest access So, from a flow standpoint, every time a user tries to add a guest, all these checks are enforced in all our apps on the backend And this is the first check that happens Now, let’s assume that the guest is already present in the tenant If the guest is already available in the tenant, then the add to Azure AD request is basically saying, “Take this guest and add them to the group,” which is automatically approved And after that, an email is sent, so this flow’s fairly simple, it’s like adding a member who already exists The next flow, which is slightly more complicated, is let’s assume that this guest is new to our tenant, right? So let’s say I want to invite somebody from My company has set up as an eligible domain, and now I add, say, John, from to my group Azure AD is going to sort of take in this request but the user is marked in sort of a pending state where the user now gets an invitation

to participate in my company So in his Conducive email, the user’s going to get a request to say that Arun has invited you to participate, and the user now has to redeem that invitation So they have to click okay and be willing to share their data and say that they are eager to participate in this group And once they have given their approval to participate, they are added to the group and then they’ll see the welcome email A common question that we get from many of our customers is, “Hey, many of my users add external guests “to their groups because they have a use case “at that point in time, but maybe after some time, “those guests no longer need to be a part of my tenant.” So to solve that scenario, we’ve introduced a feature called Membership Reviews, or Access Reviews The way it works is on a periodic basis, you, as an admin, can trigger this flow where the owners of individual groups now have to say whether each of their guests is needed or not And you can even enforce strict policies to say that if somebody does not explicitly give their approval, automatically remove the guest The way the access service works is very similar So we go to the same flow where the user is added but now, after the user’s added the access service runs periodically, picks all the groups, looks for groups with external members, and then now sends a notification to the owner So, we’ve covered the three main sections, which is group creation, group lifecycle, and membership management, so now, let’s talk about the fourth section, which is administrative policies So we did talk about the fact that we have different sets of policies, like say, creation policy, expiry policy, naming policy, in this slide we cover where are these policies created and how can you, as an admin, access these policies and how do they interact with the Groups platform So, we’ve mainly divided this slide into two components First is what we call as app-specific policies and second is what we call as suite-wide policies When we talk about app-specific policies, they might be policies that might apply only to Outlook groups, for example, things like who is allowed to email this group? Are certain people allowed to email this group or not? Or there might be SharePoint specific policies where you say, “Hey, this group can be accessed only from certain devices and not from other devices,” or Team-specific policies, where you say, “Hey, maybe this team is sort of a broadcast channel, where only messages can be posted and not replied.” All these policies are actually defined by the individual apps, so for email policies you’ll do it in Exchange Admin Center, for chat polices, in Teams, and so on These are just examples, as you know there’s over 20 apps on the Groups platform, like Yammer, Planner, we don’t have those examples listed here The second set of policies that you apply mainly from the Groups platform, are what we call as suite-wide policies So these are options that are available across all the apps that are powered by Groups, and these policies are mainly defined in three portals The primary one is the Microsoft 365 Admin Center The second one is the Azure AD Portal, and the third one is the Security and the compliance Admin Centers So here we can see that some of the options, like being able to look at all the groups, being able to restore deleted groups, those would be in the Microsoft 365 Admin Center And going forward, that is going to be one of the primary endpoints for you to manage all your groups Secondly, we do have some directory-related policies and audit logs available in the Azure AD Portal And for all the compliance features, like say, the IP label or the retention policies we talked about, we can define it in the Compliance and security Admin Center So now, let’s go back and take a look at the architecture diagram we looked at initially and do a quick recap So as we saw, all of the group operations, like say, creation or the membership management of polices, all of them flow to the Graph API All the surfaces are connected to our backend through the API platform and below the platform we have these various policy engines that are working on all the groups stored in our Azure AD Service One of the things that we do want to point out is that while groups are available for work, we’ve also made a lot of progress in making the same groups available for even personal scenarios, like, for example, using it with friends and family We have a great session coming up by my colleagues Sandra and Raul, the details are listed, so we do encourage you to watch that video to learn more So let’s take a quick look at all the videos that we have for you So here is a list of all the videos that we think you will find useful as admins And the Groups-specific videos are starred Here, you can see the video that we just talked about

which is Managing Groups for Work and Life, is also available, and we also have a very interesting session on how Microsoft internally manages Groups to make our employees productive Thank you, all, for joining me, today I hope you found this video useful and we want to leave you with our key resources, I’ll give links to the tech community, the user voice, and the roadmap, here I’m very grateful for your feedback all the time, so please do keep that feedback coming, thank you