#230: Designing for the Duo, with Rafa Conde

Lightly edited from the audio for readability: filler words and false starts removed.

00:00 David: All right. So welcome back to Developing Perspective. And one of the things that I, when I sort of restarted the show, I think I've mentioned this a few times, is I wanted to have a slightly more flexible format, something where I can do things that I don't fit into just a short kind of 15-ish minute monologue format.

00:17 I want to do other things. And the first instance of this I'm super excited for because I'm going to do an interview and I'm talking to my good friend Rafa, who I've been working with on our updates for the iPhone Duo. We've been working on this sort of since the beginning of the summer when it was resizability for the iPhone mirroring. But more since September, specifically actually knowing what the iPhone Duo is.

00:41 And I think the design side of that, which is absolutely Rafa's sort of specialty, is really interesting to unpack and talk about. There are some few things sort of on the technical side that are unique and interesting about that. But the design side is where I think the Duo really comes into its own. And so I'm delighted to be able to do that. And I appreciate you coming on and talking to me about this.

01:01 Rafa: Hello. It's a pleasure. Honor, even.

01:04 David: Yeah. And so I think the first place to start, and this is, I think, something that you and I have gone back and forth on so many times in the Duo, and I think we just have to get it out of the way, is talk about centering and content overflowing on the Duo, which I think is, from a design perspective, the most infuriating and impossible thing about it.

01:25 Because no content is actually centered, or if you center it, then it looks weird because then there's content on the sidebar. Or should your content overflow into the sidebar? And sometimes that looks good, and sometimes it doesn't. And so I think I have enough of a design sense for it to annoy me, but I think it's been driving you actually crazy.

01:45 Rafa: Yeah, no, it's nuts. Centering stuff, usually it's not something hard or something that requires nuance. It's centered or not, and the container, usually for phones especially, or just, I don't know, software, usually, the container where you're centering stuff to is usually very obvious and clear. But here it's not.

02:10 And sometimes I even wish that Apple did the thing for if your app doesn't support the Duo, you get the black bar on the side. And it looks not updated and it doesn't look nice, but it makes it simpler and things look centered. I saw that Rivian's iOS app that, you know, I don't know if...

02:29 David: They use their own black bar.

02:30 Rafa: Yeah, they decided to have their own black bar and they still put controls and stuff there, so it's not just an empty void in there, but it is a choice, a good choice in a lot of ways. But because Apple decided to do it this way, it's not an option if you want your app to feel like it belongs there. So it's really, really hard. There's a lot of nuance to it.

02:58 We've spoken to this before. I've mentioned that I live in Amsterdam at a very, very old building from the 1700s, and so nothing is centered, nothing is aligned, the wall is at an angle, the ceiling, and it's a different angle than the floor. So if you're trying to put up a frame or something on the wall, you kind of have to decide, am I gonna align this with the ceiling or with the floor or with gravity itself if I use a level? All of these are different decisions, nothing matches.

03:29 And I feel that it's very similar here in the phone. What do you want to align it with? And it's kind of like use your own intuition, what feels best, but more often than not, there's no right answer. It's like, what do you decide to align it with?

03:47 David: So I think even there, the crazy thing is what you align it with, so in your example, is variable based on the pose and context of the phone as well, in a way that it can be really weird, where what makes sense on the outer screen in portrait may not make sense on the inner screen in landscape, just visually or sort of logically, or how should your content scroller look in these different things as you move it around? And then you get into the question of, should that change? Should you have a consistent thing?

04:22 So, say, for example, you're always centering inside of the sort of content area, avoiding the safe area. And if you do that, then you flip it into inner screen portrait where the safe area goes all the way to the side. Is that weird that on the outside, in one version it looks this way, and another version it looks that way? And in your example, it's like here I'm leveling my photo to the floor, here I'm centering it to the ceiling, here I'm changing it to...based on this other thing.

04:49 And I've had to just let go of the fact that there should be a correct way. There should be an actual...it's like, no, no, there is no correct. And what is the least worst is probably variable based on which of the poses you're thinking about or looking at.

05:07 Rafa: Yeah, it does feel like every single pose, you have to look at it and design it as if it was its own device. It's hard to come up with solutions that scale across different poses, across different orientations.

05:20 Even more so, iPhones always had portrait and landscape mode, but depending on the type of app you're building, it was pretty easy to ignore landscape. It didn't make much sense, and also the way you use your phone, it's almost certainly in a portrait orientation. And then iPad became a bit more muddy, like, OK, certain people prefer to use the iPad in different orientation, but they were both large enough that felt similar. I think a certain layout would scale well between the two.

05:52 And now, the phone, over time, just got taller and taller and taller. So we've been pushing in this direction of being like, things are just a vertical stack of content. And now the Duo, we flip it in both opposite directions at the same time. We went to, like, actually, this is the smallest screen ever, and it's this squarish thing, and now your controls go on the side, and also it's the biggest screen ever, and the controls go on the side in one orientation, but not on the other.

06:24 The difficulty level just grew exponential. And I think in other platforms in the past, developers had two escape hatches. Some platforms you could ignore completely. Say Vision Pro comes out, it might not make sense for you to port your app to it. You could choose to, like, this is not something I want to deal with right now, and it would keep your existing apps isolated. The fact that the Vision Pro came out won't affect your iOS app. Kind of similar to the iPads as well, you can not have an iPad app. Sometimes it doesn't make sense, it's fine.

07:08 That's one escape hatch, you can ignore it and it won't affect your main product. But second escape hatch is, all right, say you do want to support it, say you do want to build a Vision Pro app, you can still just make it look like an iPad app. You don't have to go 3D, you don't have to go full environment stuff. You can just make it look like an iPad app. So that was easy.

07:31 But with this phone, it's still an iPhone. So it will affect your main app, assuming your main app is an iPhone app in this case, but it will affect it. It will have a lot of users in the beginning, enthusiasts, the nerdy users who will jump on the new device and fancy stuff. And they will be the ones complaining that your app hasn't been updated.

07:47 And so you don't have that escape hatch of, ignore it. You cannot ignore the Duo, your app will run in it. And you don't have the second escape hatch of, just stretch it. You don't even have that. If you do nothing or do the easy path, it just still looks broken and odd. So it's the hardest job from a design point of view of the new platform or new device, and it's a hard job that you cannot ignore or avoid.

08:18 And yeah, the choices that Apple made with their own apps, it's not that I disagree with them, but they're not obvious sometimes. And so you and I, we had the chance to go to visit one of the labs, and so we spent some time with it before it came out, with the device. And we spent some time just looking at other Apple apps, because the simulator came out and you only had Settings and Safari, I think. So it was even hard for us to study how Apple adopted this new device with their own apps.

08:52 So that was one of the first things we explored. All right, how does Find My work here? How does Maps work here? We were working on Check the Weather, so those were relevant to that. But even when you see them, some of the decisions they made, they're not obvious. Like, this doesn't look right. Why would you do it this way? And if you keep pulling that thread, eventually you realize why. There's a reason there.

09:17 I'm not saying sometimes it doesn't feel like the best reason or the best choice, but there's a reason, it's not arbitrary. But it's still, you have to do a lot of thinking to arrive at that decision. You look at the app and it doesn't come out like, yeah, this makes sense. That's how it should be.

09:33 And even Apple's own guidelines are the most wishy-washy, I think, for any other device. I remember we were like, OK, so controls go on the right. Got it. That's the rule. And they're like, well, but if you only have one control, one toolbar button, say, or two, maybe you can also put it on the top. So you don't have to do the vertical thing. There's APIs for that. You can say, no, I want my controls at the top. And you're like, OK, but wait, so the rule is they go on the side, but if there's only one or two buttons, they go on the top? Well, no, they can go on the top.

10:09 David: Go on the top.

10:10 Rafa: So you still have to make the decision.

10:11 David: Yeah.

10:12 Rafa: It's all like, what should I do here? So...

10:16 David: And a lot of those things too, there's these weird conditions where a UI that I think often looks good on, say, the outer screen of this device, is you have a hero image or some kind of content that can go all full width, and then you have left-aligned sort of title content kind of a thing. Say you imagine a recipe card, you'd have the photo of the food on the top, and then the title left-aligned, which is great until you open the phone and put it into side-by-side mode and you put that app into the left side, where suddenly now it's overflowing backwards into an area where now the buttons are on the left side and not the right side.

10:56 And so this UI then looks really potentially weird or unbalanced, that you don't have the balance of the text on the left and the buttons on the right. Now you have both of them on the left, so it's unweighted in an ungainly way.

11:07 And I think in some ways it's been a relief in the sense of... I feel like there's a certain amount on the traditional, I don't know, the standard iPhone layouts where you can look at it and definitely say, this is not a good layout. There's something about this that is not symmetric, it isn't balanced, it has a sense of proportion to it. I know enough design to sort of be dangerous in a lot of this, where I don't really know why, but I know that I can look at something and be like, that's not right. There's something not quite right there.

11:50 Whereas on this, I feel like it's so variable, and there's so many choices that there is no right answer. There is no sense of, you're sort of working towards this platonic ideal of what a layout would look like, which on the other one, I feel like you really could see. This is an experience you and I have had a bunch, where you do a redesign on a screen that I made, and I immediately, when I see it, know that it's better. I immediately...because you've applied some sense of symmetry, balance, design, aesthetic to it that is just like, yes, this is definitely better.

12:19 And the number of times on the Duo, I feel like the experience is, I look at it and it's like, it's probably better, but there isn't this sense of just straightforward clearness, or you immediately then have the thought of, this is better in this situation, but what happens when you do something else to it? Does it...it's just all these strange choices that...and I think it's probably gonna be a question of, I don't know, months, years before we really can settle into a place.

12:51 And I kind of worry a little bit about what the next version of iOS is gonna look like. Next June, is there going to be a new set of guidelines, a new set of learnings? Because hopefully Apple is learning from this as they're seeing it go out into the world with developers, with designers, having a sense of, OK, what actually worked? Because they only had their own set of apps to be using against.

13:15 And I think we had a lot of this in Liquid Glass, that from the first version of Liquid Glass to the one we got last year, there were some improvements, some changes, some things that were based on the contact with the world. And hopefully we're gonna get those. But I also don't want to commit too far down any path until we get to that point, where I feel like we may end up needing to work back or change how we've structured things based on what's coming down the road.

13:43 Rafa: And there's also this tension that it's not clear to me, at least, that this is the future of phones. So there's also that sense of, it's not like this is gonna be the future.

13:54 So yes, it's weird. Say when the...it was not the Dynamic Island, but the notch, the first notch on iPhone X. It's like, OK, this is kind of weird, right? It doesn't look as pleasing as just nothing in the screen cutting into the screen. But we were like, but this is probably the future, it feels nice. So let's explore that, let's push that front of how can we make this work better, and then eventually the Dynamic Island, so it's not just a black void, actually content can flow from it or whatever.

14:24 But in this case, it's not like, oh yes, now this is the aspect ratio going forward, so of course controls need to go on a little sidebar. So let's keep improving that to the detriment perhaps of the main normal phones. So that is not clear. So that's also, it's a weird one. Is it gonna remain to be the weird one? The Duo is the weird iPhone where iOS looks kind of weird sometimes, or is this the future and we're like, well, let's just embrace it?

14:55 There's certain decisions that it's always hard to look back and try to apply meaning to certain decisions that Apple did, like, oh, remember when they did the large titles navigation title, left-aligned and pretty large. I think that's way too far back. That's what, eight years, 10 years? I don't even know. But we're not like, oh, left-align titles, maybe because of the Duo. Probably not. But there's certain decisions that recently they've left-aligned alert boxes and stuff, or...

15:28 David: Maybe it's just...

15:28 Rafa: On a Mac. Maybe I'm getting confused. There's some decisions that maybe this was because of the Duo.

15:34 David: I think the tab bar is definitely that, right? The whole thing where in Liquid Glass the tab bar became a floating element that was separate from your content rather than being this hard rectangle at the bottom, was clearly done so that on the Duo they can flip it sideways and put it on the side. It's this thing that they're changing for that reason. And whether that looks better or is better, it has all kinds of problems on regular iPhones that we now just have to live with. But it was done, I think, in service of the Duo.

16:09 Rafa: I do wonder if we were living in this alternate universe where we did not go with floaty controls and everything has a clear separator line and maybe perhaps a different background, even crazy to think, if that wouldn't make this feel less awkward. Because UIs with sidebars exist as old as UIs go. Look at your Mac, a Finder window, there's a sidebar, and then there's a content. But people aren't freaking out, like, oh, how do I center things? No, because there's a clear separator, this is a sidebar, there's content here, and there's this other content on this side, and so the containers are clear, and your eyes can align things and get that.

16:52 But here we're freaking out about a sidebar because that sidebar is actually not a separate area, it flows, and sometimes there's things there. If you don't have tab bars, that bottom side on the...it's probably empty. And if it's empty and you have a bottom control on your UI, then if that's not aligned, your eyes try to look for the boundaries of certain containers to make sense of things and align things. And it's because there's nothing there, and if you only align it with a content view, then your eyes will...it's gonna look off center, like, this is not centered, it looks wrong.

17:32 So in this alternate universe where maybe they did go with a separator and, like, this is a sidebar. Because I never call it a sidebar, but then maybe that would actually make it feel a little bit, I don't know if better, but at least tidier.

17:51 David: I think our job would have definitely been a lot easier in that way.

17:53 Rafa: Yeah.

17:54 David: I was thinking one of the crazy examples I had in Check the Weather was the... so it has a sheet that you can pull up, from all the way in the bottom to all the way on the top. And we don't have a tab bar in it in the same way.

18:07 So sometimes you can have situations where I had things inside of the sheet that were jumping left and right to avoid the camera or not, which looked weird when it was in the bottom if it was shifted over by the width of the sidebar, because then there's just this empty space. But as you move it up, it encounters other things and has to bounce around. And that doesn't feel great either.

18:34 But I mean, even just in general, where do the buttons go? It's a strange question. I feel like I still don't love the way that that is a choice that we have to keep making over and over again of where do the toolbar buttons go?

18:50 And if you have a tab bar and a set of toolbar buttons, even just a very standard... say you have a tab bar with five items and two items in what would be your nav, your nav top left, your nav top right, that very standard setup. Even that can get really strange. Where I think I saw this in Marco's app, Unforgetful, where if the keyboard comes up in the wrong way, suddenly your nav bars collapse into an ellipse menu. It's like, that's weird. Why is it doing that?

19:22 In that case, they should probably be on the top, but if they're on the top, then they're smooshing it even more. So then maybe you don't want the buttons on the top, but where do you want the buttons? I feel like it's this game we had to keep playing of where should the button go? And it's a strange thing to have to navigate and decide that there isn't an obvious sort of choice... where should it go?

19:43 In a lot of our apps, we have a top toolbar, a bottom toolbar as well. And so should those all be collapsed together? Should they be kept separate somehow where you put some on the side, some on the top? It's...

19:57 Rafa: We're mixing controls that historically have never been... they never live in the same place, so they didn't have to compete. Like a tab bar and toolbar items, they were separate.

20:09 And with Liquid Glass, Apple muddied this a little bit, where you can have the search button, which historically has been not even a toolbar, but it was at the top, and now actually can live around a tab bar sometimes, and then they expand and stuff, like with Music, for example. OK, this is a bit messier, it hasn't been this way historically, so there's a new thing now, I guess.

20:34 But now with the Duo, the tab bar and the control, like the toolbars, they live in the same place, they share the same space, so they have to compete because it's a very limited space, depending on especially how many controls you might have there, how many buttons. They have to compete and... so now you have APIs to set priority levels on these controls, like which one is the most important? Well, depends, I guess, maybe on users.

21:02 But we found that, say, Reminders, I think the primary toolbar item is new item, right? The plus, I think it's prominent, like blue. So that's the main one. And the Duo is like, where does the main toolbar button go? It's at the bottom right, right? The bottom. So that's the prime real estate. That's the most important thing.

21:28 But then if you have a tab bar, tab bars actually take priority. No, no, no. Tab bars go there, and now I pushed your primary button to the top, but not if you have a back button, that's the topmost one. You know how this is: one minute in trying to explain the logic behind it, it already gets confusing.

21:48 And for us, I don't think we're ready to spoil this, but we've been working on a UI where it has a bottom sheet and a part of the app. And as we were still at the labs early on, we're like, all right, it's a bottom sheet. We do not have any toolbar buttons here, except for one back button, I believe, or maybe one or two. So they wouldn't overflow, they wouldn't go to the bottom. And we have a bottom sheet.

22:15 So it made sense for us at the time to make that bottom sheet expand all the way, like the full width. So, because there's no buttons there. Again, if there's nothing there, the sheet just looks awkward. Like, why aren't you taking up the full width? So we decided to go full width. And it's a short enough sheet that it's OK, it doesn't need that vertical space. It was OK, that felt better.

22:37 And then we tried it at the labs, and I was like, yep, this feels like the right decision. And then I finally got my own device, and I'm using it at my desk. And guess what I didn't test at the labs? Freaking Live Activities.

22:54 David: Sure. And then it pushes everything down.

22:58 Rafa: So what before felt like, yes, there's no controls here, the sheet should go full width. Now the Live Activities in the Dynamic Island are just pushing the controls down, like the back button and stuff, they get pushed down, and now they're competing with the sheet. No, actually, they were behind the sheet or something. It was broken. Like, OK, here we go.

23:17 Another thing you have to account for is the... yeah, don't forget, y'all. Account for a super tall Dynamic Island that can push stuff down. So it is like you have to just use your intuition, be OK. It's actually kind of freeing to know that there's no right and wrong decisions.

23:39 I think there's a couple of things that we've seen, and I think I'm confident saying that they work best on this device, at least. Things like overflowing background imagery stuff. So you use the example of a recipe app or something. Having a full picture photo of the thing that can be extended over the safe area, so that background feels nice.

24:11 Especially if it's background that there's no obvious center, say maps, like the work we did for Check the Weather. Maps is great because it's visually rich, it's nice, it's colorful. Even if you use the Maps... MapKit, Apple's own maps, they do some really clever things where labels and stuff, UI over the map, they automatically get blurred out if you go into the safe area. So it doesn't feel like it's being clipped or something, but it's really nice.

24:43 And there's no obvious center in the map. The map has no center. So say even if you have a background header that you decide to expand behind the thing, then you're like, all right, if this picture has a clear center, should I align it still with the content, maybe, or... more decisions you have to make. But anyway, something like maps works really well, some kind of pattern background, that works really well.

25:11 And then I think Apple's been pushing hard for, if you have scrollable content, that can go behind the controls and the Dynamic Island and whatever, because you can always scroll to reveal more, right? I think that's preferable over clipping it or something.

25:27 So I think one of the apps that looks the best, like one of the native apps that looks best on Duo, is the TV app. Because it's a little bit of everything I just mentioned. It's like you have beautifully rich videos and imagery that extends over the safe areas, and then as you scroll, the content is just horizontal scrollable lists that can still go behind the safe area. Just looking at the app, there's nothing that gets contained to the main content area. Everything flows in a way that doesn't feel wrong or awkward.

26:06 And then if you will have to have the main content, something that can't just overflow, I think two things that work, I think, best. One is if you have somehow a clear container that users' eyes can follow and like, OK, my brain understands that this is the content area. So stuff like sheets and cards, I think, work well. And I think with iOS 7 and all the flat stuff, we got away of container and dividers and stuff, everything flows, and I think here that approach doesn't work as well.

26:45 And third, left align your text, that is gonna be easier because then you don't have to make that decision of where do I center with. If it's left align, luckily the left edge of the screen is always the same. We don't have weird notches in there. I mean, don't give Apple any ideas.

27:10 But so... one of my silly side project apps I've done, it had a very, very simple UI where one element, like a crystal in the middle, and then I had a little pill centered at the bottom and a left and right arrow, right? On a normal iPhone, that's all centered because it makes sense, everything's centered. But on the Duo, I found that I could not make that work, no matter how much I try, like, should I align this with a container or not? So I ended up just left aligning, which looks odd. So instead of being left arrow, label, right arrow, I put label and then left and right arrow.

27:48 David: Sure. Yeah, yeah.

27:49 Rafa: Left aligned. It doesn't look great, but I think in that context, it was the best decision. But yeah, it's hard.

27:57 It's hard. I have been rethinking some UI approaches to include more imagery or something that could extend or lean more into horizontal scroll views or something.

28:12 David: Yeah.

28:13 Rafa: Yeah.

28:14 David: Because I think there is definitely something that, in my mind, as I've been trying to think through this, is that there's different categories of I don't know, UI elements, content, whatever you want, and they fall... how you deal with them is different. Where there are some... the easiest are the things that are sort of inherently expandable. So in your example, photos, maps, things like that, which are intrinsically sort of infinitely scalable. You can just fill the space with that, and it fills in a way that is sort of logical.

28:46 And with a map, you have a little bit of the tricky question of where should you center it in? So in Check the Weather, we have... the selected location is what gets centered. And so we have a little bit of a decision: is it actually centered for the whole screen or is it centered in the non-safe area screen? But overall it'll be fine because either one of those is visually probably OK because the content spills out fully. And so those are easiest ones to do.

29:12 And in some ways, that was the seed of the idea for Check the Weather was this UI that was like, how can I make an app that has this sense of expandability that can look good in every single pose on a Duo? And that's right, and it's like, well, if you had a weather app that has a really good radar map, that actually works really well rather than trying to have a UI that has to keep shifting and changing, the sheet itself that shows the forecast is more static. And so you have those kind of contents.

29:43 Then you have content that is better if you can change its size. So as you open the phone, for example, this is where a lot of this is coming in, where some things naturally are better being larger. So most scrollable content works really well in this. So in the Check the Weather case, you can scroll horizontally to see your hourly and daily forecasts. So making that wider, or in Pedometer++, where you show the step count graphs, that graph works. If, rather than showing a week, it shows two weeks, great. There's a logical way to expand and grow that.

30:19 And so those things sort of work naturally, but where things get weird is there's definitely also a third category of things which have kind of a natural maximum size where it starts falling apart. Our settings screens were a nightmare for this, where some settings you can sort of naturally make bigger and smaller. So we have a list of alternative icons you can choose between, and that's a grid. So that grid can naturally, on a narrow screen, maybe it shows four wide, on a wide screen it shows eight wide.

30:51 But there's also things where it's just a switch, it has a label and a switch, and there's a certain width where that becomes comedic. If you open your iPhone Duo and the label is all the way on the left and the switch is all the way on the right, it would just look silly. It would just be completely sort of pointless there. And so there you have these weird sort of... you have to create kind of arbitrary caps on things.

31:13 And so we ended up sort of... on some of those screens, we're kind of pushing the content a bit towards the middle just to make it not look strange in that context. But those are the ones that I found really hard. And I think to your point, it's like, ideally, in a lot of ways, if you can make those in a container, you can push them in slightly, but visually still kind of make sense, that's where the best thing is.

31:36 But ideally, you'll be in the first two categories where it's something that either is sort of intrinsically infinitely scalable or has a sort of natural expansion mode. But if yours doesn't, you're just kind of stuck and you end up with these like, well, we'll kind of center it in, or maybe we'll put it in a box. And then maybe if they fold it halfway, it shifts over to the side and goes half width. That's a thing you can sort of do.

32:00 But that third category, I... yeah, I got really frustrated with in several of our screens where it's just like there is no naturally wider version of this because it needs to sort of naturally go from the outside screen to the inside screen. You want it to feel reasonable that there's a continuity between those two experiences, and some things just... there is no logical way to go from sort of half width to full width.

32:26 Rafa: In a way, it's a similar challenge back when, if you were trying to port your iPhone app to an iPad, like, OK, now we have a lot more space. What makes sense here? But except here, first, we're not making the decision to port this app to an iPad. This is like, no, this is your iPhone app.

32:48 David: Yeah, yeah.

32:49 Rafa: It looks like this now. Do something, please. So you're forced into it. And it's exactly what he said. You have more space. Well, again, now when the Duo is open, it feels like it's a whole different conversation because it's a whole different device. But no, same thing. OK, but now it's open in, let's say, this landscape orientation. OK, now I have a lot of horizontal space. How can I make use of this space in an interesting and useful way to my app?

33:20 And sometimes, moving stuff around, showing more things, right? Or whatever. But in the case of, like your point, settings, there's nothing else. I have a list. What else can I show here to make use of the space? And like, well, I'll try to make what Apple's own Settings app makes, like a two-pane... navigation views. But that doesn't... if your app is a simple app, I don't have multiple levels of navigation in settings. Maybe to your point, I have three toggles and a thing. I don't know. What can... I can't force multiple levels. Like, I'll add a child navigation to this. Does that make sense?

34:07 David: Yeah, yeah.

34:08 Rafa: Sometimes it just doesn't make sense, it doesn't work that way. So I think it just looks odd right now, and I'll think about it later. I don't know. Yeah.

34:16 David: And I think to that point, it's the problem of some of the settings, even if you could make it work in a split view with that, where you have categories on the left and then sort of children on the side. It's like, well, what happens if you're navigating inside of that and then you fold your phone, you close the phone? And then you have this weird question of, should you be showing the child you were in? Should you be going back to the top? Should it now not have the concept of sort of parent and child, and now just be one consolidated vertical list, which is sort of what we've previously been doing, where a lot of those UIs would be just one continuous list? But that feels strange, or you could lose state, or if you're halfway through changing something, you definitely don't want to lose user data when they do that.

35:03 So you don't want to be doing some kind of thing where if the user's typing some value in and then you change the... it's like, oh, they're typing it in in one place, and they're like, "Oh, actually, I prefer the keyboard on the outside because it's more of a traditional keyboard. Great, let me close it." Well, I need to ideally keep your insertion point where it was in that view, and that's really hard if you're gonna have any amount of sort of dynamism within that UI. And so it's just really awkward.

35:34 And I think, yeah, you end up in these places where I think so often, at least right now, I think my solution to it, the best that we have is to do the sort of straightforward thing, rather than trying to get too clever, trying to be too sort of smart with a lot of the stuff, something that is generally straightforward and kind of works seems to be the best way to do it.

35:58 So a setting screen is a great example. In Pedometer++, what we ended up in is sort of a slightly inset from the sides version of the current settings, because it just didn't make sense to try and partition it. And then that can collapse if you fold the phone in half, sort of partway, like book fold, it'll shift over to the right side. And if you close it all the way, the same thing, it's on the right side equivalent of the phone, and that seems to work very... it isn't beautiful, but it is workable, and I think that is a lot of places where that's the best you can do.

36:30 Rafa: That's what I'm striving for. Could this just work? Yeah, you mentioned the book pose. That's another one that, as soon as I got my hands on a device and I tried it, immediately I was like, "Yeah, no one's gonna use this." I don't think it's worth optimizing any UIs for it. Unless the UI is about that. I don't know. But yeah, that was clear.

36:56 Because even when they first announced the phone and I saw, I think they used it as a marketing shot, like a book, so you would have, yeah, it's like a book, two pages. And I even thought, this feels like a nice way to read, say in bed, and it feels like it's easier to hold the phone. It's nice. And then I tried that and immediately, no, I don't know. I can't have it dead center in front of me, because then both screens are at an angle. It's not comfortable, pleasant to look at a screen at an angle.

37:28 So am I just gonna move the phone so that one of the screens is facing me straight on and the other one isn't? So am I just... anyway. Clearly, this is nonsense. No one's gonna do this. But for a while that was something we were trying to... and we still do, certain screens we try to adapt, "All right, if you do put the phone in that pose, let's try not to have any important tap targets in the fold and the crease." But other than that, yeah, don't spend too much time on it.

37:56 David: Yeah, no, I think it feels like one of these things where, yeah, it is weird that I think Apple, in their guidance, there's a lot of both at a technical level as well as in a guidance perspective, talking about that position where things go into the middle and then there's this avoidance area where they're telling you to move content out of it and shift things around.

38:18 And in practice, since I've... it's not like I've had mine for a huge amount of time, but it's nearly a week, I never... that is never how I want to hold my phone. It's only ever in that state halfway between being open or closed.

38:32 And I think, weirdly, the only reason that particular pose is, I think, going to come up in a lot of sort of places is it's really good for screenshots, because it gives you a sense that it is a folding phone and not just an iPad. If you just take a regular screenshot and put that somewhere, it just looks like an iPad. It doesn't have the sense of foldiness to it that in some ways you're trying to communicate. So in a marketing image, it works really well. And I think that may be why in a lot of Apple's marketing, because they were trying to emphasize the fact that it does fold, they show it in that way.

39:08 But yeah, in practice, it just doesn't seem useful. If I'm trying to actually read, what I found, the best... it's a great device for reading, but portrait is how you... at least for me, I find I actually want to hold it. I'll hold it, open it up, put it in portrait, and then it's just a nice... like, it's a hardback book. It's a good size, it's really comfortable in your hand. That actually is how I would read, in terms of the... I'm not gonna pretend that it's a book that I'm folding in half, where that just doesn't actually make sense in practice.

39:43 And so I think, yeah, broadly the kind of valley avoidance stuff that is built in, great. But I think there were some terrible UIs we tried, where in the pedometer graph, where we have a horizontal bar graph that scrolls, I was like, "Well, should the content avoid that middle thing because it says avoid?"

40:04 Rafa: Oh yeah.

40:05 David: And so I had a version of it where as you scrolled, the bars would jump across the chasm essentially as you go. And it was wild, it was comedic, but it was not good. And I think you very quickly get the sense of, yeah, don't go out of your way to do stuff there, especially for that kind of pose.

40:25 Maybe a little bit the laptop pose, where you can have that sense of there's something on the bottom and something on the top, and usually it's gonna be some form of... buttons work better on the down screen where you're pushing, whereas buttons on the top aren't gonna work quite as well. So you may need to adjust your UI, because tapping the top screen, you're just gonna knock your laptop over essentially. And so better to keep the... if you have a button, push it down into the content.

40:51 But yeah, in some ways, that was one little thing that was nice, that we'd just be able to say, "You know what? I don't think that's gonna be as important. I don't think that's something that people are gonna deal with as often." And so in the eight different poses we're trying to optimize for, we can push that one down very low on the list and not give it as much weight. So that if we can do something cool there, great, awesome. But if whatever the thing that we have that is better for the other poses doesn't work as well there, that's totally fine.

41:24 I think I feel comfortable with that so far, like a week into having my own device, that that's just not really what I want. And if anything, there's a weird behavior where if someone is folding their phone, you don't want to necessarily also have this weird sort of... have things move into sort of in this intermediate state. In some ways it feels better if it just is static as it's closing rather than trying to shift itself around it.

41:51 Rafa: Apple's... it's been pretty good at that, knowing if you're closing or if you're actually adjusting the angle. Because if your Duo is open and you just start closing, I think they kind of measure the velocity of it, but instead of having the UI adapt to it, you just see that animation, the whole open and close animation they do over the UI. So if you want your app to actually adjust, you kind of have to open or close slowly. It's... yeah. I know because I've been doing the silly accordion thing, and I know that if I do it too fast, the UI actually won't adapt. You get the Apple's own animation. So yeah, I don't know. Do you also find that this is so buggy sometimes? With the closing in?

42:38 David: Yeah, and I think it's the natural state of, I think, any system that involves thresholds and kind of heuristics is naturally going to have those kinds of things, where there are times where it has to interpret what you're doing, but without necessarily... this is the classic thing of, as soon as you add a double tap to a button, now the button can feel slow because it has to wait to see if the second tap's coming, kind of thing. And I feel like there's a lot of similar things to what you're saying there, of there's the heuristic of when does it think you're actually folding or fully folding, or where it can bring in that cool frosted glass effect that it does between the folds, but then it has to back it out quickly because it thought you were closing, but you actually weren't closing. And so it'll just come in for a second and then jump back.

43:31 Which, to some degree, there's only so much you can do, for the same reason that in the double tap button problem, you can't solve that problem necessarily, because you have to wait however long it would be until you know if there's a second tap before you give the first tap event, just because that's inherent to that problem. And so there's definitely things that I think they'll refine there, but I think some of those problems are just unsolvable.

43:56 Rafa: And they've done the great job, I think, for a 1.0. But yeah, I was on a FaceTime call with my mom, and she wanted to talk to the kids and stuff. And then I tried to open it partially just to prop it up. And the phone was like, "Oh, you want to go on the inside screen?" No, no, no. And I tried to do it slowly, slowly. No, no, no, I'm still on that outer screen. I just... and then switch screens, and then you got the thing on the swipe to change cameras or whatever.

44:27 David: Yeah.

44:28 Rafa: There's this... and I did that. But that was buggy. Anyway, I was fiddling with this phone, trying to... "No, no, go to this camera," trying to tell the phone what I wanted. That was the buggiest experience, I think, with this phone, just FaceTime and cameras and stuff. Because there's, what, three cameras on this thing, and not three different lenses, there's three places where the cameras are. So yeah, it's a hard challenge for them.

44:56 And I've been thinking a lot about how the experience of developers trying to adopt to this. And you mentioned at the top of this conversation, I would maybe work on a UI and then you would see it and you were like, "Yeah, this is better." Even though you worked on it... I think all of us designers, or if you don't consider yourself a designer, developers... I always struggle separating these two things. Anyway, but I think we all share that final destination of, I can look at something and tell you if it's good or not.

45:41 So even if you have incredible taste or you consider yourself, "I don't know any design stuff. I'm not good at this, I don't know anything," I think you still can look at something and appreciate, this feels nice or not. The difference is then knowing what makes it good and how can I make it good.

46:05 So I worked on a UI, we both looked at it, I guess, and then we were both like, "Yeah, this is good, this is better." But maybe, because I'm a designer by trade, I know how to get it there, and I can see the end result before getting there. You know what I mean? I can see it and I can work towards it. And maybe sometimes, if you're not a designer, you cannot see it. You don't know exactly where to aim, but if you see it, you know, "Oh yeah, this is it."

46:42 And so usually that's the main difference between someone who's been working as a designer and someone who hasn't. It's like I kind of know what to tweak and know what kind of buttons to push, and maybe you don't. But hopefully we'll both be very happy at the end.

47:00 With this phone, I feel like I'm in the place of a non-designer, where hopefully if I see the end result, I can say if it feels good or not. But right now, oftentimes, I don't know where to go. I don't know what buttons to push. So there's a lot of experimentation. What about this? What about that? I guess better in this way, but worse in that other way. It's a lot of trying things and just trust your own judgment, trust your own feels and vibes, is this better or not?

47:41 And I haven't... it's very rare that I see a design, a UI, I don't know, pattern or something that I'm like, "Yep, this is better in every way. This is the correct answer."

47:54 David: Yeah.

47:55 Rafa: There's always trade-offs, "OK, yeah, I see it, it's better maybe in this context, but it wouldn't work in my context," or whatever. And luckily we're also in a phase, in a time and place where with LLMs, it's very easy to experiment. So luckily we can quickly see different layouts, see different designs, and then make the judgment call there.

48:20 But yeah, I feel very... well, not powerless. I just feel like you stripped out all the tools that I have. I don't have my tool belts. Yeah.

48:32 David: Yeah.

48:32 Rafa: So I have to work in this thing without any of my tools that I've been crafting and practicing with for years and years. So I feel like a novice again, like an amateur.

48:42 David: Sure.

48:43 Rafa: Which is kind of fun.

48:44 David: Yeah, yeah, yeah.

48:45 Rafa: But yeah, I think if someone listening to this, and maybe you're struggling because you don't really know how to adapt things, and then usually we just look at Apple's apps, and that's a great starting point. But even Apple's apps are like, well, depends on the app. And they tried a bunch of stuff.

49:03 And I think if you were to ask Apple's designers, "Hey, is this good? Are you happy with where we are right now?" I would guess they were like, "Well, no, I mean, we did our best, but we're also learning, right?" I think we're all learning. So yeah, don't feel discouraged, or don't feel terrible about your own sense of design skills because, hey, even me as a designer, I feel like my design skills are not helping me right now.

49:31 David: And in some ways, what you said there, it almost gave me this feeling of, I think there is... it's a different... in the fancy, designy art word, it's a different medium. In things you've learned from other mediums, like if you were a painter and now you're trying to do sculpture, or you were a sculptor and now you're trying to do painting, there are parts of that that I think are sort of transferable, that there are certain universal things that are always going to be sort of part of that, but there's also an element of, the medium is different, and if the medium changes, then you're inherently going to have to apply those skills differently and learn and pick up a bunch of new skills that you just didn't have otherwise.

50:14 And that's very much how I feel coming into this. I generally would consider myself a developer, not a designer, just because that's not my background. My background was computer science and coding and things like that. And I've learned design the hard way of sort of picking it up, but it is very much something where, in some ways, I enjoy that the Duo... It

50:36 David: I don't feel as out of my depth because I think everyone's out of their depth. And so we're all kind of struggling through this and kind of working our way there collectively and together. So there's a little bit of comfort in that. But I appreciate the reassurance that, if you feel a bit lost on this, that is a natural feeling rather than an impossible feeling or something that you should feel a bit weird about.

51:02 Whereas on a traditional iPhone development, the rules are more established. If you wanted to learn how to do it well, there's so many great examples that you can imitate and copy and get good at by looking at. There's designers who have made the platonic ideal of almost every general style of app at this point. And so optimizing and trying to copy one of those, I think, is so much more of a viable and reasonable. Whereas here it's completely open. It's this whole new medium that no one really knows what they're doing.

51:34 Rafa: That reminds me of...I used to play music in a band and stuff, and I played bass mostly. And I know how to play guitar and stuff, and I'll just acoustic guitar, I'll put some chords, it's fine. But recently I've realized that actually I want to actually learn how to be a good guitar player.

51:57 So my thinking was like, now with agentic coding, there's a lot of dead spots. There's a lot of five minutes here, eight minutes here, 10 minutes here, where you're waiting for a prompt. And me, and I assume a lot of us in the industry, the way we've been trying to fill in these spots is just by working on something else. And so I end up finding myself working on three, four different things at the same time. And that is just...my brain is not keeping up, it's just mushy all the time.

52:29 So I was like, these five minutes here and there, I'll just practice guitar. Instead of working on four things at the same time, I'll work on two. And then with those five minutes here and there, I'll just practice a craft, an instrument.

52:42 But that was the thing of, I know how to play an instrument really well. Some of that knowledge transfers over to a different instrument, like chords, notes. There's knowledge, but then a lot of it also doesn't transfer, right? If I'm a guitar player, I can't pick up a saxophone and play. So that feels similar to this.

53:07 Yes, if you're a designer, if you've been working on this a long time on iPhone, Apple platforms, there's knowledge that is very much transferable, but some of it, it's not. And it's like a brand new thing, a brand new instrument that now you have to learn how to play. Yeah.

53:25 David: I think that's an excellent analogy, actually. In the sense of both are playing music, and some of the things apply the same in both ways. If you learned to read music and you're changing instrument, well, the staff and where the notes go and what the different symbols mean, those things carry through, but being good at playing a new instrument is gonna be different.

53:50 It's not this natural, like, oh well, because you could play the flute, you could absolutely play the bassoon. Like, no, it just isn't the way this works. Even though the music's the same, you know exactly what the note is supposed to be, you have to actually sort of get good at that.

54:05 And I think it's probably a good place to just sort of wind the thought down. It's the sense of, yeah, this is the messy part, and I think it is going to be messy for a while.

54:18 And I think that's, in some ways, I find very reassuring that it isn't just...as someone who has a lot of the imposter syndrome about the being not a designer kind of feeling, but it's nice in some ways for this device where it's like, I'm just enjoying having fun, exploring stuff, trying things out, seeing how they work, and the messiness is inherent rather than a sign of a deficiency or a problem or something that...every app I've looked at, from our apps, who were just designed by one of my favorite designers, to all the apps, Apple's apps, every app, there's always these trade-offs and these weird things that we're all having to kind of wrestle with.

55:03 And I enjoy the space for that. It feels like we're very much in the practice room, if we're going to extend the music metaphor still. And we haven't gotten to something that is like the symphony that you can perform yet. And I don't know when that will be or if that's even possible.

55:20 Some of the things that we've talked about feel somewhat intractable, but they are inherent to trying to have a device that is like five different things at once, that you're never gonna have something that is perfect in all five. But we may just find that in the end, the perfect way to do that is you just don't.

55:41 That in the end, people will use this device in like two different modes, and a lot of them will just fall away. Like we were saying, the book fold, the tent mode, the standing up book mode, maybe those just really aren't a thing in practice. That in practice, people use the outside screen and landscape open. And that's a much easier problem to solve than trying to have something that is expansive and works everywhere.

56:04 But this messy sort of initial point, it is exciting in a way, but it is also...I appreciate the reassurance that the sort of slight incompetence I feel on this device is natural, it is to be expected, and it's not something that I think is gonna go away quickly. But it is interesting because, like with so many of these things, like you were saying with the Vision Pro, you could just avoid it if you didn't want to touch it, or if it's an iPad, you just didn't want to, you didn't have to deal with it.

56:39 But this is an iPhone. The iPhone is always going to be a platform that we have to take care of if you're an Apple developer. It is so big, it is gonna be the crux.

56:52 And I have no idea how successful this device will ultimately be in the market, whether it will be like the iPhone minis or something like that, where it had a very small dedicated group of people, or the iPhone Air, even from last year, where there are people who love and adore their iPhone Airs, but there are very few of them in proportion. And so you could kind of get away with...not like ignoring in a mean way, but it was not a thing that you had to really consider.

57:19 And I don't know if the Duo will take that path and it ends up at just like 1 or 2%, maybe, of total users. And so if you had a reasonable experience, like just sort of the build and run, it kind of does whatever UIKit or SwiftUI want to do naturally. Maybe that would be reasonable.

57:39 But I think there's definitely going to be opportunities to do things that are cool and interesting. And I think I enjoy that opportunity. It's fun. Part of the things I enjoy about being an app developer is thinking of new and interesting things that isn't just dealing with solved problems where I know exactly how to display certain kinds of displays. Like we're saying, I know exactly what that should look like. And so now it's kind of fun to be on a platform where it's like no one knows, and we're all finding it out together. And I think we'll have to be patient with that.

58:13 But I think it's also maybe a thing that I'm trying to do is just look what other people do, whose design sense I admire, and trying out lots of different apps. And that reminds me of the old days of, as someone who's been making apps on the App Store since the very beginning, I used to do exactly the same thing. I would just go and look for inspiration, kind of pick up the little tips and tricks, and like, ooh, I like the way they did that thing there.

58:37 And it may not be that their whole app structure works in one of our apps, but it could be like, ooh, that little bit, that little nudge or item or element, that could work well. And that's just an exciting place to be in this whole thing.

58:52 Rafa: Yeah. It is a source of creativity for a lot of developers right now, because you can do things that you couldn't before. And even you have new sensors and APIs, or actually not sensors, but new APIs. You have the hinge stuff, right? And we haven't seen that in a while because the iPhone has been so mature.

59:12 And even, yes, iPhone minis and iPhone Airs, they're nice because they were slightly different than the main iPhone, but iOS behaved exactly the same. It's not like, oh, now I can make an app for iPhone Air that I couldn't before. That is not the case, even though maybe market share might be a similar device or story, in the sense that it is a niche. Well, I think we'll still have to see how it ends up, because right now people want the phone, still can't get the phone, so we'll see when that evens out.

59:45 But there's that comparison of, yes, it might be a niche iPhone and not the main thing you should focus on, but in terms of platform and device, it is a whole new beast that can create. We have seen a lot of the new novelty apps that developers are making for this phone that would not be possible on any other iPhone or iPad or any other device, really. So it's been fun to see, and yes, it does remind me of early App Store days.

60:15 And pull to refresh, it's always the go-to, look at this innovation and UI from a 3rd party developer that now became a standard. That was what, Tweetie 2?

60:25 David: Yeah.

60:26 Rafa: And I forget how many years that was into the App Store, but maybe two years? It was still early.

60:32 David: It was early, but still, it was a few years into it before.

60:35 Rafa: I think it was like iOS 6 or 5 that introduced that in Mail, but Tweetie came out before. So anyway, even if it's like three years, which would put it at iPhone 4, I think. Even if it was three years, that's nothing. That's nothing.

60:55 David: Yeah, we're four days in, not four years into this. So we've got a long way to go.

61:00 Rafa: Yeah, but these following years are, if you like this stuff, if you like software, if you like apps, if you like these devices, it's fun, to your point. It's fun and it'll be a creative period. And because there's no solution to these problems, we have to come up with solutions.

61:18 Apple probably's gonna do the heavy lifting here because it's their responsibility in a way, but hey, we can come up with the next pull to refresh that only makes sense on foldables or something. So that's exciting for me at least. That's fun.

61:32 David: Absolutely. Well, thank you so much for chatting to me about this. And yeah, it's a fun time, and I look forward to exploring it with you as we kind of keep adapting our apps and kind of seeing where this goes. It'll be a wild ride, I suspect.