It's an interesting question. Where do you think the line should be?
I think I understand the deeper point you're trying to make, which is that not all web platform capabilities are created equal when it comes to making PWAs a legitimate alternative to native apps.
I've tried to be thoughtful about this, and my solution was to weight the capabilities from 0.5–3. At the two ends of that range, I gave required PWA features a weight of 3, and nice-to-have/experimental features a weight of 0.5. As you can see, this doesn't really penalize Chrome for its "stunt features", and I think it provides a more realistic picture of what Firefox and Safari are capable of for most PWA use cases.
Additionally, experimental and non-standard features aren't used to compute the PWAscore. You can see raw scores (unweighted, and including experimental/non-standard features) in the tooltip of the "normal" score.
Honestly, I have no idea. Maybe categorizing things into MVP, standard, and advanced would be a good start.
When I think of most apps on my phone, these capabilities are rare or non-existent:
- USB
- AV/VR
- Payment processing or IAP (almost certainly my own tastes here tho)
- Window management(?) - I'm not even sure what this means on mobile.
But obviously, whatever deficiencies exist in background work or notifications are probably important. Looking at CIU, Firefox seems to have resolved them in more recent versions. In fact maybe some other comment has already mentioned it but FF/Android 141+ seems to have resolved a lot of those?
Confirmed that it now works for me. Thanks! (I'm sort of using it as a "what can PWA's do" study guide for My First Android PWA, so the section organization is almost as big a deal as the "does it even work" indicators - seeing "App Icon Badges: unsupported" does tell me that I don't need to bother figuring out what those even are, I can move on to something else.)
I like the idea of actually testing for the capabilities. Can you share more about what you'd find useful? Maybe a "benchmark my browser" feature that adds a column?
The data sources are credited at the bottom, and I feel like they're both gold-standard references. I'm open to suggestions!
The caniuse test suite is the most basic tests of each feature. Self admittedly it does not actually verify the feature meets any spec.
If you just did what they have but make it actually test to the spec, it’d be great.
“Caniuse” is not gold standard. It’s basically “Did the browser vendor claim to support a feature”.
iOS safari has so many bugs throughout everything on the web, if you actually have them red for every feature with a bad bug, they’d have so much worse of a score.
Eg. Does safari support audio element? Yes on the surface. But try to play from a cached blob. It’s hilariously broken.
See my comment history, there’s an app with a LONG list of safari media bugs. Yet your page lists them with a green check. Lies.
I’m not surprised if nearly every feature there “works” but try to build an app with it and you find yourself set up to fail.
Remove all the non-standard stuff. If only Blink supports something because no other rendering engine thinks it should be implemented, then it doesn’t belong here. It’s non-standard, vendor-specific behaviour that you’re making people think is standard.
I can create a filter for that, thanks for the feedback! Importantly, experimental and non-standard features are not counted in the primary score.
Experimental capabilities have a little 'flask' icon next to them, and non-standard items have a little 'warning' icon next to them, mirroring MDN conventions.
> Experimental capabilities have a little 'flask' icon next to them, and non-standard items have a little 'warning' icon next to them, mirroring MDN conventions.
This is not true. I haven’t gone through each item in detail, but there are no icons for Web Bluetooth, Web NFC, or Web USB. All of these are non-standard, Blink-only APIs that no other rendering engine has agreed to implement.
Mozilla on Web Bluetooth:
> This API provides access to the Generic Attribute Profile (GATT) of Bluetooth, which is not the lowest level of access that the specifications allow, but its generic nature makes it impossible to clearly evaluate. Like WebUSB there is significant uncertainty regarding how well prepared devices are to receive requests from arbitrary sites. The generic nature of the API means that this risk is difficult to manage. The Web Bluetooth CG has opted to only rely on user consent, which we believe is not sufficient protection. This proposal also uses a blocklist, which will require constant and active maintenance so that vulnerable devices aren't exploited. This model is unsustainable and presents a significant risk to users and their devices.
> We believe Web NFC poses risks to users security and privacy because of the wide range of functionality of the existing NFC devices on which it would be supported, because there is no system for ensuring that private information is not accidentally exposed other than relying on user consent, and because of the difficulty of meaningfully asking the user for permission to share or write data when the browser cannot explain to the user what is being shared or written.
> Because many USB devices are not designed to handle potentially-malicious interactions over the USB protocols and because those devices can have significant effects on the computer they're connected to, we believe that the security risks of exposing USB devices to the Web are too broad to risk exposing users to them or to explain properly to end users to obtain meaningful informed consent. It also poses risks that sites could use USB device identity or data stored on USB devices as tracking identifiers.
Did you manually change the status of those particular things? Because that’s not what I was getting at.
All I did was look at the most common things I know off the top of my head are mistaken for web standards when they are not. I did not review anything else. So if you just changed the status of those things, there’s more fixes you need to make.
For instance, I just took another look, and I saw that BackgroundSync is not marked as non-standard. But Mozilla are negative on it:
> We're concerned that this feature would allow users to be tracked across networks (leaking private information about location and IP address and how they change over time), and that it would allow script execution and resource consumption when it isn't clear to the user that they're interacting with the site. We might reconsider this position given evidence that these concerns can be safely addressed.
And Apple have concerns and have not committed to it either:
> Adding concerns: power as generally having activity running in the background is not great for battery life. In my experience as a user of "native" apps this can also be tricky to get right. Sometimes background syncing would have taken place, but the moment you open the app it does another sync and blows away the content you just started to read.
> Adding concerns: privacy as this would allow a website to monitor you switching IP addresses. As the draft notes there also some network attacker leaks due to this, revealing origins that you visit that use this functionality to them.
So again, this seems to be a Blink-only API that only Google have implemented, not standard functionality. How many of these fails are actually just Google doing their own thing with no buy-in from other rendering engines? “PWAscore” is a reasonable thing to build, but “Chromescore” where you measure against all the non-standard things Google have implemented by themselves is not a reasonable thing to build.
Also, you have “Not supported” for push notifications in Safari, but if you check Can I Use, you will see that it works for installed PWAs and is only unavailable to websites you visit. Since the purpose of your site is to score based on PWA support, this should be a success, not a fail.
Personally, I would remove all the non-standard stuff. Not give a big list of standard and non-standard stuff mixed together.
> Did you manually change the status of those particular things?
No, for the specific issue you mentioned, I found and fixed a logic bug. There are some features/capabilities which are manually assigned in cases where there's no CanIUse or MDN data. (Generally, I'm trying to minimize additions/overrides for maintainability's sake.)
> For instance, I just took another look, and I saw that BackgroundSync is not marked as non-standard. But Mozilla are negative on it: […] And Apple have concerns and have not committed to it either:
This is a very fair point, and again I think you for being very generous with your time and expertise. I've reported this as a bug in the data (https://github.com/mdn/content/issues/41455), and manually overridden the data for now as a workaround.
> Personally, I would remove all the non-standard stuff. Not give a big list of standard and non-standard stuff mixed together.
I've added a "Hide Experimental" checkbox, which can also be set/shared in a URL property. I'm not yet sure about making that the default, but I hope it's useful!
Ah, I was curious about this too... but now it says that none of them support it. Surely at least one has to support it or else it wouldn't be a feature right? MDN says Chrome only...
Not sure what the deal is with Chrome and tabbed mode, it has had a very long development cycle and it has to be turned on with experimental flag, and that keeps getting broken.
Firefox has the same number of "Unknown" capabilities as Chrome and Safari. These are items which aren't tracked in Can I Use or MDN, and I intend to resolve those ASAP. (I appreciate you calling me out on that.)
I think I understand the deeper point you're trying to make, which is that not all web platform capabilities are created equal when it comes to making PWAs a legitimate alternative to native apps.
I've tried to be thoughtful about this, and my solution was to weight the capabilities from 0.5–3. At the two ends of that range, I gave required PWA features a weight of 3, and nice-to-have/experimental features a weight of 0.5. As you can see, this doesn't really penalize Chrome for its "stunt features", and I think it provides a more realistic picture of what Firefox and Safari are capable of for most PWA use cases.
Additionally, experimental and non-standard features aren't used to compute the PWAscore. You can see raw scores (unweighted, and including experimental/non-standard features) in the tooltip of the "normal" score.
reply